Using Transforms

The pipeline in White Label Data allows for transforming query results using built-in transform operations. See the Layer Pipeline documentation for an example of adding transforms to the pipeline.

Note: It is also possible to transform shared query results using SQL transforms. See Shared Queries.

The following transforms are supported. Each transform operations requires specific attributes, specified below, to be added to the pipeline step in JSON. A transform step takes the following form:

{
    "steps" : [
        {
            "action" : "transform",
            "operation": "operation_name_from_list_below",
            "other_attribute1": "attribute_value",
            "other_attribute2": "attribute_value"
        }
    ]
}

Common Options

Option Value(s) Description
action transform Specifies that it is a BigQuery query.
operation Any of the transform operations listed below Specifies the name of the transform method. For example, divide, multiply, or groupBy.
input_dataframe A string The name of the DataFrame to use as an input for transformations. If no input_dataframe is specified, it uses the last DataFrame from a previous step in the pipeline.
output_dataframe A string The name to use when creating a new DataFrame. This allows you to have multiple DataFrames in the pipeline context and to map columns from multiple queries to a single visualization. If no name is specified, the DataFrame will be named after the query name.
Other Attributes Varies See each section below for additional attributes required for each transform.

Number Transforms

divide

Divides all rows of the input column by the divisor and creates a new output column.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.
divisor The number to divide the column values with.
multiply

Multiplies all rows of the input column by the multiplier and creates a new output column.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.
multiplier The number to multiply the column values by.
convertPercent

Converts a decimal column to a percentage.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.

String Transforms

format

Creates a single column comprised of values from other columns. Useful for formatting HTML columns made up of values from different columns, such as for an HTML tooltip.

Attribute Description
format A templated format string that references one or more column names between {{ and }}.
output_column_name The output column name.

Type Conversions

convertFloatToDollars

Converts a float to rounded dollars.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.
convertUnixTime

Converts an integer column in epoch seconds to a timestamp column

Attribute Description
input_column_name The input column name.
output_column_name The output column name.

Aggregate Transforms

sum

Creates a dataframe where all columns have been summed

Attribute Description
None
groupBy

Groups and aggregates rows using an aggregate function

Attribute Description
aggregate_function One of:
-concat
-average
-min
-max
-sum
-count
by_list List of columns to group by.
concat_separator When concat aggregate function is specified, will concatenate column using this separator.
concat_column The column to concatenate for concat function.

Row and Column Transforms

pivot

Pivots DataFrame to make rows into columns

Attribute Description
index Column to use to make new frame’s index.
pivot_column Column to pivot on.
value_column Column(s) to use for populating new frame’s values.
renameColumn

Renames a column.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.

Normalizing Data

normalize

Normalize values on a range from 0 to 1, as a decimal

Attribute Description
normalize_column_name The column to normalize.
min_val A minimum value in the range, instead of 0.
generateDiscreteIntegers

Assigns a unique integer to every value in a column.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.
mapIso2toIso3

Maps two digit country codes to three digit country codes. Useful for geo-based visualizations.

Attribute Description
input_column_name The input column name.
output_column_name The output column name.

3D Rendering

prepareNodesAndEdges

Creates a 3D network graph model for a set of source-target column relationships. This creates three DataFrames, one for the nodes, one for the edges, and an optional one for the center point on the line to place labels.

Attribute Description
source_column The source column for the relationship.
target_column The target column for the relationship.
nodes_dataframe Name of DataFrame for nodes.
node_key_columnname Column name to use as the key for nodes.
node_x_columnname x-coordinate of Node.
node_y_columnname y-coordinate of Node.
node_z_columnname z-coordinate of Node.
edges_dataframe Name of DataFrame for edges.
edges_x_columnname x-coordinate of Edge.
edges_y_columnname y-coordinate of Edge.
edges_z_columnname z-coordinate of Edge.
center_dataframe Name of DataFrame for center point of edge.
center_x_columnname x-coordinate of center point of Edge.
center_y_columnname x-coordinate of center point of Edge.
center_z_columnname x-coordinate of center point of Edge.

Custom Transformations

custom

Call a custom Python transform function.

Attribute Description
function_name Name of a function specified in pipeline.py in /extensions folder.

See Extending With Python.