Google BigQuery
Last updated
Was this helpful?
Last updated
Was this helpful?
is Google's data warehouse that enables scalable analysis over data. This integration allows pushing your specific input properties, or all of them, to BigQuery.
The Google BigQuery destination provides the following key features:
Events structure: our feeds BigQuery, meaning that your data is properly bridged to the expected fields in an optimized way.
Multi schema support: store event data following your preferred/existing schema or we can help you creating an .
Data control: select your properties or just check a box to send them all to BigQuery.
This destination accommodates all schemas, more specifically:
your existing table schema or,
a universal schema.
The first option is useful for those who want to use an existing table with its predefined columns: this is a common scenario when you have data already stored and is the recommended option as it's fast to configure and data can be accessed directly from the specific column. To enable this mode, in the , you just need to flag the Auto-discover table structure (recommended)
and then proceed to the section Event Property Mapping
to select the columns in BigQuery column
and their values in Your value
.
The second method relies on a , where all your data (or selected properties) is stored in just one column as a JSON string and using the BigQuery function you can retrieve specific values within. We suggest checking the section with a detailed walkthrough on how you can create the universal schema and understand if this is the proper table structure for you.
To enable this mode, in the , you can either flag Send all properties to BigQuery with universal schema
so all properties will be included or you can input the properties you want to send by using the Property name
table in the section Properties to include with universal schema
.
Authentication
Required
Your credentials with Google as set in the Commanders Act interface following: Administration
➜ Connector Credentials
➜ Add connector credentials
➜ BigQuery
Project Id
Dataset Id
Table Id
Auto-discover table structure (recommended)
Check this option to enable the auto-discover table structure feature.
Send all properties to BigQuery with
universal schema
Event property Mapping
When Auto-discover table structure
is flagged, you can map your BigQuery fields by selecting them in the BigQuery column
and set their values in Your value
.
Property name
[1]
When both Auto-discover table structure (recommended)
and Send all properties to BigQuery with universal schema
are not flagged, you can input the properties you want to include in your table with universal schema, one per line.
When flagging Send all properties to BigQuery
a specific schema is required. See the following subsections to learn how you can create an universal schema. This is not
Input a (4)
dataset identifier (E.g. "myDatasetId"), select a (5)
location type and click (6)
CREATE DATASET
.
Create a table with the following structure:
rawDataCa
String
Required
createdAt
Timestamp
Required
The esiest way to create it is to click (7)
the plus
button:
copy and paste the following query in (8)
the input area:
and then click the (9)
RUN
button.
[Any events]
[1]
rawDataCa
, createdAt
[2]
Required
Select your project identifier from the dropdown menu as reported in BigQuery console. More details are available following this .
Required
Select your dataset identifier as reported in BigQuery console. More details are available following this .
Required
Select table identifier as reported in BigQuery console. More details are available following this .
Flag this option to send all properties to BigQuery following a specific schema: see section for more details.
Access to locate your (1)
project identifier and click (2)
the three dots
on the right. Select (3)
Create dataset
from the menu or, alternatively, you can use an existing dataset and jump to the .
[1] Use to specify your matching events.
[2] Two columns: rawDataCa
contains your event properties, while createdAt
is the creation timestamp.