Now that we have an example of how we can create a table in Google BigQuery with a python script, we need to expand the capabilities of this script to pull a table definition from an external JSON file. Doing so will require a few steps:
[x] Create an example table and/or dataset definition in a JSON file, example_table_schema.json. Use this basic format for now (we will be augmenting it with more information in the future):
{
"put_table_name_here": {
"fields":
[
{
"label": "verbose name of field",
"column_name": "field_column_name",
"data_type": "string"
},
etc.
]
}
}
There will be multiple entries in the fields list, one per column in the new table. In the sample file, include at least 3 different fields each with a different datatype (integer, string, boolean, etc.)
[x] Next, store this file in the repo, and in the example script that exists for the table creation, load this file’s data using the json library.
[x] Finally, augment the field/schema creation section of the script so that it uses the loaded JSON data from the file. There should be no hard-coded field or table names in the python script, all of that will be stored in the JSON file.
Now that we have an example of how we can create a table in Google BigQuery with a python script, we need to expand the capabilities of this script to pull a table definition from an external JSON file. Doing so will require a few steps:
[x] Create an example table and/or dataset definition in a JSON file, example_table_schema.json. Use this basic format for now (we will be augmenting it with more information in the future):
There will be multiple entries in the fields list, one per column in the new table. In the sample file, include at least 3 different fields each with a different datatype (integer, string, boolean, etc.)
[x] Next, store this file in the repo, and in the example script that exists for the table creation, load this file’s data using the json library.
[x] Finally, augment the field/schema creation section of the script so that it uses the loaded JSON data from the file. There should be no hard-coded field or table names in the python script, all of that will be stored in the JSON file.