Elimate the reference to authorization in the DDL in the cloud.

DougEbel commented 1 year ago

We need to remove the authorization from the DDL. We should also review some of the cloud tables that have many files. Those may have been written from a different platform so there is 1 file per AMP writing the files which results in poorer performance.

I have made a backup of clearscape_analytics_demo_data to clearscape_analytics_demo_data_bak.

I had to e-mail the spreadsheet index since it can't be pasted to this note.

DougEbel commented 11 months ago

Completed in the pass 4 version

DougEbel commented 9 months ago

This is not complete so we can not remove the "auth" from gs_tables_db. This means that there are still keys around to allow full access to our GCP buckets: example 1 line:

create foreign table gs_tables_db."TRNG_AustinBikeShare_Weather", external security gs_tables_db.auth using (location('/gs/storage.googleapis.com/demonow_development/TRNG-AustinBikeShare/Weather/'));

audit of all DDL

select databasename, count(*) from gs_tables_db.ddl where sql_lines like '%auth%' and sql_lines not like '%public_auth%' order by databasename group by databasename

databasename Count(*) AustinBikeShare_cloud 2 DEMO_CreditCard_cloud 1 DEMO_CreditCard_local 1 DEMO_FoodReviews_cloud 1 DEMO_FoodReviews_local 1 DEMO_Modelops_cloud 3 DEMO_Modelops_local 3 DEMO_ParkinsonsDisease_local 1 DEMO_Sonar_cloud 1 DEMO_Sonar_local 1 TRNG_AustinBikeShare_cloud 2

Teradata / jupyter-demos

Elimate the reference to authorization in the DDL in the cloud. #379