GoogleBigQuery reverse engineering not working

[david-u]

Hi,

I have just installed DBSchema (8.2.7) on my mac. I was able to connect to Google Big Query using a Service Account (I had to play a bit to change/select the right jar files for the driver to work).

Once unticking the "Database uses Default Schema", I can now see the list of Project.DataSets. Unfortunately there is now way to retreive the list of tables/views.

This is a recent addition to BigQuery so I understand it's not yet integrated in DBSchema. Is there a way for me to tweak the commands used to retreive tables and views?

Here is the google documentation for extracting their metadata: https://cloud.google.com/bigquery/docs/information-schema-intro

Many thanks, David

[wise-coders]

Hello David, Yes, we can fix this. I think we also need the fix you did with the jar files, we got also into issues with them. Could you please send us a zip with the driver files you used (in this issue or per email to support at dbschema.com ) ? This would save us time. We tried also to use the latest driver from Simba, but this is crashing Java for some reason.

[david-u]

Hi,

I downloaded the latest JDBC driver releases (1.2.2.1004) SimbaJDBCDriverforGoogleBigQuery42_1.2.2.1004 from https://cloud.google.com/bigquery/providers/simba-drivers

I had to do a lot for try-and-error to find out a combination of jars that would allow a connection to our big query. I was able to connect with our Service Account and could see the list of projects and datasets.

Here is the screenshot showing the list of jar files used.

If these options to determine how to load the catalogue are open to users, please let me know and I am happy to investigate and share info.

Thanks, David

[dprutean]

Now we have fixed DbSchema to be compatible with Google BigQuery. Now is possible to :

1. Reverse engineer the schema
2. Create partition keys, cluster keys Here the download links: https://dbschema.com/download-beta/DbSchema_8_2_8.exe or https://dbschema.com/download-beta/DbSchema_8_2_8.dmg Are still under the 8.2.8 version number, but a higher build number. Please first remove the folder .DbSchema from C:\Users\YourUser. This will delete all local settings and driver. Then switch in the connection dialog forth and back to another database so a fresh driver is downloaded.

If you have the chance to test this, please report any issue. We didn't find and ALTER TABLE ADD COLUMN statement for BigQuery. If you find other statements they are configurable in Project / Settings / SQL Syntax. Please also report them if possible.

[wise-coders]

Hello David, Any chance to try the beta version ?

[david-u]

Hi,

I couldn't download the beta, but the latest built did work to get the table names, fields and data types. For the views unfortunately there is no fields extracted, as it's not yet on Big Query metadata. The one thing that could be added is extracting the SQL code of the views to be extracted into DBSchema.

David

DAVID UREÑA Head of Data Analytics Strategy | Data Organisation David.Urena@Veepee.com %20David.Urena@Veepee.com Street Llull • 08005 • Barcelona [image: veepee] https://www.veepee.com/[image: privalia] https://www.privalia.com/

El sáb., 21 mar. 2020 a las 20:23, Wise Coders (notifications@github.com) escribió:

Hello David, Any chance to try the beta version ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/wise-coders/dbschema/issues/18#issuecomment-602090936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOV2MNBACNF34D6KQQWUTJLRIUH3TANCNFSM4K5MLMKA .

[wise-coders]

Hello David,

Do you know which is the system meta table where we can get the view source code ? We would fix this.

Best regards, Dragos Pruteanu

-- DbSchema - Wise Coders GbmH 82256 Fürstenfeldbruck bei München Email: support@dbschema.com - Web: [1] http://dbschema.com/

--

[1] http://www.example.com/