[x] For now, copy BigQuery resource for a testing resource. This will be updated with a second resource that will require a full data dump from Firestore around time of transition
[x] Add BigQuery sdk/creds to ESP
[x] Fork report scripts to pull from BigQuery instead of Firestore
[ ] Update demux endpoints to additionally write data to BigQuery - This could actually be the endpoint because users would be able to run them from ESP.
[ ] Create gtac webtool for users to run query outside of ESP that uses report scripts to create downloadable report
Using BigQuery consolidates DB resources without adding another Database app.
BigQuery is a more appropriate resource than a relational DB app.
There is already basic development completed for using BigQuery
This will serve as a backup to FireStore and remove the need to migrate if Firestore is ever discontinued.
Much easier to migrate if BigQuery is ever discontinued.
A complete ingest of everything in FireStore and the addition of new data will provide an easily queryable resource with all historic data (minus Smartsheet for now), this will supplement the QC app gold standard data set. And be a much better alternative to FS.
The foundations of this more basic project will most likely be the basis for the QC app as their is significant overlap which will enable that project to move forward in tandem. And a sooner completion time frame for the QC app as the foundational requirements will be in place
In the event Flowcell query becomes obsolete in the very near future there is an improved resource available.
Reduced run times, no user should be required to wait 10 minutes for a DB query to complete.
Related: https://github.com/GTAC-MGI/GTAC-ESP-LIMS/issues/481