Enable Multiple Data Sources

[SeanLeRoy]

Description This changeset enables users to select more than one data source in a single query. This is largely a proof-of-concept implementation of this feature as I think we could spend more time to make this much more performant like by perhaps creating views that are representations of a query's data sources.

How to review Most of the changes to look for are in the SQLBuilder & logics.tsx

Testing Tested on desktop and in the web adding various types of data source combination and performing queries.

[BrianWhitneyAI]

Seeing this when trying to load in various datasets locally.

fro variance set

:9001/?source=%7B%22…ri%22%3A%7B%7D%7D:1 Uncaught (in promise) Catalog Error: Table with name Variance Paper Dataset does not exist!
Did you mean "pg_database"?
LINE 4:             FROM "Variance Paper Dataset"

And this for the second dataset

localhost/:1 Uncaught (in promise) Catalog Error: Table with name AIND Image Atlas - AIND Image Atlas does not exist!
Did you mean "pg_settings"?
LINE 4:             FROM "AIND Image Atlas - AIND Image Atlas"

[aswallace]

I'm now able to select, filter, group on multiple data sources, both web and desktop. Still having issues with downloading manifests: Web:

• I can download successfully for non AICS sources, but when I try to download from AICS source it claims to download successfully, but then the browser gives me a "File wasn't available on site" error and doesn't actually complete the download

Desktop:

• Still having the header issue with downloading CSVs
• I'm still seeing the error Brian mentioned when it tries to reload an external data source, and the modal + error won't go away even if I select a new file

[SeanLeRoy]

I'm now able to select, filter, group on multiple data sources, both web and desktop. Still having issues with downloading manifests: Web:

• I can download successfully for non AICS sources, but when I try to download from AICS source it claims to download successfully, but then the browser gives me a "File wasn't available on site" error and doesn't actually complete the download This was an issue with the type I was sending to the databaseservice, should work now.

Desktop:

• Still having the header issue with downloading CSVs Added option "HEADER" to CSV exports, should work

• I'm still seeing the error Brian mentioned when it tries to reload an external data source, and the modal + error won't go away even if I select a new file Fixed, the cache of annotations was saving [] as a valid list of annotations, seems to be working now

This should all be fixed now.

[aswallace]

This should all be fixed now.

Confirmed that these fixes all worked for me! Only running into a couple small issues now:

• Desktop: Sometimes when I try to add a new query/data source it adds it to the old/current query rather than creating a new one? I'm not exactly sure how to reproduce

• If the columns aren't all exactly the same for both data sources (e.g., one is missing one column), I get errors (first screenshot is desktop, second is web):

  

• Saw that when a query with multiple data sources has a source changed, it always throws an error even if it's successful: Uncaught (in promise) Catalog Error: Table with name AICS-13, AICS-5 does not exist! Did you mean "AICS-13"? FROM "AICS-13, AICS-5"

Also two future feature requests:

• would be really nice if we could delete queries without opening them, since clicking on them to delete means the app automatically attempts to run the query, which can be annoying if it's an old data source I don't intend to replace
• Some errors don't render in the file list UI, so it can look like it's just loading forever if you don't know to check console