One idea by @bcorrie for the analysis API is that it would handle querying and downloading data from any/all repositories which is then run through analysis. Should we think about writing one? Make it an AIRR ADC extension, or an iR+ service? It seems like it is needed in multiple places: the Gateway needs it to provide downloads, the analysis API needs it to collect the data for analysis, VDJServer will need it as part of the Community Data Portal and/or for running analysis. I think ImmuneDB would also like it to encapsulate multiple ImmuneDB's behind a single API. Should everybody write their own?
Such a service isn't a simple thing though.
Needs to be an asynchronous service because the data collection can take some time.
Needs to handle the differences in repositories.
Needs to be fault tolerant to retry queries for failures.
Needs to manage the disk space for the downloads.
Needs to expire old requests and clean up disk space.
Needs to package the data and provide download.
Needs to notify users and/or clients.
Needs to support usage from the command line but also in GUIs and from other services.
schristley
commented
1 year ago
One idea by @bcorrie for the analysis API is that it would handle querying and downloading data from any/all repositories which is then run through analysis. Should we think about writing one? Make it an AIRR ADC extension, or an iR+ service? It seems like it is needed in multiple places: the Gateway needs it to provide downloads, the analysis API needs it to collect the data for analysis, VDJServer will need it as part of the Community Data Portal and/or for running analysis. I think ImmuneDB would also like it to encapsulate multiple ImmuneDB's behind a single API. Should everybody write their own?
Such a service isn't a simple thing though.
Am I missing anything?