search

NASA-IMPACT / covid-api

MIT License
14 stars 3 forks source link

EPIC: analysis ready data access/data retrieval from Dashboard #138

Closed leothomas closed 3 years ago

leothomas commented 3 years ago

"Power users" might want to grab data from the dashboard to perform further analysis on. This might be underlying COG's, a jupyter notebook, .tif files, or .csv files corresponding to time series graphs

leothomas commented 3 years ago

Some thoughts on the design of this feature:

Using the API to find COGs and download them directly from S3 is documented here. These instructions might be confusing to non-technical users, as they require knowledge of REST APIs, the AWS CLI and downloading file objects from the terminal using cURL or wget (although I would assume that most people interested in downloading COGs have at least some knowledge of the above topics).

WebMap Tiles:

The binary content of the web map tiles generated by the API can be wrapped in a StreamingResponse object, which will be downloaded by most browsers

Source COGs:

The source COG should not be downloaded to the user's machine using the same StreamingResponse technique as above, as this would entail loading the entire COG into the lambda's runtime memory. Even using the FileResponse object would require the entire COG to be copied over to the Lambda's runtime environment

I suggest returning an S3 presigned url as a RedictResponse - which should also have the same result of downloading the file in the user's browser.

CSV's for mean/median values (from the AOI feature and /timelapse endpoint)

Using the CSV Middleware from the Trace Emissions API we can easily add an accept header that specifies the text/csv mimetype as a return type - this will convert the JSON result of the API call into a CSV object which will be returned as a StreamingResponse (and download in most browsers).

olafveerman commented 3 years ago

This is currently being dealt with on the frontend. Closing this issue.