Open-EO / openeo-web-editor

An interactive and easy to use web-based editor for the OpenEO API.
https://editor.openeo.org
Apache License 2.0
12 stars 17 forks source link

Try to download all files at once #268

Open m-mohr opened 2 years ago

m-mohr commented 2 years ago

My suggestion in regard of user convenience would be the possibility to download the whole layerstack by integrating this opportunity into the action buttons in the web editor. As for now, this opens a window where it is possible to download one layer after another.

Try to add a download button to the modal where you can download all files at once. Related to #265 (don't download metadata by default)

m-mohr commented 1 year ago

Closing as this is restricted by the browsers (either memory or number of parallel downloads) and would basically need a server-side solution which would need to zip the files and offer them for download.

m-mohr commented 1 year ago

There seems to be a Chrome only API: https://github.com/WICG/file-system-access

Anyway, no solution seems to work across all browsers in a reliable way.

Patrick1G commented 12 months ago

@m-mohr some option for downloading potentially hundreds of files from the editor would be valuable, e.g. here: https://github.com/eodcgmbh/sen2like/discussions/3#discussioncomment-7111804

if the browser does not support this, could the save_results process be adapted or a new process be defined e.g. save_custom_results()... For example having the ability to stack a time series by spectral bands would be super valuable.. Or even simply adding all result files into a single zip file would be valuable..

m-mohr commented 12 months ago

@patrick-griffiths I understand and agree that this would be valuable and I would be very happy to implement this if there would be a technical solution available in the Browser. Unfortunately, it looks like there is non (yet). We also discussed this recently in the openEO meeting with a wide variety of developers and we agreed that there is no good solution due to Browser limitations. It needs a server side implementation if we want to provide a user-friendly solution, even for adding them to a ZIP file. So this should be discussed as part of the overall architecture. Maybe the aggregator can do this as an added functionality in openEO Platform based on normal content-negotiation mechanisms (happy to clarify this in the API).

Patrick1G commented 12 months ago

@m-mohr - then lets propose this (server side implementation, via the aggregator) as one of the evolution enhancements we could still do within the platform contract during the team call on Thursday... Wonder if you could come up with a cost estimate for such a task...

In the meantime, I wonder if we could create a workaround in the Editor using the UDP functionality?

m-mohr commented 12 months ago

In the meantime, I wonder if we could create a workaround in the Editor using the UDP functionality?

I'm not aware of any implemented process that would allow this.

Patrick1G commented 12 months ago

@m-mohr - maybe along the python client solution that @ValentinaHutter had suggested:

results = job.get_results()
metadata = results.get_metadata()
results.download_files("./data/")
m-mohr commented 12 months ago

I don't understand how that should work?! I can't run Python in the Browser.

Patrick1G commented 11 months ago

I was wonder if we could pack the python functionality for downloading all files at once into a User Defined Process that can then be (re)used in the editor to download all files at once. But maybe the problem with the browser would still persist?

m-mohr commented 6 months ago

Some smart people seem to have found a solution that we could try an re-use: https://github.com/jimmywarting/StreamSaver.js -> Saving multiple files as a zip

Patrick1G commented 6 months ago

@m-mohr - lets put in a dedicated task for our final SAP!

m-mohr commented 6 months ago

It has limitations though, which seem to be around 500MB - 1 GB in total. So a server-side solution is still preferrable.