nasa-fornax / fornax-demo-notebooks

Demo notebooks for the Fornax project
https://nasa-fornax.github.io/fornax-demo-notebooks/
BSD 3-Clause "New" or "Revised" License
8 stars 19 forks source link

Herschel module is slow #290

Open jkrick opened 1 month ago

jkrick commented 1 month ago

There is no clear way to make the module faster, but documenting here that the module is very slow, and any future work on this module might want to attempt to address this. There are many bottlenecks to this, one of which is the age of the dataset so it is hard to get anyone's attention to help work on it from the archive site. The problem is that all of the data for a particular observation needs to be downloaded in order to get to one single file, but this is the way the ESA archive serves the data.

One solution that could be explored is using the high level data products on the IRSA site, but it appears as though these might be similarly slow.