parsotat / BatAnalysis

A python HEASOFT wrapper for processing Swift-BAT data.
MIT License
30 stars 11 forks source link

Added multi-threaded downloads, issue #7 #9

Closed dmopalmer closed 1 year ago

dmopalmer commented 1 year ago

Implementation for #7 . Multi-threaded downloading.

download_swiftdata(..., jobs=10,...) spawns 10 threads (the default), each of which downloads a different set of OBSID data.

If jobs==1, the main thread is used. Otherwise jobs additional threads are produced while the main thread waits.

parsotat commented 1 year ago

@dmopalmer can you add a bit more information in the docustring of the download_swiftdata function outlining some of the functionality?

Can you also add a docustring to the _download_single_observation that outlines the sequence of searching for the requested data, with relevant comments? An outline of this relatively complex function will be really helpful.

dmopalmer commented 1 year ago

I will add more to the docstring for download_swiftdata(), which it needs. This extremely complex function really needs an entire example notebook (which I will write, but not immediately.)

The _download_single_observation() is a private function (hence the leading _) not for general use. It is for encapsulating an observation download as a task that can be put on a separate thread. I will improve its documentation.

dmopalmer commented 1 year ago

Improved docstrings and an example notebook (but not its .py version) have been added to this PR.

parsotat commented 1 year ago

This looks really good to me. The last thing that should be done is to increment the version appropriately so the pypi package that I upload has the versioning associated with these improvements.

dmopalmer commented 1 year ago

Do you want me to add the jupyter nbconvert python file for the notebook?

parsotat commented 1 year ago

I don't think that is necessary as I think that the Jupyter notebook is the best format for this tutorial/explanation of downloading data.