search

Caltech-IPAC / irsa-tutorials

IRSA Tutorials
https://caltech-ipac.github.io/irsa-tutorials/
BSD 3-Clause "New" or "Revised" License
2 stars 2 forks source link

Parallelize the Roman Simulated Time Domain Survey notebook #8

Closed vandesai1 closed 1 month ago

vandesai1 commented 3 months ago

There is a cell in the Roman Simulated Time Domain Survey notebook that says it will take a couple of minutes. It is a for loop. Parallelize this to make it go faster!

troyraen commented 2 months ago

The NEOWISE parquet notebook revealed an issue that affects this, documented in #15. TL;DR : If we parallelize this, it will work fine on Linux (including science platforms) but won't work for Mac and Windows users out of the box -- they'll have to manually copy the function into a separate .py file first. (There are other options, but come with tradeoffs.)

@vandesai1 do you still want this parallelized? Fine with me either way, and I can do it if you want it.

troyraen commented 1 month ago

We decided to leave this as-is for now. The discussion of how to structure repos (code in separate file vs in the notebook) and which notebooks live in which repo (irsa-tutorials vs fornax-demo-notebooks) will be ongoing.