[NERSC] add astrodash to td_env

[AmandaWasserman]

Hello, Description A clear and concise description of what the issue is:

I am installing astrodash onto NERSC in TD_SOFTWARE. I have already cloned the repository, but it seems like we need to pip install astrodash. The documentation for this is at the following link: https://astrodash.readthedocs.io/en/latest/installation.html

Choose all applicable topics by placing an 'X' between the [ ]:

• [ ] jupyter
• [ ] jupyter terminal
• [ ] Cori interactive command line
• [ ] Batch jobs
• [x] python
• [ ] CSCRATCH
• [ ] Community File System
• [ ] HPSS (tape)
• [ ] Data transfer and Globus
• [ ] New User Account or account access problems

To Reproduce Steps to reproduce the behavior:

1. Go to '...' cd /global/cfs/cdirs/lsst/groups/TD/SOFTWARE/astrodash/astrodash
2. Try to run python classify.py -h
3. See error ModuleNotFoundError: No module named 'astrodash'

Thank you, Amanda Wasserman

[heather999]

Hi @AmandaWasserman Thanks for the request - I just have a couple of questions and adding @RickKessler and @gnarayan for their insight. Is astrodash meant to be added to the td_env environment for all of TD to use? Or are you looking to have it installed for your own use? Looking at the astrodash website - it appears to me that the code is a little on the older side, and the only recent modifications seems to be associated with updating the training models. I"m a little worried about whether there might be code in there that is tied to older versions of numpy, scipy etc - but we can do a test install and see what happens. Do we know the developer and expect the code is being maintained?

[RickKessler]

astrodash is intended for TD use, and we know the developer.

[heather999]

Great - ok working on this.. and a couple more questions!

• Do we need pyqt5 - which seems to need to be installed separately as it doesn't come along with the pip install? it would be needed to use the GUI for astrodash https://astrodash.readthedocs.io/en/latest/installation.html#dependencies

• Running the example - python classify.py -hresults in downloading a data file:

  python classify.py -h
  Downloading data files from https://github.com/daniel-muthukrishna/astrodash/blob/master/astrodash/models_v06.zip?raw=true... 
  This download contains the Tensorflow models trained using deep learning. 
  The file to be downloaded is ~200MB. However, this is a one time download, and will only occur
  the very first time that this version of DASH is installed.

I'm assuming we should have this file downloaded as part of the astrodash install.

[AmandaWasserman]

I think in the end we will probably not use the GUI, but it might be good to have as we're testing various things. Might be worth having pyqt5.

I believe that file should be downloaded. When I recently downloaded DASH on my computer, the zip file that came with cloning was not unzippable. When I reached out to the developer he sent me to a Zenodo link to download the real models_v06.zip here: https://github.com/daniel-muthukrishna/astrodash

Rick and I downloaded this to the astrodash directory that we had created in the TD software directory if that helps.

Let me know if there's more info I can add that would help

[heather999]

Hi @AmandaWasserman there is a new "dev" version of td_env with astrodash (and the model06 included). You can try it out by doing: source /global/cfs/cdirs/lsst/groups/TD/setup_td_dev.sh The astrodash GUI is available as well as importing the library. I set up an env variable to point to the scripts included in the package in case that is useful: $TD_ASTRODASH_DIR - so I could do something like: python $TD_ASTRODASH_DIR/classify.py -h This also points to the location where the model06 files are installed.

Just let me know if this is generally working or there are things missing.

[heather999]

astrodash is included with td_env 0.0.7 which was released today and is available via source /global/cfs/cdirs/lsst/groups/TD/setup_td.sh Just let me know if you run into any trouble!

[heather999]

Think this is done!