Move demo files from schmidt9 to PZ directory at NERSC

[sschmidt23]

We have a couple of intermediate size files that we grab in some example notebooks, those are currently stored in /global/cfs/cdirs/lsst/www/schmidt9, however, we recently created /global/cfs/cdirs/lsst/www/PZ, we should move the files from the schmidt9 directory over to the PZ directory so that all files are in the same place, and also update the curl location in the couple of demo notebooks.

[aimalz]

Will resolution of this issue indeed entail eliminating the need for the src/rail/examples_data directory?

[eacharles]

Doubt it…. Those are just bigger data files. There are useful config files in src/rail/examples_data, I think we are just going to have to live with it.

-e

On Jun 8, 2023, at 1:07 PM, Alex Malz @.***> wrote:

Will resolution of this issue indeed entail eliminating the need for the src/rail/examples_data directory?

— Reply to this email directly, view it on GitHub https://github.com/LSSTDESC/rail_hub/issues/26, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRIGITL2HASPE4476UL2CLXKIWG3ANCNFSM6AAAAAAY3RQGVE. You are receiving this because you are subscribed to this thread.

[eacharles]

Curling all the config files would just be a pain.

On Jun 8, 2023, at 1:09 PM, Eric Charles @.***> wrote:

Doubt it…. Those are just bigger data files. There are useful config files in src/rail/examples_data, I think we are just going to have to live with it.

-e

On Jun 8, 2023, at 1:07 PM, Alex Malz @. @.>> wrote:

Will resolution of this issue indeed entail eliminating the need for the src/rail/examples_data directory?

— Reply to this email directly, view it on GitHub https://github.com/LSSTDESC/rail_hub/issues/26, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRIGITL2HASPE4476UL2CLXKIWG3ANCNFSM6AAAAAAY3RQGVE. You are receiving this because you are subscribed to this thread.

[aimalz]

Sorry, I didn't mean to imply that the contents of src/rail/examples_data should be deleted! I'm pretty strongly in favor of keeping all the data files together in the public space on NERSC rather than committing to the ongoing maintenance of picking and choosing where each file belongs, and thus what refers to a file in some subdirectory within src/rail versus what curls from a shared PUBDATALOC+\[path ending\], on a case-by-case basis every time someone tries to push data to the repo. Good point about the configs though -- now is a good time to confirm the accuracy of my assumption that the configs were destined for rail_hub. (Although writing that out highlights an embarrassing likely think-o on my part: am I incorrect in understanding that there's an unavoidable limitation of either ceci or the namespace setup that forces config/data files to live in src/rail rather than in examples or tests?)

[eacharles]

By config files I mean estimator specific text files like the names of columns or filter files or whatever else is needed to run the estimator that we decided not to try to pull out into the configuration dictionary that ceci uses. Basically if ceci config says “go read a file to get the filters or the column names or whatever, a default version of it ends up in examples_data.

Please don’t start suggesting that we should move this stuff at this point. It been a process just getting it all here.

-e

[aimalz]

Gotcha, no argument there -- I thought you were referring to something totally different and thus asked a nonsensical question.