LSSTDESC / rail_attic

Redshift Assessment Infrastructure Layers
MIT License
14 stars 9 forks source link

Moving data #356

Closed OliviaLynn closed 1 year ago

OliviaLynn commented 1 year ago

Returning to the question of "does data really need to live inside src/rail?"

I've done my best to research data storage best practices for python packages, but have not found anything that says it must be kept right next to the code for a case like RAIL (in which our data, in theory, is only used by notebooks and tests).

I will be exploring this to figure out if we really can move the data. (I feel like I've encountered one place where data was used by actual rail code in a way that surprised me, but I don't remember off the top of my head.)

Let me know if you have thoughts or further information on this.

eacharles commented 1 year ago

In short, I don’t think it has to live there, but I wasted a few hours trying to setup the pyproject.toml file to pick it up correctly if it wasn’t there and figured i had better things to do with my time.

If you can come up with a way to get it to work, say in a top level ‘data’ directory, that would be great.

-e

On Apr 28, 2023, at 1:53 PM, Olivia R. Lynn @.***> wrote:

Returning to the question of "does data really need to live inside src/rail?"

I've done my best to research data storage best practices for python packages, but have not found anything that says it must be kept right next to the code for a case like RAIL (in which our data, in theory, is only used by notebooks and tests).

I will be exploring this to figure out if we really can move the data. (I feel like I've encountered one place where data was used by actual rail code in a way that surprised me, but I don't remember off the top of my head.)

Let me know if you have thoughts or further information on this.

— Reply to this email directly, view it on GitHub https://github.com/LSSTDESC/RAIL/issues/356, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRIGITFY4OCJBVWTC5D3WTXDQU55ANCNFSM6AAAAAAXPWIRBY. You are receiving this because you are subscribed to this thread.

aimalz commented 1 year ago

I'm going to close this issue because LSSTDESC/rail_hub#26 contains the specifics of where files are being moved to and encompasses propagating the changes through the tests and demos.