skyfielders / python-skyfield

Elegant astronomy for Python
MIT License
1.41k stars 211 forks source link

XStatic Package for data files #284

Closed ReimarBauer closed 5 years ago

ReimarBauer commented 5 years ago

Hi,

XStatic is a packaging standard to package external (often 3rd party) static files as a python package, so they are easily usable on all operating systems, with any package management system or even without one.

I am thinking about to create one for the file we use "de421.bsp". If I add the path where that file already exists the Loader does not download it again, or does it?

Adding the data by a XStatic package will require a maintanance step if the data gets updates, of those packages but then data also has a revision.

I am also not sure how that idea scales with a lot files, but it may solve a lot different kind of operation system related questions.

Please let me know what you think about that idea.

brandon-rhodes commented 5 years ago

Skyfield originally had a system for distributing ephemerides as Python packages, but I abandoned it. The files are very large — the popular file for Jupiter’s moons, for example, is 932 MB — and I didn’t think it was reasonable for the Python folks to have to support the download of all that data. Given that the file’s owners, the JPL, have official ways to download them, I prefer Skyfield to stick with those distribution mechanisms, I think.

brandon-rhodes commented 5 years ago

Having reflected on the matter for a few more days, I continue to favor 3rd party files coming from 3rd parties, given their sizes; which also lets the 3rd parties update them without my having to put out a new package. I'm going to close this for now, but feel free to comment if you have further thoughts!

ReimarBauer commented 5 years ago

If the files wouldn't that big or the internet would be more stable we could access all those files on their origin location by a Pyfilesystem2 address.

I agree copying 1GB files in a repository is a worse idea. Not thought on the bigger files and also versions of not so small smaller files.