NOTE: Moved from astropy/astropy#700, originally reported by @kbarbary back in Jan 29, 2013.
Affiliated packages for various subfields of astronomy are likely to include package data specific to the subfield. It would be nice to have a step-by-step "HOW TO" (with an example) in the developer documentation on the proper way to include such data.
There are currently guidelines on package data and configuration here. The process is fairly obvious for small data (<100kb), but it is not exactly clear how to extend the outlined process to a third-party server for the affiliated package. One might be able to figure it out by reading the docs for astropy.config and astropy.utils.data, but it would be nice to have a simple example that shows the following:
Adding a default configuration file for the affiliated package (wherein the dataurl for the affiliated package is set)
How to appropriately name your data files on the server side.
A simple class or function in the affiliated package that uses the data.
The documentation might also address some related questions that came up for me regarding data file sizes:
Related question 1: How big is too big? (for even the externally hosted data scheme) Specifically, I'm thinking here of the SFD dust maps, which are ~200MB.
Related question 2: Is/should/could there be a way to install or cache files somewhere other than the user's home directory? (1) For a system-wide install used by multiple users, you might want to only have one copy of the package data. (2) Also, on some systems, disk quotas in home directories are quite small and data is meant to reside elsewhere (or should the user just set XDG_CACHE_HOME accordingly in this case?)
pllim
commented
1 year ago
NOTE: Moved from astropy/astropy#700, originally reported by @kbarbary back in Jan 29, 2013.
Affiliated packages for various subfields of astronomy are likely to include package data specific to the subfield. It would be nice to have a step-by-step "HOW TO" (with an example) in the developer documentation on the proper way to include such data.
There are currently guidelines on package data and configuration here. The process is fairly obvious for small data (<100kb), but it is not exactly clear how to extend the outlined process to a third-party server for the affiliated package. One might be able to figure it out by reading the docs for
astropy.configandastropy.utils.data, but it would be nice to have a simple example that shows the following:dataurlfor the affiliated package is set)The documentation might also address some related questions that came up for me regarding data file sizes:
Related question 1: How big is too big? (for even the externally hosted data scheme) Specifically, I'm thinking here of the SFD dust maps, which are ~200MB.
Related question 2: Is/should/could there be a way to install or cache files somewhere other than the user's home directory? (1) For a system-wide install used by multiple users, you might want to only have one copy of the package data. (2) Also, on some systems, disk quotas in home directories are quite small and data is meant to reside elsewhere (or should the user just set
XDG_CACHE_HOMEaccordingly in this case?)