search

JuliaIO / JLD2.jl

HDF5-compatible file format in pure Julia
Other
547 stars 85 forks source link

Remove large_fractal_heap.h5 (60Mb) file from package? #443

Closed marius311 closed 1 year ago

marius311 commented 1 year ago

Can the large 60MB large_fractal_heap.h5 file be removed frorm this repo, or its size reduced somehow? Eg between all my environments I have 4 different versions of JLD2 installed so 240MB from this file alone, which is a not-insignificant fraction of the 40GB home quota I have on a cluster I use alot.

JonasIsensee commented 1 year ago

Is there a clever way to include files as artifacts for tests?

I remember creating this file to cover some edge cases that only occur in hdf5 files with a sufficiently large size.

Datseris commented 1 year ago

@JonasIsensee yeap that is easy. Have a look of how we did it for Agents.jl for including a map of goettingen as a test data for Open Street Map.

https://github.com/JuliaDynamics/Agents.jl/blob/main/src/spaces/openstreetmap.jl#L12

https://github.com/JuliaDynamics/Agents.jl/blob/main/src/spaces/openstreetmap.jl#L161-L170

https://github.com/JuliaDynamics/Agents.jl/blob/main/Artifacts.toml

(the Artifacts docs tell you how to generate all this sha keys for your artifact)

JonasIsensee commented 1 year ago

hmm, your example does not seem to work? The artifacts.toml refers to master not main. But even if I change it to main, I can't access it or understand where this file would be located. -> It can't be in the repo, right?

Ah, never mind. You just put all the files into a separate repository.. Yes, that would work.