search

JuliaIO / JLD2.jl

HDF5-compatible file format in pure Julia
Other
561 stars 92 forks source link

Roadmap to 1.0 #162

Open DilumAluthge opened 5 years ago

DilumAluthge commented 5 years ago

What needs to be done before we can release JLD2 version 1.0.0?

Breaking changes we want to make before releasing 1.0:

Nonbreaking changes we would like to make. These can be made after releasing 1.0:

alyst commented 5 years ago

It would be nice to address #133 or have some way to control the compression from @save.

It would also be nice to do time/filesize profiling in typical workloads, because I have an impression that JLD2 is not very fast, and the filesizes are larger than alternatives (serialization, .RData files). Also, enabling compression didn't reduce the file sizes as much as I have hoped.

timholy commented 5 years ago

Another thought would be implement some variant of addrequire. I don't want to portray this as essential because no matter what, I think we have to support files written in older formats whenever we move to 1.0.

DilumAluthge commented 5 years ago

All of those are nice and good ideas, but none of them are breaking, right?

The only reason to delay the release of 1.0 is if there are breaking changes we want to make. Once we release 1.0 we can still make as many non-breaking changes as we want.

timholy commented 5 years ago

I don't think any are breaking.

alyst commented 5 years ago

Compression configuration is theoretically API-breaking. It would be nice (IWBN) to decide what to do with it before releasing 1.0. If the planned change would not break the API, it could be done after 1.0.

There could be some storage format tweaks that improve the performance (e.g. #135) or resolve data corruption issues (e.g. #163), IWBN to introduce the changes before 1.0 as well.

Maybe the current issues should be triaged as 1.0-blocking/nonblocking? Even if the fixes would be non-breaking, IWBN to tag 1.0 as a declaration of package maturity.

alyst commented 5 years ago

Also, #126 & #151 were non-breaking, but many JLD2 files created before 0.1.6 are unusable because of the wrong type names. This is something that should be avoided for files created with JLD2 1.0+.

cossio commented 2 years ago

Is there anything holding back a 1.0 release at this point?

JonasIsensee commented 1 month ago

As an update on this: The compression part of the library needs a reworking. This can't be done without breaking API changes. That is the main thing holding back 1.0 at this point.