oxinabox
commented
4 years ago I think the correct behavour is if anythjing goes wrong during upgrade, we print a warning, then just make the Manifest and Project empty or something.
Open oxinabox opened 4 years ago
oxinabox
commented
4 years ago I think the correct behavour is if anythjing goes wrong during upgrade, we print a warning, then just make the Manifest and Project empty or something.
rofinn
commented
4 years ago During an upgrade, if we can't add specific package versions then we fallback to just trying to the package without any version constraints, since they're sufficiently different between julia 0.6 and 1.0. I don't think having an empty manifest or project is the right approach.
Overall, I don't think it's JLSO's responsibility to figure out which packages are needed for the data being saved. I'll note that in this situation there is enough information that you can simply reconstruct the file by simply modifying the BSON data directly, which is the primary benefit of the JLSO format.
oxinabox
commented
4 years ago I don't think having an empty manifest or project is the right approach.
I disagree.
If all else fails it is still better to be able to load the data than not.
rofinn
commented
4 years ago You can still load the raw dictionary with BSON though, the point of JLSO is to avoid throwing away information about the original environment as best as possible.
oxinabox
commented
4 years ago We can always display a warning and a recommendation to pin JLSO to old version til we fix what ever bug.
oxinabox
commented
4 years ago @rofinn said:
@yakir12 If you're positive that you don't need those packages then you could just modify the bson and rewrite the file.
using BSON
d = BSON.load("trackdata.jlso")
for pkg in ["Bills", "Budgets", ...]
delete!(d["metadata"]["pkgs"], pkg)
end
BSON.bson("trackdata-reduced.jlso", d)Then you can load the new file as you'd expect.
rofinn
commented
4 years ago We can always display a warning and a recommendation to pin JLSO to old version til we fix what ever bug.
The issue is that this isn't a bug... this is exactly what JLSO was designed to do. In this case, the issue is that the old environment cannot be constructed with the information available and we need user intervention to tell us what data we can throw away during the upgrade. There is the JLSO.upgrade(src, dest, project, manifest) method which we could mention in the error message, but that should be something that users explicitly call rather than us falling back to throwing away the pkg information.
yakir12
commented
4 years ago So I just went crazy and blindly deleted all the packages JLSO complained about, realizing that (actually) some of those are needed... But, it all just worked! It took a bit of time, installed a bunch of things it needed, but everything works. Very happy.
In the future, it might be prudent to warn the user, when saving a jlso file, that it's always better for posterity if all the used packages are online if not registered and that local packages should be moved online (and add TheOnlineVersion).
The trick of knowing which packages are needed and which are not (when users can just write using PackageThatIsNotUsed) is I image difficult, but it would be nice. I often add and using packages I end up not needing when I develop stuff and need to clean up later. A tool for that would be nifty.
Originally posted by @yakir12 in https://github.com/invenia/JLSO.jl/issues/32#issuecomment-565054885
I guess this is related. Anything I can do about it, none of those package were relevant, but yea, when I saved that file they were in the stack. But they aren't needed to parse the data in the file.