Open sjdv1982 opened 1 year ago
khinsen
commented
1 year ago The "data not extractable" messages are expected. "aptool checkout" is meant for code and documentation, and can in fact only handle text. Anything labelled "data" is an HDF5 dataset. It doesn't have an obvious on-disk representation that could be checked out.
Accessing the data thus requires HDF5. The ActivePaper file is an HDF5 file, so you can open it using h5py and simply access all the datasets.
khinsen
commented
1 year ago Nevertheless, aptool update -v should work, and it does on my machine. It generates lots of warnings from matplotlib, plus one from Python, which are due to Python and matplotlib having evolved in the meantime (neither caring much about backwards compatibility). Here is the complete output I get:
~/Temp/ap-tutorial $ guix shell python python-activepapers python-matplotlib -- aptool update -v
Dataset /data/coordinate_trajectories/translation_molecular_frame_9 is stale or dummy, running /code/trajectory_processing
Dataset /data/correlation_functions/mean_square_displacement_molecular_frame_8 is stale or dummy, running /code/correlation_functions
Dataset /documentation/msd_rr_off_diagonal.pdf is stale or dummy, running /code/plots
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'x' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'y' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'z' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'x' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'y' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'z' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'x' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'y' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'z' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'x' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'y' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'z' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'xz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'yz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zx' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zy' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
7f13f833dd00:/code/plots:24: UserWarning: Legend does not support 'zz' instances.
A proxy artist may be used instead.
See: https://matplotlib.org/users/legend_guide.html#creating-artists-specifically-for-adding-to-the-legend-aka-proxy-artists
Dataset /data/diffusion_tensor_msd_laboratory_frame is stale or dummy, running /code/diffusion_tensor
7f13ee3f7fd0:/code/python-packages/fitting:9: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.So the big question is: what's different between our environments? Guix has 214 dependencies of ActivePapers, but that's because it includes all compile-time tools, recursively. Picking from that list the packages you have listed above, as much as possible, I find:
python-activepapers@0.2.2
bzip2@1.0.8
python-h5py@3.6.0
hdf5@1.10.7
libffi@3.3
gfortran@10.3.0
libstdc++@10.3.0
zlib@1.2.11
python-matplotlib@3.5.2
ncurses@6.2.20210619
python-numpy@1.21.6
openblas@0.3.20
openssl@1.1.1l
python-pyparsing@3.0.6
python@3.9.9
readline@8.1.1
python-six@1.16.0
sqlite@3.36.0
python-tempdir@0.7.1
tk@8.6.11.1
python-wheel@0.37.0
xz@5.2.5
zlib@1.2.11Lots of small differences, nothing that looks obviously problematic.
khinsen
commented
1 year ago Looking at your traceback, what strikes me is the file name ActivePapers.Py-0.1.4-py3.5.egg, in an environment claiming to contain ActivePapers 0.2.2. There are more than three years between 0.1.4 and 0.2.2.
Could you check what the file `activepapers/version.py' contains in your environment?
sjdv1982
commented
1 year ago Looking at your traceback, what strikes me is the file name
ActivePapers.Py-0.1.4-py3.5.egg, in an environment claiming to contain ActivePapers 0.2.2. There are more than three years between 0.1.4 and 0.2.2.Could you check what the file `activepapers/version.py' contains in your environment?
Ah, I also tried the PyPI ActivePapers version, and I copy-pasted the wrong traceback. Now edited. But it doesn't look different.
The "data not extractable" messages are expected. "aptool checkout" is meant for code and documentation, and can in fact only handle text. Anything labelled "data" is an HDF5 dataset. It doesn't have an obvious on-disk representation that could be checked out.
What I would expect is either:
A list of zero-byte files is not what I expected.
Accessing the data thus requires HDF5. The ActivePaper file is an HDF5 file, so you can open it using h5py and simply access all the datasets.
It's a bit scary to have scientific data hidden from non-programmers. If you wish, I could cook up an extract data script that does the equivalent of tar -xf lysozyme.ap
sjdv1982
commented
1 year ago So the big question is: what's different between our environments? Guix has 214 dependencies of ActivePapers, but that's because it includes all compile-time tools, recursively. Picking from that list the packages you have listed above, as much as possible, I find:
python-activepapers@0.2.2 bzip2@1.0.8 python-h5py@3.6.0 hdf5@1.10.7 libffi@3.3 gfortran@10.3.0 libstdc++@10.3.0 zlib@1.2.11 python-matplotlib@3.5.2 ncurses@6.2.20210619 python-numpy@1.21.6 openblas@0.3.20 openssl@1.1.1l python-pyparsing@3.0.6 python@3.9.9 readline@8.1.1 python-six@1.16.0 sqlite@3.36.0 python-tempdir@0.7.1 tk@8.6.11.1 python-wheel@0.37.0 xz@5.2.5 zlib@1.2.11Lots of small differences, nothing that looks obviously problematic.
This is the best I can do in order to match your package versions:
name: active-papers
channels:
- conda-forge
dependencies:
- python=3.9
- pip
- hdf5=1.10
- numpy==1.21
- pip:
- h5py==3.6.*
- tempdir==0.7.*
- matplotlibNo difference in the traceback
File "/home/sjoerd/miniconda3/envs/active-papers/lib/python3.9/site-packages/ActivePapers.Py-0.2.2-py3.9.egg/activepapers/storage.py", line 418, in iter_dependencies
yield self.file[dep]
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "/home/sjoerd/miniconda3/envs/active-papers/lib/python3.9/site-packages/h5py/_hl/group.py", line 305, in __getitem__
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: 'Unable to open object (component not found)'EDIT: better matching of the package versions
khinsen
commented
1 year ago I agree about data checkout, and opened an issue for that (#21).
Thanks for trying to match my versions! This still looks weird. I'll have to debug this under Miniconda.
Active papers installation
Environment
active_papers.ymlInstallation commands
Installed packages
Output:
Running the tutorial
Output:
Output: