Lack of instruction on how to actually use the program

[legendPerceptor]

Description

After installing the program with spack, what's the next step to make sure the python interpreter can find libpressio? Is there a pip install that should happen? Can you update the readme or reply in the issue to give a more easy-to-follow instruction on the usage of this software?

Current problem

The installation failed on NERSC and Bebop but succeeded in my virtual machine. I can spack load libpressio on my Ubuntu, but I don't know how to use it in python. How can I import libpressio in my python code?

[robertu94]

@legendPerceptor Yes, I agree the README needs some help. I’m also working on an improved homepage, but I’ve focused the improvements that I can make to feature abs reliability. This is on the backlog.

My team and I have used the software on Bebop so we know it compiles and runs there. My guess is that you either need a newer compiler or to adjust your compatabilty settings. To do the latter probably need to ‘spack install libpressio+sz+zfp+python ^ libstdcompat+cpp_unstable cpp_compat=14’ but my memory here is a little fuzzy. An error message for NERSC would be helpful if this doesn’t fix it. In my mind this is also a README issue

in the meantime can you tell me if this tutorial helps understand higher level usage? It has both python and C examples with some high level description. https://github.com/FTHPC/libpressio_tutorial There are also a few talks that I gave on LibPressio on YouTube on is a tutorial talking through high level concepts.

You also mentioned PIP, this is also on the backlog since I know many Climate researcher uses it (I do too for lots of personal stuff). What makes this challenging is I personally did most of the work to get the 30+ compressors we support and their transitive dependencies packaged for Spack since this was a requirement of our funding. Doing this again for PiP is important but complicated in the number of thing needed as well as that not every compressor package supports the older gcc compiler used for Pip packages there are Additional problems on Windows that I’m about half way through solving. If you give me a useful subset of functionality I can try and prioritize that.

Lastly, I will be part of the Scientific Data Compression tutorial at ISC as well as at the ECP Annual meeting in the coming weeks. Since you mentioned Bebop and NERSC I guessed you might know about these.

[legendPerceptor]

Hi Robert,

Not that complicated. I just want to know how to run the demo program. After the installation with Spack, how to run it? My python’s virtual environment cannot see libpressio, how to let it see it?

On Sat, Apr 16, 2022 at 5:04 AM Robert Underwood @.***> wrote:

@legendPerceptor https://github.com/legendPerceptor Yes, I agree the README needs some help. I’m also working on an improved homepage, but I’ve focused the improvements that I can make to feature abs reliability. This is on the backlog.

My team and I have used the software on Bebop so we know it compiles and runs there. My guess is that you either need a newer compiler or to adjust your compatabilty settings. To do the latter probably need to ‘spack install libpressio+sz+zfp+python ^ libstdcompat+cpp_unstable cpp_compat=14’ but my memory here is a little fuzzy. An error message for NERSC would be helpful if this doesn’t fix it. In my mind this is also a README issue

in the meantime can you tell me if this tutorial helps understand higher level usage? It has both python and C examples with some high level description. https://github.com/FTHPC/libpressio_tutorial There are also a few talks that I gave on LibPressio on YouTube on is a tutorial talking through high level concepts.

You also mentioned PIP, this is also on the backlog since I know many Climate researcher uses it (I do too for lots of personal stuff). What makes this challenging is I personally did most of the work to get the 30+ compressors we support and their transitive dependencies packaged for Spack since this was a requirement of our funding. Doing this again for PiP is important but complicated in the number of thing needed as well as that not every compressor package supports the older gcc compiler used for Pip packages there are Additional problems on Windows that I’m about half way through solving. If you give me a useful subset of functionality I can try and prioritize that.

Lastly, I will be part of the Scientific Data Compression tutorial at ISC as well as at the ECP Annual meeting in the coming weeks. Since you mentioned Bebop and NERSC I guessed you might know about these.

— Reply to this email directly, view it on GitHub https://github.com/robertu94/libpressio/issues/26#issuecomment-1100626959, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFAYCOEBOR7MAUOVO3SDWNLVFKGBZANCNFSM5TR26DIQ . You are receiving this because you were mentioned.Message ID: @.***>

[robertu94]

@legendPerceptor

Three things.

1. Run spack load libpressio which is kind of like module load. Spack allows multiple versions of each package to be installed at once, and spack load brings makes them callable.
2. By default libpressio doesn't built the python bindings to save build time. You need to enable them with a spack install command like the one I provided above. 3, if you didn't change anything, spack probably built you a copy of python (this is the default). If this is the case, you will need recreate your virtual environment with the python that spack built python (my advice) or rebuild libpressio with your virtual environment's python which can be messy. The latter requires external packages to be configured.

[legendPerceptor]

Hi Robert,

Did you see the last email? After loading python and libpressio in with spack, I still cannot use it. In the python environment, there is no libpressio package. I also cannot see the pressio command.

Best, Yuanjian

Yuanjian Liu @.***> 于2022年4月16日周六 11:24写道：

Hi Robert, I still cannot import libpressio. What did I miss? [image: Screen Shot 2022-04-16 at 11.23.30 AM.png]

Robert Underwood @.***> 于2022年4月16日周六 07:14写道：

@legendPerceptor https://github.com/legendPerceptor

Two things.

1. Run spack load libpressio which is kind of like module load. Spack allows multiple versions of each package to be installed at once, and module load brings makes them callable. 2, if you didn't change anything, spack probably built you a copy of python (this is the default). If this is the case, you will need recreate your virtual environment with the python that spack built python (my advice) or rebuild libpressio with your virtual environment's python which can be messy. The latter requires external packages https://spack.readthedocs.io/en/latest/build_settings.html#external-packages to be configured.

— Reply to this email directly, view it on GitHub https://github.com/robertu94/libpressio/issues/26#issuecomment-1100650894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFAYCOE6S3ABDRNSG3AMDP3VFKVJBANCNFSM5TR26DIQ . You are receiving this because you were mentioned.Message ID: @.***>

[legendPerceptor]

It seems that the last email that has a screenshot does not come through to Github. So after loading the packages with spack, python still cannot see the libpressio package and there is no pressio command.

[legendPerceptor]

Hi Robert, I still cannot import libpressio. What did I miss? [image: Screen Shot 2022-04-16 at 11.23.30 AM.png]

Robert Underwood @.***> 于2022年4月16日周六 07:14写道：

@legendPerceptor https://github.com/legendPerceptor

Two things.

1. Run spack load libpressio which is kind of like module load. Spack allows multiple versions of each package to be installed at once, and module load brings makes them callable. 2, if you didn't change anything, spack probably built you a copy of python (this is the default). If this is the case, you will need recreate your virtual environment with the python that spack built python (my advice) or rebuild libpressio with your virtual environment's python which can be messy. The latter requires external packages https://spack.readthedocs.io/en/latest/build_settings.html#external-packages to be configured.

— Reply to this email directly, view it on GitHub https://github.com/robertu94/libpressio/issues/26#issuecomment-1100650894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFAYCOE6S3ABDRNSG3AMDP3VFKVJBANCNFSM5TR26DIQ . You are receiving this because you were mentioned.Message ID: @.***>

[robertu94]

@legendPerceptor My apologies. I thought we met and resolved this.

Can you first please try updating spack, libpressio, and robertu94/spack_packages? https://robertu94.github.io/guides/updating_libpressio_with_spack

You may also try guides for the ANL machines if needed: https://robertu94.github.io/guides

When you install LibPressio, and please use libpressio+python+sz (at least at first)

[robertu94]

Following up here. I'm going to break what I see as the that major requests here into sub requests and close this

• installable via pip
• update the README to include better (updated) installation instructions (especially for ANL machines)
• better highlight usage examples and the tutorial in the README