integrator.integrateImageStack causes (BNL jupyterhub) kernel to die

[cbishop4]

When integrating an image stack, kernel seems to always die. This has been reported across multiple users.

[pbeaucage]

Thanks for the issue! This is a problem on the BNL side with their installation of PyFAI/OpenCL. As a workaround from the PyHyper side, you can pass integration method = 'csr' when creating an integrator to use CPU-only integration. This may be slower on a machine with a working OpenCL install but avoids this crash.

[dsunday]

I tried switching to integration_method = 'csr' it works sometimes, however the kernel has died twice now when trying this.

[EliotGann]

They are trying to fix this on the Jupyter side, basically making different environments. I think they decided making a new environment and installing everything does fix it although it’s not an awesome solution. Making your own environment might be the fastest way to fix it permanently though. Maybe Peter can give a simplest environment building instructions?

[EliotGann]

https://jupyter.nsls2.bnl.gov/hub/guide Has instructions for making your own permanent kernel, just what packages to install to have pyhyper running is the question (with tiled integration etc)

[pbeaucage]

The requirements in requirements.txt should be up to date as with most any Python package. There are many sub-dependency sets in there, however. To get everything pip install pyhyperscattering[bluesky,performance,ui]. If there are missing dependencies there, please raise that as a separate issue.

[EliotGann]

@cbishop4 @dsunday can you follow those directions in my link and run Peter’s command and let us know if that works for you? You should have a kernel of your own that will not change, so if it works it should stay working.

[dsunday]

When running in a new environment this error occurs when loading SST1RSoXSDB

----> 4 load = PyHyperScattering.load.SST1RSoXSDB(corr_mode='none')

File ~/.conda/envs/RSOXS2/lib/python3.11/site-packages/PyHyperScattering/SST1RSoXSDB.py:96, in SST1RSoXSDB.init(self, corr_mode, user_corr_fun, dark_subtract, dark_pedestal, exposure_offset, catalog, catalog_kwargs, use_precise_positions, use_chunked_loading) 94 self.use_chunked_loading = use_chunked_loading 95 if catalog is None: ---> 96 self.c = from_profile("rsoxs", **catalog_kwargs) 97 else: 98 self.c = catalog

NameError: name 'from_profile' is not defined

[pbeaucage]

In separate troubleshooting of this issue, we identified that requirements-databroker needs certain sub-requirements specs on tiled and databroker. (tiled[client] and databroker[all]). adding that in a linked PR/feature branch.

[pbeaucage]

PR #115

[dsunday]

I used tiled[all], rather than tiled[client]. We should check to see if that will also work.

[danielballan]

Using tiled[all] should deliver a superset of tiled[client]. For your used case, tiled[client] should be sufficient (and faster).

[dsunday]

1. See Instructions at https://jupyter.nsls2.bnl.gov/hub/guide for making your own environment a. Install pyhyperscattenring i. pip install pyhyperscattering[bluesky,performance,ui] b. Install Tiled/Databroker i. Pip install --pre --upgrade tiled[client] databroker[all] (tiled[all] will also work]
2. May also need hvplot c. Make sure you run this line of code to make the option for a new kernel available i. Python -m ipykernel install --user --name $ENV_NAME --display-name “$ENV_NAME” d. Go to Kernel -> Change Kernel – your environment name should be available there

[cbishop4]

After some wrestling, I think I finally have all the packages right in my environment. However, I'm running into this now when I call load_db = phs.load.SST1RSoXSDB(corr_mode='none'):

Server error '503 Service Unavailable' for url 'http://local-tiled-app/api/?root_path=true' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/503

Looking into the website, it says this is usually caused by a server that is down for maintenance or overloaded. Is this a thing I need to wait for, or is there a workaround?

It was throwing other errors before I force reinstalled my tiled to 0.1.0a74, and my databroker to 2.0.0b10.

[pbeaucage]

That tiled is extremely old, likely not compatible with newer stuff on the system and maybe even incompatible with the data storage format. How did you get that? pip install --pre --upgrade tiled[client] databroker[all] should get you the newest, working version. I'm hoping I didn't typo the PyHyper requirements-bluesky.txt file.

[cbishop4]

Oh here's what probably happened, I was looking at an old folder with PHS files in it. That did indeed fix it. I had done it earlier and it wasn't working, so I manually wrangled a bunch of stuff around, but I won't complain that it's working now. Thanks!

[cbishop4]

(@pbeaucage , please let me know if we should start moving this to its own issue) I was having trouble installing tiled, as I was not able to build wheels for blosc. I had been working in Python 3.12.xx (whatever the most current is). I tried making an environment with python version 3.10, which solved the issue. Have others run into this?

Otherwise, everything looks smooth for now - hope I don't run into dependency issues down the road with the older python version.

[EliotGann]

I think 3.12 is still a little new/experimental? 3.11 might be the best choice at the moment.

[pbeaucage]

I think these issues are now resolved.

In general, contributors to/maintainers of PyHyper have little ability to resolve issues with other site computing environments so the most productive route to getting these things resolved is either working with facility support (NSLS2 DSSI, likely through your beamline scientist or other support contact) or filing an issue at the relevant repos (here, https://github.com/nsls2-conda-envs/nsls2-collection-tiled ). That said, this was a nice find and we did resolve several underlying, contributing factors in the PyHyper codebase/requirements spec.

If there are other issues that are directly with PyHyper, please feel free to reopen or make a new issue.