search

BCDA-APS / apstools

various tools for use with Bluesky at the APS
https://bcda-aps.github.io/apstools/latest/
Other
16 stars 9 forks source link

apsbss: warning messages #386

Closed prjemian closed 4 years ago

prjemian commented 4 years ago

@jkirchman emails:

I began using your apsbss tools yesterday and I'm encountering a python warning message for a number of subcommands:

519 jkirchman@s27bcda dm>  apsbss update 27id:dm:
/APSshare/anaconda3/Bluesky/lib/python3.7/site-packages/pims/image_reader.py:26: 
RuntimeWarning: PIMS image_reader.py could not find scikit-image. 
Falling back to matplotlib's imread(), which uses floats instead of 
integers. This may break your scripts.
(To ignore this warning, include the line 
"warnings.simplefilter("ignore", RuntimeWarning)" in your script.)
   warnings.warn(RuntimeWarning(ski_preferred))

I have updated my path to include the apsbss python script. Do I instead need to activate a virtual environment? Or is there a different method to address this warning?

BTW, I think you must have changed the parameters needed for current subcommand. The example at https://apstools.readthedocs.io/en/latest/applications/apsbss.html#what-proposal-and-esaf-id-numbers-to-use shows the proper use as:

    $ apsbss current 9id:bss: 9-ID-B,C

but it only works for me when I omit the IOC prefix parameter:

    $ apsbss current 9-ID-B,C
prjemian commented 4 years ago

@jkirchman: Since apsbss does not use imaging (so no PIMS, scikit-image, or matplotlib), I'm guessing this happens due to other factors, most likely environment. If you are using apsbss from /APSshare/anaconda3/Bluesky/bin/apsbss (which should use /APSshare/anaconda3/Bluesky/bin/python`) and you get this error, then maybe it would help if you sourced a virtual environment first. Try this:

bash
export PYTHONPATH=
source /APSshare/anaconda3/Bluesky/bin/activate base
apsbss update 27id:dm:

Same problem?

prjemian commented 4 years ago

Also, do you have a PYTHONPATH environment variable set? Might help to unset that. Amending my previous comment for this.

prjemian commented 4 years ago

@jkirchman: Is there a PYTHONPATH variable?

prjemian commented 4 years ago

I'm also suspicious of package versions installed in /APSshare/anaconda3/Bluesky/. I'll compare with the bluesky_2020_5 environment for the USAXS instrument.

prjemian commented 4 years ago

Compare packages on /APSshare and the USAXS custom environment:

``` ============================= ================ ================== ================ ======================== ================== ================ package APSshare version APSshare build APSshare channel USAXS version USAXS build USAXS channel ============================= ================ ================== ================ ======================== ================== ================ _libgcc_mutex 0.1 main 0.1 main alabaster 0.7.12 pypi_0 pypi appdirs 1.4.3 py37h28b3542_0 1.4.3 py37h28b3542_0 aps-dm-api 3.2.2 py37_1 aps-anl-tag 3.2.2 py37_1 aps-anl-tag apstools 1.3.3 py_0 aps-anl-tag 1.3.3 py_0 aps-anl-tag area-detector-handlers 0.0.4 py_0 nsls2forge 0.0.4 py_0 nsls2forge asciitree 0.3.3 py_2 0.3.3 py_2 asteval 0.9.18 pyh5ca1d4c_0 nsls2forge 0.9.16 pyh5ca1d4c_0 conda-forge astroid 2.4.2 py37_0 2.4.2 py37_0 astropy 4.0.1.post1 pypi_0 pypi astroscrappy 1.0.8 pypi_0 pypi attrs 19.3.0 py_0 19.3.0 py_0 babel 2.8.0 pypi_0 pypi backcall 0.2.0 py_0 0.2.0 py_0 blas 1.0 mkl 1.0 mkl bleach 3.1.5 py_0 3.1.5 py_0 bluesky 1.6.4 py_0 nsls2forge 1.6.4 py_0 conda-forge bokeh 2.1.1 py37_0 2.1.1 py37_0 boltons 19.1.0 py_0 nsls2forge 20.2.0 pyh9f0ad1d_0 conda-forge brotlipy 0.7.0 py37h7b6447c_1000 ca-certificates 2020.6.24 0 2020.6.24 0 cachetools 4.1.0 py_1 4.1.0 py_1 cairo 1.14.12 h8948797_3 1.14.12 h8948797_3 certifi 2020.6.20 py37_0 2020.6.20 py37_0 cffi 1.14.0 py37h2e261b9_0 1.14.0 py37h2e261b9_0 chardet 3.0.4 py37_1003 3.0.4 py37_1003 click 7.1.2 py_0 7.1.2 py_0 cloudpickle 1.5.0 py_0 1.5.0 py_0 coloredlogs 10.0 py37_0 10.0 py37_0 conda 4.8.3 py37_0 conda-package-handling 1.6.1 py37h7b6447c_0 contexter 0.1.4 pypi_0 pypi coverage 5.0 py37h7b6447c_0 5.1 pypi_0 pypi coveralls 1.5.1 py37_0 cryptography 2.9.2 py37h1ba5d50_0 2.9.2 py37h1ba5d50_0 cycler 0.10.0 py37_0 0.10.0 py37_0 cytoolz 0.10.1 py37h7b6447c_0 0.10.1 py37h7b6447c_0 dask 2.20.0 py_0 2.20.0 py_0 dask-core 2.20.0 py_0 2.20.0 py_0 databroker 1.0.6 py_0 nsls2forge 1.0.6 py_0 nsls2forge dbus 1.13.16 hb2f20db_0 1.13.16 hb2f20db_0 decorator 4.4.2 py_0 4.4.2 py_0 defusedxml 0.6.0 py_0 0.6.0 py_0 deprecated 1.2.9 pypi_0 pypi 1.2.7 py_0 dill 0.3.1.1 pypi_0 pypi distlib 0.3.0 pypi_0 pypi distributed 2.20.0 py37_0 2.20.0 py37_0 docopt 0.6.2 py37_0 doct 1.1.0 py_0 nsls2forge 1.1.0 py_0 conda-forge docutils 0.16 pypi_0 pypi entrypoints 0.3 py37_0 0.3 py37_0 epics-base 7.0.4 pl526_1 aps-anl-tag 7.0.4 pl526_1 aps-anl-tag epicscorelibs 7.0.3.99.2.0 py37h68659b9_0 conda-forge 7.0.3.99.2.0 py37h68659b9_0 conda-forge event-model 1.15.2 py_0 nsls2forge 1.15.0 py_0 conda-forge expat 2.2.9 he6710b0_2 2.2.9 he6710b0_2 fabio 0.10.0 pypi_0 pypi fasteners 0.15 py_0 0.15 py_0 filelock 3.0.12 pypi_0 pypi fontconfig 2.13.0 h9420a91_0 2.13.0 h9420a91_0 freetype 2.10.2 h5ab3b9f_0 2.10.2 h5ab3b9f_0 fsspec 0.7.4 py_0 0.7.4 py_0 future 0.18.2 py37_1 gitdb 4.0.5 pypi_0 pypi gitpython 0.3.6 pypi_0 pypi glib 2.63.1 h5a9c865_0 2.63.1 h5a9c865_0 gobject-introspection 1.56.1 py37hbc4ca2d_2 1.56.1 py37hbc4ca2d_2 gsl 2.4 h14c3975_4 2.4 h14c3975_4 gst-plugins-base 1.14.0 hbbd80ab_1 1.14.0 hbbd80ab_1 gstreamer 1.14.0 hb453b48_1 1.14.0 hb453b48_1 h5py 2.10.0 py37h7918eee_0 2.10.0 py37h7918eee_0 happi 1.6.0 py_0 conda-forge 1.6.0 py_0 conda-forge hdf5 1.10.4 hb1b8bf9_0 1.10.4 hb1b8bf9_0 heapdict 1.0.1 py_0 1.0.1 py_0 helpdev 0.7.1 pypi_0 pypi historydict 1.2.3 py_0 nsls2forge 1.2.3 py_0 conda-forge hkl 5.0.0.2173 py37_6 lightsource2-tag 5.0.0.2173 py37h17bde16_1 nsls2forge hklpy 0.3.13 py37_0 lightsource2-tag 0.3.13 py37_0 lightsource2-tag humanfriendly 8.2 py37_0 8.2 py37_0 humanize 0.5.1 py37_1 0.5.1 py37_1 icu 58.2 he6710b0_3 58.2 he6710b0_3 idna 2.10 py_0 2.9 py_1 imagecodecs-lite 2019.12.3 py37h8f50634_0 conda-forge 2019.12.3 py37h8f50634_0 conda-forge imageio 2.8.0 pypi_0 pypi imagesize 1.2.0 pypi_0 pypi importlib-metadata 1.7.0 py37_0 1.7.0 py37_0 importlib_metadata 1.7.0 0 1.7.0 0 intake 0.6.0 py_0 0.6.0 py_0 intel-openmp 2020.1 217 2020.1 217 ipykernel 5.3.2 py37h5ca1d4c_0 5.3.2 py37h5ca1d4c_0 ipython 7.13.0 py37h5ca1d4c_0 7.13.0 py37h5ca1d4c_0 ipython_genutils 0.2.0 py37_0 0.2.0 py37_0 ipywidgets 7.5.1 py_0 7.5.1 py_0 isort 4.3.21 py37_0 4.3.21 py37_0 jedi 0.17.1 py37_0 0.17.1 py37_0 jinja2 2.11.2 py_0 2.11.2 py_0 jpeg 9b h024ee3a_2 9b h024ee3a_2 json5 0.9.5 py_0 jsonschema 3.2.0 py37_1 3.2.0 py37_1 jupyter 1.0.0 py37_7 1.0.0 py37_7 jupyter_client 6.1.6 py_0 6.1.6 py_0 jupyter_console 6.1.0 py_0 6.1.0 py_0 jupyter_core 4.6.3 py37_0 4.6.3 py37_0 jupyterlab 1.2.6 pyhf63ae98_0 jupyterlab_server 1.2.0 py_0 kiwisolver 1.2.0 py37hfd86e86_0 1.2.0 py37hfd86e86_0 lazy-object-proxy 1.4.3 py37h7b6447c_0 1.4.3 py37h7b6447c_0 lcms2 2.11 h396b838_0 2.11 h396b838_0 ld_impl_linux-64 2.33.1 h53a641e_7 2.33.1 h53a641e_7 libedit 3.1.20191231 h14c3975_1 3.1.20191231 h14c3975_1 libffi 3.2.1 hd88cf55_4 3.2.1 hd88cf55_4 libgcc-ng 9.1.0 hdf63c60_0 9.1.0 hdf63c60_0 libgfortran-ng 7.3.0 hdf63c60_0 7.3.0 hdf63c60_0 libiconv 1.15 h63c8f33_5 1.15 h63c8f33_5 libpng 1.6.37 hbc83047_0 1.6.37 hbc83047_0 libsodium 1.0.18 h7b6447c_0 1.0.18 h7b6447c_0 libstdcxx-ng 9.1.0 hdf63c60_0 9.1.0 hdf63c60_0 libtiff 4.1.0 h2733197_1 4.1.0 h2733197_1 libuuid 1.0.3 h1bed415_2 1.0.3 h1bed415_2 libxcb 1.14 h7b6447c_0 1.14 h7b6447c_0 libxml2 2.9.10 he19cac6_1 2.9.10 he19cac6_1 libxslt 1.1.34 hc22bd24_0 1.1.34 hc22bd24_0 lmfit 1.0.0 py_1 nsls2forge 1.0.0 py_0 conda-forge locket 0.2.0 py37_1 0.2.0 py37_1 lxml 4.5.0 py37hefd8a0e_0 4.5.0 py37hefd8a0e_0 lz4-c 1.9.2 he6710b0_0 1.9.2 he6710b0_0 markupsafe 1.1.1 py37h14c3975_1 1.1.1 py37h14c3975_1 matplotlib 3.2.2 0 3.2.2 0 matplotlib-base 3.2.2 py37hef1b27d_0 3.2.2 py37hef1b27d_0 mccabe 0.6.1 py37_1 0.6.1 py37_1 mistune 0.8.4 py37h14c3975_1001 0.8.4 py37h14c3975_1001 mkl 2020.1 217 2020.1 217 mkl-service 2.3.0 py37he904b0f_0 2.3.0 py37he904b0f_0 mkl_fft 1.1.0 py37h23d657b_0 1.1.0 py37h23d657b_0 mkl_random 1.1.1 py37h0573a6f_0 1.1.1 py37h0573a6f_0 mongoquery 1.1.0 py_0 nsls2forge 1.3.5 py_0 conda-forge monotonic 1.5 py_0 1.5 py_0 msgpack-numpy 0.4.4.3 py_0 0.4.4.3 py_0 msgpack-python 1.0.0 py37hfd86e86_1 1.0.0 py37hfd86e86_1 nbconvert 5.6.1 py37_1 5.6.1 py37_1 nbformat 5.0.7 py_0 5.0.7 py_0 ncurses 6.2 he6710b0_1 6.2 he6710b0_1 networkx 2.4 py_1 2.4 py_1 nose 1.3.7 pypi_0 pypi notebook 6.0.3 py37_0 6.0.3 py37_0 numcodecs 0.6.4 py37he6710b0_0 0.6.4 py37he6710b0_0 numexpr 2.7.1 pypi_0 pypi numpy 1.18.5 py37ha1c710e_0 1.18.5 py37ha1c710e_0 numpy-base 1.18.5 py37hde5b4d6_0 1.18.5 py37hde5b4d6_0 olefile 0.46 py37_0 0.46 py37_0 openssl 1.1.1g h7b6447c_0 1.1.1g h7b6447c_0 ophyd 1.5.2 py_0 nsls2forge 1.5.1 py_0 conda-forge packaging 20.4 py_0 20.4 py_0 pandas 1.0.5 py37h0573a6f_0 1.0.5 py37h0573a6f_0 pandoc 2.10 0 2.10 0 pandocfilters 1.4.2 py37_1 1.4.2 py37_1 parso 0.7.0 py_0 0.7.0 py_0 partd 1.1.0 py_0 1.1.0 py_0 pathlib 1.0.1 pypi_0 pypi pcre 8.44 he6710b0_0 8.44 he6710b0_0 perl 5.26.2 h14c3975_0 5.26.2 h14c3975_0 pexpect 4.8.0 py37_1 4.8.0 py37_1 pickleshare 0.7.5 py37_1001 0.7.5 py37_1001 pillow 7.2.0 py37hb39fc2d_0 7.2.0 py37hb39fc2d_0 pims 0.5 py_1 0.5 py_1 pip 20.1.1 py37_1 20.1.1 py37_1 pixman 0.40.0 h7b6447c_0 0.40.0 h7b6447c_0 prettytable 0.7.2 py_0 nsls2forge 0.7.2 py_3 conda-forge prometheus_client 0.8.0 py_0 0.8.0 py_0 prompt-toolkit 3.0.5 py_0 3.0.5 py_0 prompt_toolkit 3.0.5 0 3.0.5 0 psutil 5.7.0 py37h7b6447c_0 5.7.0 py37h7b6447c_0 ptyprocess 0.6.0 py37_0 0.6.0 py37_0 punx 0.2.2 pypi_0 pypi 0.2.5 pypi_0 pypi pvview 1.0.0 py_0 aps-anl-tag 1.0.0 py_0 aps-anl-tag pycairo 1.19.1 py37h2a1e443_0 1.19.1 py37h2a1e443_0 pycosat 0.6.3 py37h7b6447c_0 pycparser 2.20 py_2 2.20 py_2 pydm 1.10.1 pyh9f0ad1d_0 conda-forge 1.10.1 pyh9f0ad1d_0 conda-forge pyepics 3.4.2 py37_0 nsls2forge 3.4.2 py37hc8dfbb8_0 conda-forge pyfai 0.19.0 pypi_0 pypi pygithub 1.51 pypi_0 pypi 1.43.6 py37hc8dfbb8_1 conda-forge pygments 2.6.1 py_0 2.6.1 py_0 pygobject 3.30.5 py37hfc86313_1 conda-forge 3.30.5 py37hfc86313_1 conda-forge pyjwt 1.7.1 pypi_0 pypi 1.7.1 py37_0 pylint 2.5.0 py37_1 2.5.0 py37_1 pymongo 3.9.0 py37he6710b0_0 3.9.0 py37he6710b0_0 pyopenssl 19.1.0 py_1 19.1.0 py_1 pyparsing 2.4.7 py_0 2.4.7 py_0 pypi-publisher 0.0.4 pypi_0 pypi pyqt 5.9.2 py37h05f1152_2 5.9.2 py37h05f1152_2 pyqt5 5.14.2 pypi_0 pypi pyqt5-sip 12.7.2 pypi_0 pypi pyqtgraph 0.11.0 py_0 0.11.0 py_0 pyresttable 2020.0.2 py_0 aps-anl-tag 2020.0.2 py_0 aps-anl-tag pyrsistent 0.16.0 py37h7b6447c_0 0.16.0 py37h7b6447c_0 pysocks 1.7.1 py37_1 1.7.1 py37_1 python 3.7.7 hcf32534_0_cpython 3.7.7 hcf32534_0_cpython python-dateutil 2.8.1 py_0 2.8.1 py_0 python_abi 3.7 1_cp37m conda-forge 3.7 1_cp37m conda-forge pytz 2020.1 py_0 2020.1 py_0 pywavelets 1.1.1 pypi_0 pypi pyyaml 5.3.1 py37h7b6447c_1 5.3.1 py37h7b6447c_1 pyzmq 19.0.1 py37he6710b0_1 19.0.1 py37he6710b0_1 qdarkstyle 2.8.1 pypi_0 pypi qt 5.9.7 h5867ecd_1 5.9.7 h5867ecd_1 qtconsole 4.7.5 py_0 4.7.5 py_0 qtmodern 0.2.0 pypi_0 pypi qtpy 1.9.0 py_0 1.9.0 py_0 readline 8.0 h7b6447c_0 8.0 h7b6447c_0 requests 2.24.0 py_0 2.23.0 py37_0 ruamel_yaml 0.15.87 py37h7b6447c_1 sasmodels 1.0.2 pypi_0 pypi scandir 1.10.0 pypi_0 pypi scikit-beam 0.0.21 pypi_0 pypi scikit-image 0.16.2 pypi_0 pypi scipy 1.5.0 py37h0b6359f_0 1.5.0 py37h0b6359f_0 send2trash 1.5.0 py37_0 1.5.0 py37_0 setuptools 49.2.0 py37_0 49.2.0 py37_0 signalslot 0.1.2 pypi_0 pypi silx 0.12.0 pypi_0 pypi simplejson 3.17.0 py37h7b6447c_0 3.17.0 py37h7b6447c_0 sip 4.19.8 py37hf484d3e_0 4.19.8 py37hf484d3e_0 six 1.15.0 py_0 1.15.0 py_0 slicerator 1.0.0 py_0 1.0.0 py_0 smmap 3.0.4 pypi_0 pypi snowballstemmer 2.0.0 pypi_0 pypi sortedcontainers 2.2.2 py_0 2.2.2 py_0 spec2nexus 2021.1.7 h39e3cac_0 aps-anl-tag 2021.1.7 h39e3cac_0 aps-anl-tag sphinx 3.0.3 pypi_0 pypi sphinx-rtd-theme 0.4.3 pypi_0 pypi sphinxcontrib-applehelp 1.0.2 pypi_0 pypi sphinxcontrib-devhelp 1.0.2 pypi_0 pypi sphinxcontrib-htmlhelp 1.0.3 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.3 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.4 pypi_0 pypi sqlite 3.32.3 h62c20be_0 3.32.3 h62c20be_0 stdlogpj 1.0.3 py_0 aps-anl-tag 1.0.3 py_0 aps-anl-tag suitcase-mongo 0.2.0 py_0 nsls2forge 0.2.0 py_0 nsls2forge suitcase-msgpack 0.2.4 py_0 nsls2forge 0.2.4 py_0 nsls2forge suitcase-utils 0.5.0 py_2 nsls2forge 0.5.0 py_2 nsls2forge super_state_machine 2.0.2 py_0 nsls2forge 2.0.2 py37_1000 conda-forge tblib 1.6.0 py_0 1.6.0 py_0 terminado 0.8.3 py37_0 0.8.3 py37_0 testpath 0.4.4 py_0 0.4.4 py_0 tifffile 0.15.1 py37heb32a55_2 0.15.1 py37heb32a55_2 tk 8.6.10 hbc83047_0 8.6.10 hbc83047_0 toml 0.10.1 py_0 0.10.1 py_0 toolz 0.10.0 py_0 0.10.0 py_0 tornado 6.0.4 py37h7b6447c_1 6.0.4 py37h7b6447c_1 tqdm 4.47.0 py_0 4.47.0 py_0 traitlets 4.3.3 py37_0 4.3.3 py37_0 typed-ast 1.4.1 py37h7b6447c_0 1.4.1 py37h7b6447c_0 typing_extensions 3.7.4.2 py_0 3.7.4.2 py_0 tzlocal 2.0.0 py37_0 2.0.0 py37_0 uncertainties 3.1.2 py_0 nsls2forge 3.1.4 py_0 conda-forge urllib3 1.25.9 py_0 1.25.8 py37_0 virtualenv 20.0.20 pypi_0 pypi wcwidth 0.2.5 py_0 0.2.5 py_0 weakrefmethod 1.0.3 pypi_0 pypi webencodings 0.5.1 py37_1 0.5.1 py37_1 wheel 0.34.2 py37_0 0.34.2 py37_0 widgetsnbextension 3.5.1 py37_0 3.5.1 py37_0 wrapt 1.11.2 py37h7b6447c_0 1.11.2 py37h7b6447c_0 xarray 0.16.0 py_0 0.16.0 py_0 xicam 0+untagged.1367.gcfad222 pypi_0 pypi xicam-saxs 0.1.0 pypi_0 pypi xicam-xpcs 1.1.0 pypi_0 pypi xlrd 1.2.0 py37_0 1.2.0 py37_0 xz 5.2.5 h7b6447c_0 5.2.5 h7b6447c_0 yaml 0.2.5 h7b6447c_0 0.2.5 h7b6447c_0 zarr 2.3.2 py_0 2.3.2 py_0 zeromq 4.3.2 he6710b0_2 4.3.2 he6710b0_2 zict 2.0.0 py_0 2.0.0 py_0 zipp 3.1.0 py_0 3.1.0 py_0 zlib 1.2.11 h7b6447c_3 1.2.11 h7b6447c_3 zstd 1.4.5 h0b5b093_0 1.4.5 h0b5b093_0 ============================= ================ ================== ================ ======================== ================== ================ ```
prjemian commented 4 years ago

Notably:

USAXS has scikit-beam 0.0.21 and scikit-image 0.16.2 installed via pip, APSshare has neither. This is likely the problem.

prjemian commented 4 years ago

Will pip install scikit-beam and pip install scikit-image in /APSshare/anaconda3/Bluesky

prjemian commented 4 years ago

BTW, here is the code to compare two conda environments (that generated the table above):


from collections import defaultdict
import os
import pyRestTable
# import sys

SUITE = dict(
    # These files contain the output of ``conda list``
    APSshare="APSshare.txt", 
    USAXS="USAXS.txt"
    )

def main():
    path = os.path.dirname(__file__)
    db = defaultdict(lambda: {})
    for env, fname in SUITE.items():
        with open(os.path.join(path, fname), "r") as fp:
            for line in fp.readlines():
                if line.startswith("#"):
                    continue
                parts = line.strip().split()
                if len(parts) not in (3, 4):
                    continue
                elif len(parts) == 3:
                    parts.append("")
                package, version, build, channel = parts
                db[package][env] = (version, build, channel)

    table = pyRestTable.Table()
    table.addLabel("package")
    for env in sorted(SUITE.keys()):
        table.addLabel(f"{env} version")
        table.addLabel(f"{env} build")
        table.addLabel(f"{env} channel")
    for package, pkg_info in sorted(db.items()):
        row = [package,]
        for env in sorted(SUITE.keys()):
            if env in pkg_info:
                row += pkg_info[env]
            else:
                row += ["", "", ""]
        table.addRow(row)
    print(table)

if __name__ == "__main__":
    main()
prjemian commented 4 years ago

open new issue if problem recurs