troubles regarding the use of jupyter notebooks

[jimoreira]

hi, i'm getting familiar with this tool. I'm using it without a problem from the windows prompt or iphyton but when a try to open a jupyter notebook it keeps freezing. I can see that the implementation has some years by now, could that be the problem? I could be important to use py6s from inside them to be able to use their outputs all together on a script. thanks.

[robintw]

I use Py6S from a Jupyter notebook regularly and have no problems.

Can you get Jupyter notebooks working when you're not using Py6S? What I mean is, does the notebook freeze as soon as you open it, or does it only freeze when you import Py6S?

If it freezes as soon as you open it, or freezes even if you don't use Py6S then I suggest contacting the Jupyter team as it sounds like it could be a Jupyter problem. I'll be happy to try and help if it is a Py6S-specific issue - although it's not a problem I've experienced.

[jimoreira]

It freezes and has some problem with the kernel I think, it open the first time, when a try to make de testing code, and kept thinking and get stucked. And then I try again and don't even open. I'm using it for other applications with no problems. I had to mention that I need to install it to the environment first, via anaconda, because it wasn't set. Thanks for your quick reply!

[robintw]

That's really strange, I've never come across this problem before.

Can you send me the text that is displayed in the terminal window that you launched the jupyter notebook from, while it's running? (That's the place that you typed jupyter notebook in). There might be some errors shown there. If not then I'm not really sure where to go from there...

[jimoreira]

here it goes:

[I 16:19:34.728 NotebookApp] The port 8888 is already in use, trying another port.
[I 16:19:34.764 NotebookApp] Serving notebooks from local directory: C:\Users\javie
[I 16:19:34.765 NotebookApp] The Jupyter Notebook is running at:
[I 16:19:34.765 NotebookApp] http://localhost:8889/?token=887bd97cb7c09b1ff1c8c39aadd0d2a6f26bf5e05ac3b5e4
[I 16:19:34.765 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 16:19:34.768 NotebookApp]

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8889/?token=887bd97cb7c09b1ff1c8c39aadd0d2a6f26bf5e05ac3b5e4
[I 16:19:35.008 NotebookApp] Accepting one-time-token-authenticated connection from ::1

[robintw]

Hmm, that all looks normal. It doesn't look like the kernel has crashed or anything (I assume you imported Py6S before sending me that output?). I really can't think what might be causing this. I'll think about it for a few days and see if I can come up with any ideas.

In the meantime you could try creating a new conda environment and trying it again. Installing everything 'in one go' should make sure that everything is compatible. For example:

conda create -n test-environment -c conda-forge python=3 jupyter py6s

should work, and gives you an environment test-environment that you can activate (with source activate test-environment) and then try jupyter in.

[jimoreira]

good idea, but still the same. `(test-environment) C:\Users\javie>jupyter notebook [I 18:02:16.225 NotebookApp] Serving notebooks from local directory: C:\Users\javie [I 18:02:16.226 NotebookApp] The Jupyter Notebook is running at: [I 18:02:16.226 NotebookApp] http://localhost:8888/?token=bf68a817b5f6e69b25af577b2edb1ae2c3d05a2ed4d3d918 [I 18:02:16.226 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 18:02:16.228 NotebookApp]

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://localhost:8888/?token=bf68a817b5f6e69b25af577b2edb1ae2c3d05a2ed4d3d918

[I 18:02:16.410 NotebookApp] Accepting one-time-token-authenticated connection from ::1`

I´m running this from anaconda prompt, since in windows prompt when i try to activate, it doesnt recognize the command. Because i have tried the long installation way in order to could use the repository ([https://github.com/samsammurphy/gee-atmcorr-S2.git]) .

Although if y run python in that windows prompt and try to do the test this happens 6S wrapper script by Robin Wilson Using 6S located at C:\Users\javie\build\6SV\1.1\6SV1.1\sixsV1.1 Running 6S using a set of test parameters Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\javie\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Py6S\sixs.py", line 355, in test print("6sV version: %s" % (test.outputs.version)) File "C:\Users\javie\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Py6S\outputs.py", line 92, in __getattr__ raise OutputParsingError("The specifed output variable does not exist.") Py6S.sixs_exceptions.OutputParsingError: The specifed output variable does not exist.

[robintw]

Interesting.

What happens if you run python in the Anaconda prompt and do the test? (Following the instructions at https://py6s.readthedocs.io/en/latest/installation.html#testing-py6s).

Also, can you send me the full code you're using in the notebook to do the testing? (You can attach the .ipynb file if that's easiest).

[jimoreira]

just a comment before answering. My final interest is to run this to perform atm calib. in Rapideye and if it possible on PlanetLabs images. I saw that Rapideye is available. If you have any script to perform this i will appreciate it.

If it isn´t inside de env it doesn´t find the module. The code inside the .ipynb is just; `>>> from Py6S import *

SixS.test()` the same code that works fine if i launch python on the anaconda prompt inside the correct env. thanks so much for your interest and your help!

[jimoreira]

2 possible ways.

1. Open the anaconda prompt and activate the env. launch jupyter notebook. Result, success opening the notebook but the test doesn't run. ("from Py6S import *....")
2. Open anaconda Gui and when you try to launch jupyter notebook is not available, so you install it, and when you launch it, the freezing part mentioned above occurs.

As i mentioned running the test inside anaconda prompt, inside env, and calling python directly, works fine. The issue is with the interaction between the env and jupyter notebook, could be some issue with the version?

[robintw]

I really can't understand why you're having a problem using this in Jupyter notebooks. If it works in the Python console then it should work in the notebook.

Could you send me the output of conda env export (run it once you've activated the environment). I won't be able to test it directly as I don't have a Windows machine to test on, but it's possible I might be able to see something wrong in it.

[jimoreira]

name: Py6S-env
channels:
  - conda-forge
  - defaults
dependencies:
  - cycler=0.10.0=py_1
  - freetype=2.8.1=vc14_0
  - icu=58.2=vc14_0
  - jpeg=9b=vc14_2
  - kiwisolver=1.0.1=py36he980bc4_1002
  - libpng=1.6.34=vc14_0
  - matplotlib=2.2.2=py36_1
  - mkl_fft=1.0.6=py36_0
  - mkl_random=1.0.1=py36_0
  - pip=18.1=py36_1000
  - py6s=1.7.2=py_0
  - pyparsing=2.2.2=py_0
  - pyqt=5.6.0=py36h764d66f_7
  - pysolar=0.6=py36_0
  - python=3.6.6=he025d50_0
  - python-dateutil=2.7.3=py_0
  - pytz=2018.5=py_0
  - qt=5.6.2=vc14_1
  - setuptools=40.4.3=py36_0
  - sip=4.18.1=py36h6538335_0
  - six=1.11.0=py36_1001
  - sixs=1.1=h8933c1f_1
  - tornado=5.1.1=py36hfa6e2cd_1000
  - vc=14=0
  - vs2015_runtime=14.0.25420=0
  - wheel=0.32.2=py36_0
  - wincertstore=0.2=py36_1002
  - zlib=1.2.11=vc14_0
  - backcall=0.1.0=py36_0
  - blas=1.0=mkl
  - bleach=3.0.2=py36_0
  - ca-certificates=2018.03.07=0
  - certifi=2018.10.15=py36_0
  - colorama=0.4.0=py36_0
  - decorator=4.3.0=py36_0
  - entrypoints=0.2.3=py36_2
  - icc_rt=2017.0.4=h97af966_0
  - intel-openmp=2019.0=118
  - ipykernel=5.1.0=py36h39e3cac_0
  - ipython=7.0.1=py36h39e3cac_0
  - ipython_genutils=0.2.0=py36h3c5d0ee_0
  - jedi=0.13.1=py36_0
  - jinja2=2.10=py36_0
  - jsonschema=2.6.0=py36h7636477_0
  - jupyter_client=5.2.3=py36_0
  - jupyter_core=4.4.0=py36_0
  - libsodium=1.0.16=h9d3ae62_0
  - m2w64-gcc-libgfortran=5.3.0=6
  - m2w64-gcc-libs=5.3.0=7
  - m2w64-gcc-libs-core=5.3.0=7
  - m2w64-gmp=6.1.0=2
  - m2w64-libwinpthread-git=5.0.0.4634.697f757=2
  - markupsafe=1.0=py36hfa6e2cd_1
  - mistune=0.8.3=py36hfa6e2cd_1
  - mkl=2019.0=118
  - msys2-conda-epoch=20160418=1
  - nbconvert=5.3.1=py36_0
  - nbformat=4.4.0=py36h3a5bc1b_0
  - numpy=1.15.3=py36ha559c80_0
  - numpy-base=1.15.3=py36h8128ebf_0
  - openssl=1.0.2p=hfa6e2cd_0
  - pandoc=2.2.3.2=0
  - pandocfilters=1.4.2=py36_1
  - parso=0.3.1=py36_0
  - pickleshare=0.7.5=py36_0
  - prometheus_client=0.4.2=py36_0
  - prompt_toolkit=2.0.6=py36_0
  - pygments=2.2.0=py36hb010967_0
  - pywinpty=0.5.4=py36_0
  - pyzmq=17.1.2=py36hfa6e2cd_0
  - scipy=1.1.0=py36h4f6bf74_1
  - send2trash=1.5.0=py36_0
  - simplegeneric=0.8.1=py36_2
  - terminado=0.8.1=py36_1
  - testpath=0.4.2=py36_0
  - traitlets=4.3.2=py36h096827d_0
  - wcwidth=0.1.7=py36h3d5aa90_0
  - webencodings=0.5.1=py36_1
  - winpty=0.4.3=4
  - zeromq=4.2.5=he025d50_1
prefix: C:\Users\javie\Anaconda3\envs\Py6S-env```

[robintw]

It looks like you might not have the jupyter notebook properly installed in that environment. There is no notebook package listed in that output, whereas I do have the notebook package installed.

Try activating the environment and running conda install jupyter and see if it offers to install anything or do any changes. If not, try conda install notebook.

Once you've done those, try loading the jupyter notebook again (activate the environment and then run jupyter notebook) and see if it works. If not, then send me the output of the following commands:

where jupyter (run this on the command line after activating the environment) import sys; print(sys.path) (run this in the first cell of a new notebook, after activating the environment and running jupyter notebook)

Hopefully one of these will help us get to the bottom of this!

[jimoreira]

ok, i had tried the "conda install jupyter" and the output is shown is this screenshot; Then i launch jupyter notebook, and again, when trying to open a new python notebook, it freezes (not even ctrl+c works).

So, "where jupyter" ''' (Py6S-env) C:\Users\javie>where jupyter C:\Users\javie\Anaconda3\envs\py6s-env\Scripts\jupyter.exe C:\Users\javie\Anaconda3\Scripts\jupyter.exe C:\Users\javie\AppData\Local\Programs\Python\Python36-32\Scripts\jupyter.exe'''

for the line "import sys; print(sys.path) i'm not able to follow your instructions, since, i'm unable to open a notebook from inside this env (the freenzing part..).

2 curious thing about this, first, the jupyter "home" page its launched, but i cannot change directory, just work from inside the path where i launch it. Second, i keep trying to open new notebooks, and they didn't work, but the notebook is created, and i can't delete them. I have a few by now.

[robintw]

I'm a bit confused: does the Jupyter notebook always freeze when you open it, or just when you try and import Py6S? I thought it was just when you imported Py6S.

If the Jupyter notebook always freezes when you open it, then I think you'll need to contact the Jupyter team for support - they will be able to help you better than I can.

[jimoreira]

It freezes when i'm opening from inside the Py6S-env. Other ways it work fine. but the freezing is before importing the Py6S, it's when i'm trying to open them. The issue is with the env in interaction with jupyter notebook.

Is it ok to run 6S from python 3.6?

[robintw]

Ok, I think this is a Jupyter problem then. Any package simply being installed in an environment shouldn't stop the jupyter notebook opening. I'd suggest reporting a possible bug to the Jupyter team and see if they can work out what the problem is.

You can try fiddling around with versions of different packages and creating environments with and without specific packages, but I suspect it is a Jupyter or a Jupyter packaging issue.

Yes it is ok to run Py6S from Python 3.6.

[jimoreira]

ok, thank you very much for your time. You had dedicated more than enough time to this issue, i appreciate it. I will find the way to use it eventually.