search

anaconda / nb_conda

Conda environment and package access extension from within Jupyter
BSD 3-Clause "New" or "Revised" License
140 stars 32 forks source link

unhandled AttributeError on listing current env #39

Closed timsnyder closed 7 years ago

timsnyder commented 7 years ago

When trying to use the Kernel->Conda Packages menu entry provided by nb_conda, I'm encountering the following unhandled exception:

[E 17:32:30.477 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File "/sarc/spa/users/tsnyder/.conda/envs/cat-1/lib/python3.5/site-packages/notebook/base/handlers.py", line 457, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File "/sarc/spa/users/tsnyder/.conda/envs/cat-1/lib/python3.5/site-packages/nb_conda/handlers.py", line 62, in get
        self.finish(json.dumps(self.env_manager.env_packages(env)))
      File "/sarc/spa/users/tsnyder/.conda/envs/cat-1/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in env_packages
        "packages": [pkg_info(package) for package in data]
      File "/sarc/spa/users/tsnyder/.conda/envs/cat-1/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in <listcomp>
        "packages": [pkg_info(package) for package in data]
      File "/sarc/spa/users/tsnyder/.conda/envs/cat-1/lib/python3.5/site-packages/nb_conda/envmanager.py", line 16, in pkg_info
        name, version, build = s.rsplit('-', 2)
    AttributeError: 'dict' object has no attribute 'rsplit'

cat-1.yaml.txt is the output of conda env export for my environment. Before filing this ticket, I also had @groutr repro the behavior.

ijstokes commented 7 years ago

Confirmed. I get the same:

[I 22:35:32.923 NotebookApp] Kernel started: e3c331a9-33c5-4074-a2db-4b3998a8abde
[E 22:35:53.548 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/notebook/base/handlers.py", line 457, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/handlers.py", line 62, in get
        self.finish(json.dumps(self.env_manager.env_packages(env)))
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in env_packages
        "packages": [pkg_info(package) for package in data]
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in <listcomp>
        "packages": [pkg_info(package) for package in data]
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 16, in pkg_info
        name, version, build = s.rsplit('-', 2)
    AttributeError: 'dict' object has no attribute 'rsplit'
[E 22:35:53.552 NotebookApp] {
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:50.0) Gecko/20100101 Firefox/50.0",
      "Host": "localhost:8888",
      "Referer": "http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=conda-root-py",
      "Connection": "keep-alive",
      "Accept-Encoding": "gzip, deflate",
      "Accept-Language": "en,en-US;q=0.5",
      "X-Requested-With": "XMLHttpRequest",
      "Accept": "application/json, text/javascript, */*; q=0.01"
    }
[E 22:35:53.553 NotebookApp] 500 GET /conda/environments/root?_=1484624132520 (::1) 692.02ms referer=http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=conda-root-py
[E 22:35:54.228 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/notebook/base/handlers.py", line 457, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/handlers.py", line 62, in get
        self.finish(json.dumps(self.env_manager.env_packages(env)))
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in env_packages
        "packages": [pkg_info(package) for package in data]
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in <listcomp>
        "packages": [pkg_info(package) for package in data]
      File "/Users/ijstokes/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 16, in pkg_info
        name, version, build = s.rsplit('-', 2)
    AttributeError: 'dict' object has no attribute 'rsplit'
[E 22:35:54.229 NotebookApp] {
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:50.0) Gecko/20100101 Firefox/50.0",
      "Host": "localhost:8888",
      "Referer": "http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=conda-root-py",
      "Connection": "keep-alive",
      "Accept-Encoding": "gzip, deflate",
      "Accept-Language": "en,en-US;q=0.5",
      "X-Requested-With": "XMLHttpRequest",
      "Accept": "application/json, text/javascript, */*; q=0.01"
    }
[E 22:35:54.229 NotebookApp] 500 GET /conda/environments/root?_=1484624132521 (::1) 675.43ms referer=http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=conda-root-py
ijstokes commented 7 years ago

@csoja: this may be one we want to get fixed quickly before 4.3 is finalized.

@kalefranz it looks like the pkg_info JSON object has changed from a string to a dict. Is that correct? If so, another example of how a stable Conda API would help here.

csoja commented 7 years ago

@koverholt @damianavila

kalefranz commented 7 years ago

I think this is related to https://github.com/ContinuumIO/navigator. It's an egregious leak of conda's internal abstractions. I didn't realize it was all just being barfed out as json, and that's what the interface was relying on then.

The problem we've solved internally in conda relates to a line of code

name, version, build = s.rsplit('-', 2)

apparently in nb_conda and a ton of other places. This string parsing of the package name is NOT reliable, and it will become more unreliable over time. I'm going to try to reconstruct the json output for the interface in 4.3 to what I think are the important pieces from 4.2.

damianavila commented 7 years ago

@timsnyder @ijstokes which version of conda are you using?

This string parsing of the package name is NOT reliable, and it will become more unreliable over time

@kalefranz what do you recommend us to use instead? Thanks!

MayankShah1 commented 7 years ago

@damianavila I'm facing the same problem when I update my conda to 4.3.5 but it's working fine with my downloaded conda version of 4.2.9

csoja commented 7 years ago

conda v4.3.6 should fix this issue - it is available in the conda-canary channel, if you'd like to give it a try to confirm that it fixes the issue. https://anaconda.org/conda-canary/conda conda install -c conda-canary conda=4.3.6

damianavila commented 7 years ago

conda v4.3.6 should fix this issue - it is available in the conda-canary channel,

Thanks for the info @csoja!

kalefranz commented 7 years ago

4.3.6 didn't, but 4.3.7 should. Should be out in another 36-48 hours.

On Jan 19, 2017, at 3:10 PM, Damian Avila notifications@github.com wrote:

conda v4.3.6 should fix this issue - it is available in the conda-canary channel,

Thanks for the info @csoja!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

kalefranz commented 7 years ago

4.3.7 is out now.

tikwanleap commented 7 years ago

I have conda version 4.3.8 installed and I'm still getting this error.

kalefranz commented 7 years ago

@tikwanleap Can you post your stack trace, even if it's exactly the same? Also please provide conda info.

kalefranz commented 7 years ago

Conda is working as expected, but I've created an additional test. https://github.com/conda/conda/pull/4480

I'm making a PR here that brings nb_conda up-to-date with conda 4.3.

kalefranz commented 7 years ago

https://github.com/Anaconda-Platform/nb_conda/pull/42

tikwanleap commented 7 years ago

stack trace:

[E 14:55:30.899 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File "C:\Program Files\Anaconda3\lib\site-packages\notebook\base\handlers.py", line 457, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File "C:\Program Files\Anaconda3\lib\site-packages\nb_conda\handlers.py", line 62, in get
        self.finish(json.dumps(self.env_manager.env_packages(env)))
      File "C:\Program Files\Anaconda3\lib\site-packages\nb_conda\envmanager.py", line 124, in env_packages
        "packages": [pkg_info(package) for package in data]
      File "C:\Program Files\Anaconda3\lib\site-packages\nb_conda\envmanager.py", line 124, in <listcomp>
        "packages": [pkg_info(package) for package in data]
      File "C:\Program Files\Anaconda3\lib\site-packages\nb_conda\envmanager.py", line 16, in pkg_info
        name, version, build = s.rsplit('-', 2)
    AttributeError: 'dict' object has no attribute 'rsplit'

conda info:

Current conda install:

               platform : win-64
          conda version : 4.3.8
       conda is private : False
      conda-env version : 4.3.8
    conda-build version : 2.0.2
         python version : 3.5.2.final.0
       requests version : 2.12.4
       root environment : C:\Program Files\Anaconda3  (read only)
    default environment : C:\Program Files\Anaconda3
       envs directories : C:\Users\Sam\.conda\envs
                          C:\Program Files\Anaconda3\envs
          package cache : C:\Users\Sam\.conda\envs\.pkgs
                          C:\Program Files\Anaconda3\pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/win-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/win-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/win-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://repo.continuum.io/pkgs/msys2/win-64
                          https://repo.continuum.io/pkgs/msys2/noarch
            config file : None
           offline mode : False
             user-agent : conda/4.3.8 requests/2.12.4 CPython/3.5.2 Windows/10 Windows/10.0.14393
tikwanleap commented 7 years ago

@kalefranz Thanks for fixing this bug so quickly in PR #42!

May I propose an alternative version that avoids using the hasattr() function?

I read this article that advises against using hasattr(): https://hynek.me/articles/hasattr/

I have tested it locally on my computer and it also fixes the bug.

def pkg_info(s):
    try:
        name, version, build = s.rsplit('-', 2)
    except AttributeError:
        name    = s['name']
        version = s['version']
        build   = s.get('build_string') or s['build']

    return {
        'name':    name,
        'version': version,
        'build':   build
    }
kalefranz commented 7 years ago

@tikwanleap I updated the PR with different code. The exceptional case now should actually be the string splitting. Apparently, calling __getitem__ on a string raises

TypeError: string indices must be integers

in both python2 and python3.

tikwanleap commented 7 years ago

@kalefranz Looks good! Thanks!

jkoelling commented 7 years ago

I'm also still getting this error in 4.3.8.

Stack trace:

[E 15:02:26.024 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File "/Users/jkoellin/anaconda/lib/python3.5/site-packages/notebook/base/handlers.py", line 503, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File "/Users/jkoellin/anaconda/lib/python3.5/site-packages/nb_conda/handlers.py", line 62, in get
        self.finish(json.dumps(self.env_manager.env_packages(env)))
      File "/Users/jkoellin/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in env_packages
        "packages": [pkg_info(package) for package in data]
      File "/Users/jkoellin/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 124, in <listcomp>
        "packages": [pkg_info(package) for package in data]
      File "/Users/jkoellin/anaconda/lib/python3.5/site-packages/nb_conda/envmanager.py", line 16, in pkg_info
        name, version, build = s.rsplit('-', 2)
    AttributeError: 'dict' object has no attribute 'rsplit'

conda info:

Current conda install:

               platform : osx-64
          conda version : 4.3.8
       conda is private : False
      conda-env version : 4.3.8
    conda-build version : 2.1.3
         python version : 3.5.2.final.0
       requests version : 2.12.4
       root environment : /Users/jkoellin/anaconda  (writable)
    default environment : /Users/jkoellin/anaconda
       envs directories : /Users/jkoellin/anaconda/envs
          package cache : /Users/jkoellin/anaconda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/osx-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/osx-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/osx-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False
             user-agent : conda/4.3.8 requests/2.12.4 CPython/3.5.2 Darwin/16.3.0 OSX/10.12.2
                UID:GID : 501:20

I've also tried 4.3.9 using conda install -c conda-canary conda=4.3.9 to no avail. Stack trace is the same. conda info looks like this:

Current conda install:

               platform : osx-64
          conda version : 4.3.9
       conda is private : False
      conda-env version : 4.3.9
    conda-build version : 2.1.3
         python version : 3.5.2.final.0
       requests version : 2.12.4
       root environment : /Users/jkoellin/anaconda  (writable)
    default environment : /Users/jkoellin/anaconda
       envs directories : /Users/jkoellin/anaconda/envs
          package cache : /Users/jkoellin/anaconda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/osx-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/osx-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/osx-64
                          https://repo.continuum.io/pkgs/pro/noarch
            config file : None
           offline mode : False
             user-agent : conda/4.3.9 requests/2.12.4 CPython/3.5.2 Darwin/16.3.0 OSX/10.12.2
                UID:GID : 501:20
jkoelling commented 7 years ago

There is a question regarding this bug on stackoverflow. I've posted my temporary solution using the branch containing the fix there. Have I missed a more official channel to pull in the fix?

timsnyder commented 7 years ago

@damianavila sorry I missed your question from a few weeks ago. Looking at the env export I attached in the original bug description, I had whatever was pulled in by 'anaconda 4.2.0'. I don't see the conda package itself in the env export output. Let me know if you want me to take a closer look at the conda I had installed when I opened this.

BTW, I'm also around at AnacondaCON today and tomorrow morning.

damianavila commented 7 years ago

Have I missed a more official channel to pull in the fix?

Nop, this is the official one but a little bit behind because of other priorities... sorry about that. Soon we will have some time to address all these issues.

damianavila commented 7 years ago

@damianavila sorry I missed your question from a few weeks ago

No problem, the issue is identified. We just need to merge things and wrap up a release. Hopefully this will happen in the next few weeks.

timsnyder commented 7 years ago

Just a head's up... since my userbase is starting to run into this more and more in our install of AEN, I've resorted to monkeypatching Kale's fix into the wakari-compute site-packages. It made me a little nausious for a minute but I couldn't wait any longer.

znmeb commented 7 years ago

I'm still seeing this as of 20170325 - what's the latest temporary fix / workaround? BTW, I'm stuck at Python 3.5 (or 2.7) because of https://github.com/ContinuumIO/anaconda-issues/issues/1423

Maybe I should just drop back to Conda 4.2.0??

damianavila commented 7 years ago

Just a head's up... since my userbase is starting to run into this more and more in our install of AEN, I've resorted to monkeypatching Kale's fix into the wakari-compute site-packages. It made me a little nausious for a minute but I couldn't wait any longer.

I understand... I will ping here when the fix is merged and packaged.

damianavila commented 7 years ago

Two option for now:

I'm still seeing this as of 20170325 - what's the latest temporary fix / workaround? BTW, I'm stuck at Python 3.5 (or 2.7) because of ContinuumIO/anaconda-issues#1423

The fix lives here: https://github.com/Anaconda-Platform/nb_conda/pull/42, but still not merged nor packaged

Maybe I should just drop back to Conda 4.2.0??

That would be a workaround for the time being... hopefully I will be able to build packages containing the fix by next week.

spk83 commented 7 years ago

Is nb_conda 2.2.0 published to any conda channel yet? I have Anaconda 4.3.1 running and I am facing the same issue. I saw that PR has been merged and released in version 2.2.0 but I am not able to find a way to update nb_conda. Newer version isn't being displayed with conda search or conda update command. Could you suggest a way to get this updated version in conda env? Thanks!

spk83 commented 7 years ago

Never mind, found it on conda-forge!

kalefranz commented 7 years ago

@csoja looks like nb_conda 2.2.0 is tagged. In case you didn't already have it on the package build list...

damianavila commented 7 years ago

@kalefranz it is tagged but we still have to figure out (discuss) some things before putting it in defaults. I am pretty sure @csoja already have it on her package build list :wink:

wszhan commented 7 years ago

I have tried all the links in this page and still it doesn't work...

This is the information in my console after all these attempts...

[E 10:26:44.531 NotebookApp] Unhandled error in API request
    Traceback (most recent call last):
      File "/Users/W.Zhan/anaconda/envs/DAND/lib/python2.7/site-packages/notebook/base/handlers.py", line 481, in wrapper
        result = yield gen.maybe_future(method(self, *args, **kwargs))
      File "/Users/W.Zhan/anaconda/envs/DAND/lib/python2.7/site-packages/nb_conda/handlers.py", line 62, in get
        self.finish(json.dumps(self.env_manager.env_packages(env)))
      File "/Users/W.Zhan/anaconda/envs/DAND/lib/python2.7/site-packages/nb_conda/envmanager.py", line 124, in env_packages
        "packages": [pkg_info(package) for package in data]
      File "/Users/W.Zhan/anaconda/envs/DAND/lib/python2.7/site-packages/nb_conda/envmanager.py", line 16, in pkg_info
        name, version, build = s.rsplit('-', 2)
    AttributeError: 'dict' object has no attribute 'rsplit'
[E 10:26:44.534 NotebookApp] {
      "Accept-Language": "zh-CN,zh;q=0.8,en;q=0.6,zh-TW;q=0.4", 
      "Accept-Encoding": "gzip, deflate, br", 
      "X-Requested-With": "XMLHttpRequest", 
      "Host": "localhost:8888", 
      "Accept": "application/json, text/javascript, */*; q=0.01", 
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36", 
      "Connection": "keep-alive", 
      "Referer": "http://localhost:8888/tree?token=b9a8574133a77c73460674ff3ba9b25002ae5f6f6b2f7962", 
      "Cookie": "_xsrf=2|90b6c559|58a8ed8888a507537448bb337b7e8097|1504236202; username-localhost-8888=\"2|1:0|10:1504434543|23:username-localhost-8888|44:ZTA5ZjY5NmU5NDE2NDg0YTg3MzA5MGRlOGRhNjgzOTk=|1cabf2309b2a321488c32105bbfd6d50f62aee984effdf331ab27ce0339022aa\""
    }
[E 10:26:44.534 NotebookApp] 500 GET /conda/environments/DAND?_=1504491998266 (::1) 1082.29ms referer=http://localhost:8888/tree?token=b9a8574133a77c73460674ff3ba9b25002ae5f6f6b2f7962

Now I am stuck with Jupyter here and can't move on...

Please help, thanks!

damianavila commented 7 years ago

@wesleyzhan, what is your conda version? And the nb_conda version?

jfogarty commented 7 years ago

I too was having this issue (conda 4.3.25, Linux Mint 17.2 Rafaela, Ubuntu 14.04.2 LTS, Trusty Tahr). I attempted the usual update remedies to no avail so I patched the code on line 16 of .../site-packages/nb_conda/envmanager.py.

    name, version, build = s.rsplit('-', 2)

becomes:

    if isinstance(s, dict):
        s = s[u'dist_name']
    name, version, build = s.rsplit('-', 2)

This seems minimally invasive, since examination of the dict that is passed always contains a well formed dist_name, and apparently the code expects exactly that. This was helpful to me, YMMV.

drevicko commented 7 years ago

@jfogarty you'd be better off applying #42, there's only a few lines changed there.