MrtnMndt / Deep_Openset_Recognition_through_Uncertainty

PyTorch code for our paper: Open Set Recognition Through Deep Neural Network Uncertainty: Does Out-of-Distribution Detection Require Generative Classifiers? https://arxiv.org/abs/1908.09625
MIT License
87 stars 19 forks source link

ImportError: dlopen: cannot load any more object with static TLS #2

Closed 9ayhub closed 4 years ago

9ayhub commented 4 years ago

It seems that the main problem is the use of librosa and pytorch, I can't use both libraries at the same time, even if just executing the imports of main.py and datasets.py in order.

bishe@hulk:~/kunning/DORtU$ python main.py
Traceback (most recent call last):
  File "/home/bishe/anaconda3/lib/python3.6/site-packages/sklearn/__check_build/__init__.py", line 44, in <module>
    from ._check_build import check_build  # noqa
ImportError: dlopen: cannot load any more object with static TLS

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 18, in <module>
    import lib.Datasets.datasets as datasets
  File "/home/bishe/kunning/DORtU/lib/Datasets/datasets.py", line 6, in <module>
    import librosa
  File "/home/bishe/anaconda3/lib/python3.6/site-packages/librosa/__init__.py", line 14, in <module>
    from . import decompose
  File "/home/bishe/anaconda3/lib/python3.6/site-packages/librosa/decompose.py", line 19, in <module>
    import sklearn.decomposition
  File "/home/bishe/anaconda3/lib/python3.6/site-packages/sklearn/__init__.py", line 81, in <module>
    from . import __check_build  # noqa: F401
  File "/home/bishe/anaconda3/lib/python3.6/site-packages/sklearn/__check_build/__init__.py", line 46, in <module>
    raise_build_error(e)
  File "/home/bishe/anaconda3/lib/python3.6/site-packages/sklearn/__check_build/__init__.py", line 41, in raise_build_error
    %s""" % (e, local_dir, ''.join(dir_content).strip(), msg))
ImportError: dlopen: cannot load any more object with static TLS
___________________________________________________________________________
Contents of /home/bishe/anaconda3/lib/python3.6/site-packages/sklearn/__check_build:
__init__.py               setup.py                  _check_build.cpython-36m-x86_64-linux-gnu.so
__pycache__
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.

If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.

If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
MrtnMndt commented 4 years ago

Hey, The way I see it, your issue is fairly unrelated to our repository as your error occurs at the import stage already. Given that we import only standard packages, I don’t see how this is related to our specific code. Therefore, I don’t think this is the right place to ask your question.

Googling your error, the first few links suggest that your issue is related to installation of PyTorch through conda in conjunction with importing some libraries like sklearn (as suggested also by your above error). The way I understood it, by taking a brief look at some of these links, is that for some reason import order seems to matter, as a library named “libglob” needs to be imported early on and conda doesn’t guarantee this by itself somehow.

I suggest you take a look at: https://github.com/pytorch/pytorch/issues/2575 And follow some of the suggestions there to get a fully functioning pytorch installation. The most common suggestion, as is also the recommendation for our installation instructions, is to install pytorch via pip instead of conda. Other than that, it seems you can play around with import ordering as a nasty work-around. Me and my colleagues are unable to reproduce your issue on multiple machines. If it persists for you, I suggest debugging your PyTorch installation with a post in the PyTorch forum or issue in their GitHub.