ImportError: cannot import name 'load_dataset' from 'datasets'

[eadsa1998]

Environment info

• transformers version: 4.6.0
• Platform: Windows-10-10.0.19041-SP0
• Python version: 3.8.3
• PyTorch version (GPU?): 1.7.1 (True)
• Using GPU in script?: Possibly?
• Using distributed or parallel set-up in script?: No

Who can help

@LysandreJik and @lhoestq helped on the other issues that I looked at so they might be able to help here. But I'll take anyone really.

Information

I am attempting to run finBERT and am having trouble with the datasets package. I looked at a couple of other issues from people who had similar problems but none of their solutions worked for me. I'm sorry if I didn't provide some information or missed something obvious, I'm new to programming and very new to machine learning so I don't quite know what/where everything is yet!

The problem arises when using:

• [ ] my own modified scripts: (give details below) I am using the first model in this example script from the finBERT model developers.

The tasks I am working on is:

• [ ] my own task or dataset: (give details below) I am just trying to use transformers to run finBERT without using the API.

To reproduce

Steps to reproduce the behavior:

1. Import datasets That's as far as I'm able to get before I get this error:

""""" ImportError Traceback (most recent call last)

in 24 from sklearn.metrics import classification_report 25 import transformers ---> 26 from transformers import AutoModel, BertTokenizerFast 27 28 ~\anaconda3\lib\site-packages\transformers\__init__.py in __getattr__(self, name) 2485 if name == "__version__": 2486 return __version__ -> 2487 return super().__getattr__(name) 2488 2489 sys.modules[__name__] = _LazyModule(__name__, _import_structure) ~\anaconda3\lib\site-packages\transformers\file_utils.py in __getattr__(self, name) 1698 elif name in self._class_to_module.keys(): 1699 module = self._get_module(self._class_to_module[name]) -> 1700 value = getattr(module, name) 1701 else: 1702 raise AttributeError(f"module {self.__name__} has no attribute {name}") ~\anaconda3\lib\site-packages\transformers\file_utils.py in __getattr__(self, name) 1697 value = self._get_module(name) 1698 elif name in self._class_to_module.keys(): -> 1699 module = self._get_module(self._class_to_module[name]) 1700 value = getattr(module, name) 1701 else: ~\anaconda3\lib\site-packages\transformers\models\auto\__init__.py in _get_module(self, module_name) 196 197 def _get_module(self, module_name: str): --> 198 return importlib.import_module("." + module_name, self.__name__) 199 200 sys.modules[__name__] = _LazyModule(__name__, _import_structure) ~\anaconda3\lib\importlib\__init__.py in import_module(name, package) 125 break 126 level += 1 --> 127 return _bootstrap._gcd_import(name[level:], package, level) 128 129 ~\anaconda3\lib\site-packages\transformers\models\auto\modeling_auto.py in 197 from ..pegasus.modeling_pegasus import PegasusForCausalLM, PegasusForConditionalGeneration, PegasusModel 198 from ..prophetnet.modeling_prophetnet import ProphetNetForCausalLM, ProphetNetForConditionalGeneration, ProphetNetModel --> 199 from ..rag.modeling_rag import ( # noqa: F401 - need to import all RagModels to be in globals() function 200 RagModel, 201 RagSequenceForGeneration, ~\anaconda3\lib\site-packages\transformers\models\rag\modeling_rag.py in 27 from ...utils import logging 28 from .configuration_rag import RagConfig ---> 29 from .retrieval_rag import RagRetriever 30 31 ~\anaconda3\lib\site-packages\transformers\models\rag\retrieval_rag.py in 37 38 if is_datasets_available(): ---> 39 from datasets import Dataset, load_dataset, load_from_disk 40 41 if is_faiss_available(): ImportError: cannot import name 'load_dataset' from 'datasets' (C:\Users\bookw\Dropbox\Equity-Analyst-Project\equity-analysts-sentiment\datasets.py) """'" ## Expected behavior I would expect the package to import correctly.

[lhoestq]

Hi ! When you import datasets, python looks at your installed packages, but also at the modules defined in the directory from which you run your code. It is the case because the current working directory is added to your python path when you run your code.

In your case I think it tries to load your datasets.py in the equity-analysts-sentiment folder, since the name is conflicting. If you rename this file you should be good.

[eadsa1998]

Ok so I renamed the file and it still wouldn't run. I also tried moving it around to run it in other directories and see if I had better luck but I still got this same error everywhere I tried it.

[lhoestq]

If you're still having this error:

ImportError: cannot import name 'load_dataset' from 'datasets' (C:\Users\bookw\Dropbox\Equity-Analyst-Project\equity-analysts-sentiment\datasets.py)

Then it probably means that C:\Users\bookw\Dropbox\Equity-Analyst-Project\equity-analysts-sentiment is still in your python path. Can you check that you didn't add this path to your python path via environment variables or via your IDE ? I know that some of them like PyCharm add project directories to the python path automatically for example.

[eadsa1998]

I don't think I'm using a virtual enviroment or IDE, just Jupyter Notebooks. I'll paste my python path below but I don't see that in there.

C:\Users\bookw\anaconda3;C:\Users\bookw\anaconda3\Library\mingw-w64\bin;C:\Users\bookw\anaconda3\Library\usr\bin;C:\Users\book w\anaconda3\Library\bin;C:\Users\bookw\anaconda3\Scripts;C:\Users\bookw\anaconda3\bin;C:\Users\bookw\anaconda3\condabin;C:\Use rs\bookw\anaconda3;C:\Users\bookw\anaconda3\Library\mingw-w64\bin;C:\Users\bookw\anaconda3\Library\usr\bin;C:\Users\bookw\anac onda3\Library\bin;C:\Users\bookw\anaconda3\Scripts;C:\Program Files\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\W indows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files (x86)\ NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Git\cmd;C:\Program Files\P uTTY;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C: \WINDOWS\System32\OpenSSH;C:\Users\bookw\AppData\Local\Microsoft\WindowsApps;C:\Users\bookw\AppData\Local\Programs\MiKTeX\mikt ex\bin\x64;C:\Users\bookw.dotnet\tools;.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[mwunderlich]

I am facing the same issue when trying to follow the datasets tutorial from the Huggingface course. The line from datasets import load_dataset causes the following error: ImportError: cannot import name 'load_dataset' from 'datasets' (unknown location).

My environment:

• macOS Big Sur 11.6. on M1 Macbook
• python 3.8.0
• conda 4.11.0
• transformers 4.16.2
• datasets 1.18.3 (installed with conda install -c huggingface -c conda-forge datasets)
• torch 1.10.2

The Colab notebook provided by the course works fine. This error occurs only locally. Could this be an M1 related issue on the Macbook? I have had problems in the past with conda installations and also with tensorflow on the M1.

@eadsa1998 Did you manage to resolve the problem?

[hazemMondy]

the same issue also

[telmomenezes]

I had the same issue and solved it by reinstalling the datasets package.

[o3523o]

the same issue also now

[HarperHao]

I'm having the same issue, and it still don't work after reinstalling the datasets package.

[lhoestq]

Can you check that you don't have a directory named "datasets" or a file "datasets.py" in your working directory or in directories in your python path (including the ones that your IDE may be adding) ?

The ImportError can also show the location of the diretory/file that is imported instead of the datasets package

[bkowshik]

Thank you @lhoestq I had the datasets folder. 😅

[ppeterpp]

try pip install datasets

[whoisusmanali]

When I tried to run the from dataset import load_dataset but it is showing ImportError: cannot import name 'load_dataset' from 'dataset' (/usr/local/lib/python3.10/dist-packages/dataset/init.py) This error and tried everything but still not working. @eadsa1998 did you resolve this issue.

[ditarnnt]

hi @whoisusmanali did you resolve the issue? i also have the same issue like yours (path)

[Ice-Citron]

run

pip install datasets==2.21.0

@ditarnnt @whoisusmanali