search

atomashevic / transforEmotion

Sentiment Analysis for Text, Image and Video Using Transformer Models
GNU General Public License v3.0
24 stars 2 forks source link

Error on first running of transformer_scores(text = ... #21

Open ibernetti63 opened 2 months ago

ibernetti63 commented 2 months ago

in the first running of transformer_scores(text = ... I recive the following error: Requirement already satisfied: oauthlib>=3.0.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (3.2.2) Importing transformers and torch modules... Errore in py_module_import(module, convert = convert) : NameError: name '_C' is not defined Run reticulate::py_last_error() for details.

reticulate::py_last_error()

── Python Exception Message ───────────────────────────────────────────────── Traceback (most recent call last): File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook return _run_hook(name, _hook) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 96, in _run_hook module = hook() File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 120, in _hook return _find_andload(name, import) File "C:\Users\ibern\AppData\Local\R-MINI~1\envs\TRANSF~1\lib\site-packages\transformers__init.py", line 26, in from . import dependency_versions_check File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook return _run_hook(name, _hook) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 96, in _run_hook module = hook() File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 120, in _hook return _find_andload(name, import) File "C:\Users\ibern\AppData\Local\R-MINI~1\envs\TRANSF~1\lib\site-packages\transformers\dependency_versions_check.py", line 16, in from .utils.versions import require_version, require_version_core File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook return _run_hook(name, _hook) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 96, in _run_hook module = hook() File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 120, in _hook return _find_andload(name, import) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook return _run_hook(name, _hook) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 96, in _run_hook module = hook() File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 120, in _hook return _find_andload(name, import) File "C:\Users\ibern\AppData\Local\R-MINI~1\envs\TRANSF~1\lib\site-packages\transformers\utils__init__.py", line 31, in from .generic import ( File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook return _run_hook(name, _hook) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 96, in _run_hook module = hook() File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 120, in _hook return _find_andload(name, import) File "C:\Users\ibern\AppData\Local\R-MINI~1\envs\TRANSF~1\lib\site-packages\transformers\utils\generic.py", line 432, in import torch.utils._pytree as _torch_pytree File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 122, in _find_and_load_hook return _run_hook(name, _hook) File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 96, in _run_hook module = hook() File "C:\Users\ibern\AppData\Local\R\win-library\4.2\reticulate\python\rpytools\loader.py", line 120, in _hook return _find_andload(name, import) File "C:\Users\ibern\AppData\Local\R-MINI~1\envs\TRANSF~1\lib\site-packages\torch\init__.py", line 465, in for name in dir(_C): NameError: name '_C' is not defined

── R Traceback ────────────────────────────────────────────────────────────── ▆

  1. ├─transforEmotion::transformer_scores(...)
  2. │ └─reticulate::import("transformers")
  3. │ └─reticulate:::py_module_import(module, convert = convert)
  4. ├─base::tryCatch(...)
  5. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
  6. │ ├─base (local) tryCatchOne(...)
  7. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
  8. │ └─base (local) tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
  9. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
    1. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
    2. └─base::evalq(<fn>(FALSE, 1L), <env>)
    3. └─base::evalq(<fn>(FALSE, 1L), <env>) See reticulate::py_last_error()$r_trace$full_call for more details.
ibernetti63 commented 2 months ago

I try on a completly new PC (win 11 Dell) with a fresh install of R and I have a differnt error in the firs run of transformer_scores(text = ... : Importing transformers and torch modules... Errore in py_module_import(module, convert = convert) : TypeError: encode() argument 'encoding' must be str, not int Run reticulate::py_last_error() for details.

reticulate::py_last_error() Errore in py_call_impl(callable, call_args$unnamed, call_args$named) : Evaluation error: TypeError: encode() argument 'encoding' must be str, not int Run reticulate::py_last_error() for details..

atomashevic commented 2 months ago

Hi,

Thanks for opening the issue. The first error, looks like this issue https://github.com/atomashevic/transforEmotion/issues/4. Take a look and see whether any of the solutions proposed there works for your case.

The second error, looks like something else, I'll take a look at both and I'll let you know if there is a fix.

atomashevic commented 2 months ago

@ibernetti63 for the first issue can you give me the output of the following code

library(transforEmotion)
library(reticulate)

transforEmotion:: setup_miniconda()

use_condaenv("transforEmotion", required = TRUE)

py_run_string("import torch; print(torch.__version__)")
py_run_string("import transformers; print(transformers.__version__)")

Also, did you try the example:

# Load data
data(neo_ipip_extraversion)

text <- neo_ipip_extraversion$friendliness[1:5]

# Cross-Encoder DistilRoBERTa
transformer_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 )
)

I wasn't able to reproduce both issues on a fresh install (Linux), @AlexChristensen can you try on a Windows machine?

ibernetti63 commented 2 months ago

transforEmotionError.txt

Thank you for your quick reply. In the attached text file I have pasted the output of the commands. I used a PC with Windows 11 and R 4.4.1. Before running the commands I uninstalled miniconda to start from a clean situation. Regards Iacopo

AlexChristensen commented 2 months ago

I run Linux on both of my computers, so I won't be able to test Windows.

However, it seems like {pytorch} is acting up. You can try:

reticulate::conda_install(
    envname = "transforEmotion",
    packages = "torch==2.1.1",
    pip = TRUE
)

To make sure {pytorch} installs or is installed.

It would be helpful to have the output of reticulate::py_last_error() whenever there is an error that occurs

AlexChristensen commented 2 months ago

accidentally closed..... reopening

ibernetti63 commented 2 months ago

I run Linux on both of my computers, so I won't be able to test Windows.

However, it seems like {pytorch} is acting up. You can try:

reticulate::conda_install(
    envname = "transforEmotion",
    packages = "torch==2.1.1",
    pip = TRUE
)

To make sure {pytorch} installs or is installed.

It would be helpful to have the output of reticulate::py_last_error() whenever there is an error that occurs

library(reticulate) use_condaenv("transforEmotion", required = TRUE) reticulate::conda_install(

  • envname = "transforEmotion",
  • packages = "torch==2.1.1",
  • pip = TRUE
  • )

C:\Users\ibern\Documents>CALL "C:\Users\ibern\AppData\Local\r-miniconda\condabin\activate.bat" "transforEmotion"

C:\Users\ibern\Documents>conda.bat activate "transforEmotion" Requirement already satisfied: torch==2.1.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (2.1.1) Requirement already satisfied: filelock in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (3.15.4) Requirement already satisfied: typing-extensions in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (4.12.2) Requirement already satisfied: sympy in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (1.13.2) Requirement already satisfied: networkx in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (3.3) Requirement already satisfied: jinja2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (3.1.4) Requirement already satisfied: fsspec in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (2024.6.1) Requirement already satisfied: MarkupSafe>=2.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from jinja2->torch==2.1.1) (2.1.5) Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from sympy->torch==2.1.1) (1.3.0) [1] "\"torch==2.1.1\""

library(transforEmotion)

transforEmotion (version 0.1.5)

Important: If you are using RStudio, please make sure you have the latest version installed (2023.09.1-494). For help getting started, type browseVignettes("transforEmotion")

For bugs and errors, submit an issue to https://github.com/atomashevic/transforEmotion/issues Before running an analysis for the first time after installing the package, please run transforEmotion::setup_miniconda() to install the necessary Python modules.

Load data

data(neo_ipip_extraversion)

text <- neo_ipip_extraversion$friendliness[1:5]

Cross-Encoder DistilRoBERTa

transformer_scores(

  • text = text,
  • classes = c(
  • "friendly", "gregarious", "assertive",
  • "active", "excitement", "cheerful"
  • )
  • )

Installing modules for 'transforEmotion'...

C:\Users\ibern\Documents>CALL "C:\Users\ibern\AppData\Local\r-miniconda\condabin\activate.bat" "transforEmotion"

C:\Users\ibern\Documents>conda.bat activate "transforEmotion" Requirement already satisfied: accelerate==0.29.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (0.29.3) Requirement already satisfied: llama-index==0.10.30 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (0.10.30) Requirement already satisfied: nltk==3.8.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (3.8.1) Requirement already satisfied: pandas==2.1.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (2.1.3) Requirement already satisfied: pypdf==4.0.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (4.0.1) Requirement already satisfied: pytz==2024.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (2024.1) Requirement already satisfied: qdrant-client==1.8.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (1.8.2) Requirement already satisfied: sentencepiece==0.2.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (0.2.0) Requirement already satisfied: sentence-transformers==2.7.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (2.7.0) Requirement already satisfied: tensorflow==2.14.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (2.14.1) Requirement already satisfied: torch==2.1.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (2.1.1) Requirement already satisfied: transformers==4.35.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (4.35.0) Requirement already satisfied: pytubefix==6.9.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (6.9.2) Requirement already satisfied: numpy>=1.17 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from accelerate==0.29.3) (1.26.4) Requirement already satisfied: packaging>=20.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from accelerate==0.29.3) (24.1) Requirement already satisfied: psutil in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from accelerate==0.29.3) (6.0.0) Requirement already satisfied: pyyaml in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from accelerate==0.29.3) (6.0.2) Requirement already satisfied: huggingface-hub in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from accelerate==0.29.3) (0.17.3) Requirement already satisfied: safetensors>=0.3.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from accelerate==0.29.3) (0.4.4) Requirement already satisfied: llama-index-agent-openai<0.3.0,>=0.1.4 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.2.9) Requirement already satisfied: llama-index-cli<0.2.0,>=0.1.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.13) Requirement already satisfied: llama-index-core<0.11.0,>=0.10.30 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.10.68.post1) Requirement already satisfied: llama-index-embeddings-openai<0.2.0,>=0.1.5 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.11) Requirement already satisfied: llama-index-indices-managed-llama-cloud<0.2.0,>=0.1.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.6) Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.9.48.post3) Requirement already satisfied: llama-index-llms-openai<0.2.0,>=0.1.13 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.31) Requirement already satisfied: llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.9) Requirement already satisfied: llama-index-program-openai<0.2.0,>=0.1.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.7) Requirement already satisfied: llama-index-question-gen-openai<0.2.0,>=0.1.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.3) Requirement already satisfied: llama-index-readers-file<0.2.0,>=0.1.4 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.33) Requirement already satisfied: llama-index-readers-llama-parse<0.2.0,>=0.1.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index==0.10.30) (0.1.6) Requirement already satisfied: click in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from nltk==3.8.1) (8.1.7) Requirement already satisfied: joblib in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from nltk==3.8.1) (1.4.2) Requirement already satisfied: regex>=2021.8.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from nltk==3.8.1) (2024.7.24) Requirement already satisfied: tqdm in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from nltk==3.8.1) (4.66.5) Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from pandas==2.1.3) (2.9.0.post0) Requirement already satisfied: tzdata>=2022.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from pandas==2.1.3) (2024.1) Requirement already satisfied: grpcio>=1.41.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from qdrant-client==1.8.2) (1.66.1) Requirement already satisfied: grpcio-tools>=1.41.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from qdrant-client==1.8.2) (1.62.3) Requirement already satisfied: httpx>=0.20.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx[http2]>=0.20.0->qdrant-client==1.8.2) (0.27.2) Requirement already satisfied: portalocker<3.0.0,>=2.7.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from qdrant-client==1.8.2) (2.10.1) Requirement already satisfied: pydantic>=1.10.8 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from qdrant-client==1.8.2) (2.8.2) Requirement already satisfied: urllib3<3,>=1.26.14 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from qdrant-client==1.8.2) (2.2.2) Requirement already satisfied: scikit-learn in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from sentence-transformers==2.7.0) (1.5.1) Requirement already satisfied: scipy in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from sentence-transformers==2.7.0) (1.14.1) Requirement already satisfied: Pillow in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from sentence-transformers==2.7.0) (10.4.0) Requirement already satisfied: tensorflow-intel==2.14.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow==2.14.1) (2.14.1) Requirement already satisfied: filelock in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (3.15.4) Requirement already satisfied: typing-extensions in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (4.12.2) Requirement already satisfied: sympy in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (1.13.2) Requirement already satisfied: networkx in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (3.3) Requirement already satisfied: jinja2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (3.1.4) Requirement already satisfied: fsspec in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from torch==2.1.1) (2024.6.1) Requirement already satisfied: requests in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from transformers==4.35.0) (2.32.3) Requirement already satisfied: tokenizers<0.15,>=0.14 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from transformers==4.35.0) (0.14.1) Requirement already satisfied: absl-py>=1.0.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.1.0) Requirement already satisfied: astunparse>=1.6.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (1.6.3) Requirement already satisfied: flatbuffers>=23.5.26 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (24.3.25) Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.6.0) Requirement already satisfied: google-pasta>=0.1.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.2.0) Requirement already satisfied: h5py>=2.9.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (3.11.0) Requirement already satisfied: libclang>=13.0.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (18.1.1) Requirement already satisfied: ml-dtypes==0.2.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.2.0) Requirement already satisfied: opt-einsum>=2.3.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (3.3.0) Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (4.25.4) Requirement already satisfied: setuptools in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (72.2.0) Requirement already satisfied: six>=1.12.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (1.16.0) Requirement already satisfied: termcolor>=1.1.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.4.0) Requirement already satisfied: wrapt<1.15,>=1.11.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (1.14.1) Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.31.0) Requirement already satisfied: tensorboard<2.15,>=2.14 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.14.1) Requirement already satisfied: tensorflow-estimator<2.15,>=2.14.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.14.0) Requirement already satisfied: keras<2.15,>=2.14.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.14.0) Requirement already satisfied: anyio in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (4.4.0) Requirement already satisfied: certifi in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (2024.8.30) Requirement already satisfied: httpcore==1. in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (1.0.5) Requirement already satisfied: idna in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (3.8) Requirement already satisfied: sniffio in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (1.3.1) Requirement already satisfied: h11<0.15,>=0.13 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpcore==1.->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (0.14.0) Requirement already satisfied: h2<5,>=3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from httpx[http2]>=0.20.0->qdrant-client==1.8.2) (4.1.0) Requirement already satisfied: openai>=1.14.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-agent-openai<0.3.0,>=0.1.4->llama-index==0.10.30) (1.43.0) Requirement already satisfied: SQLAlchemy>=1.4.49 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (2.0.32) Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (3.10.5) Requirement already satisfied: dataclasses-json in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (0.6.7) Requirement already satisfied: deprecated>=1.2.9.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.2.14) Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.0.8) Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.6.0) Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.2.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (8.5.0) Requirement already satisfied: tiktoken>=0.3.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (0.7.0) Requirement already satisfied: typing-inspect>=0.8.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (0.9.0) Requirement already satisfied: llamaindex-py-client<0.2.0,>=0.1.19 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-indices-managed-llama-cloud<0.2.0,>=0.1.2->llama-index==0.10.30) (0.1.19) Requirement already satisfied: beautifulsoup4<5.0.0,>=4.12.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.30) (4.12.3) Requirement already satisfied: striprtf<0.0.27,>=0.0.26 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.30) (0.0.26) Requirement already satisfied: llama-parse>=0.4.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from llama-index-readers-llama-parse<0.2.0,>=0.1.2->llama-index==0.10.30) (0.4.9) Requirement already satisfied: pywin32>=226 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from portalocker<3.0.0,>=2.7.0->qdrant-client==1.8.2) (306) Requirement already satisfied: annotated-types>=0.4.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from pydantic>=1.10.8->qdrant-client==1.8.2) (0.7.0) Requirement already satisfied: pydantic-core==2.20.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from pydantic>=1.10.8->qdrant-client==1.8.2) (2.20.1) Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from requests->transformers==4.35.0) (3.3.2) Requirement already satisfied: colorama in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tqdm->nltk==3.8.1) (0.4.6) Requirement already satisfied: MarkupSafe>=2.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from jinja2->torch==2.1.1) (2.1.5) Requirement already satisfied: threadpoolctl>=3.1.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from scikit-learn->sentence-transformers==2.7.0) (3.5.0) Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from sympy->torch==2.1.1) (1.3.0) Requirement already satisfied: aiohappyeyeballs>=2.3.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (2.4.0) Requirement already satisfied: aiosignal>=1.1.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (24.2.0) Requirement already satisfied: frozenlist>=1.1.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.4.1) Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (6.0.5) Requirement already satisfied: yarl<2.0,>=1.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.9.4) Requirement already satisfied: async-timeout<5.0,>=4.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (4.0.3) Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from astunparse>=1.6.0->tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.44.0) Requirement already satisfied: soupsieve>1.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from beautifulsoup4<5.0.0,>=4.12.3->llama-index-readers-file<0.2.0,>=0.1.4->llama-index==0.10.30) (2.6) Requirement already satisfied: hyperframe<7,>=6.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (6.0.1) Requirement already satisfied: hpack<5,>=4.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (4.0.0) Requirement already satisfied: distro<2,>=1.7.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from openai>=1.14.0->llama-index-agent-openai<0.3.0,>=0.1.4->llama-index==0.10.30) (1.9.0) Requirement already satisfied: jiter<1,>=0.4.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from openai>=1.14.0->llama-index-agent-openai<0.3.0,>=0.1.4->llama-index==0.10.30) (0.5.0) Requirement already satisfied: exceptiongroup>=1.0.2 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from anyio->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client==1.8.2) (1.2.2) Requirement already satisfied: greenlet!=0.4.17 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from SQLAlchemy>=1.4.49->SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (3.0.3) Requirement already satisfied: google-auth<3,>=1.6.3 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.34.0) Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (1.0.0) Requirement already satisfied: markdown>=2.6.8 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (3.7) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (3.0.4) Requirement already satisfied: mypy-extensions>=0.3.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from typing-inspect>=0.8.0->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (1.0.0) Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from dataclasses-json->llama-index-core<0.11.0,>=0.10.30->llama-index==0.10.30) (3.22.0) Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (5.5.0) Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.4.0) Requirement already satisfied: rsa<5,>=3.1.4 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (4.9) Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (2.0.0) Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (0.6.0) Requirement already satisfied: oauthlib>=3.0.0 in c:\users\ibern\appdata\local\r-miniconda\envs\transforemotion\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.15,>=2.14->tensorflow-intel==2.14.1->tensorflow==2.14.1) (3.2.2) Importing transformers and torch modules... Errore in py_module_import(module, convert = convert) : TypeError: encode() argument 'encoding' must be str, not int Run reticulate::py_last_error() for details. In aggiunta: Messaggi di avvertimento: 1: In py_run_string_impl(patch, local = TRUE) : Error encountered when flushing python buffers sys.stderr and sys.stdout 2: In py_run_string_impl(patch, local = TRUE) : Error encountered when flushing python buffers sys.stderr and sys.stdout 3: In py_run_string_impl("import sys; sys.path.insert(0, '')", local = TRUE) : Error encountered when flushing python buffers sys.stderr and sys.stdout

atomashevic commented 2 months ago

Hi @ibernetti63,

Can you test the following steps:

reticulate::miniconda_uninstall()

devtools::install_github("atomashevic/transforEmotion@cpu-only-libraries", force = TRUE)

library(transforEmotion)

setup_miniconda()

and then

# Load data
data(neo_ipip_extraversion)

text <- neo_ipip_extraversion$friendliness[1:5]

# Cross-Encoder DistilRoBERTa
transformer_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 )
)

Please paste any error or warning messages you get. Thanks for your patience with this issue.

ibernetti63 commented 3 weeks ago

Cross-Encoder DistilRoBERTa

transformer_scores(

  • text = text,
  • classes = c(
  • "friendly", "gregarious", "assertive",
  • "active", "excitement", "cheerful"
  • )
  • )

All modules are already installed. Importing transformers and torch modules... Error flushing Python's stdout/stderr buffers. Auto-flushing is now disabled. Errore in py_module_import(module, convert = convert) : TypeError: encode() argument 'encoding' must be str, not int Run reticulate::py_last_error() for details. In aggiunta: Messaggi di avvertimento: 1: In py_run_string_impl(patch, local = TRUE) : Error encountered when flushing python buffers sys.stderr and sys.stdout 2: In py_run_string_impl(patch, local = TRUE) : Error encountered when flushing python buffers sys.stderr and sys.stdout 3: In py_run_string_impl("import sys; sys.path.insert(0, '')", local = TRUE) : Error encountered when flushing python buffers sys.stderr and sys.stdout

snvv commented 6 days ago

Hello,

I have a similar problem.

I followed the instructions to set up Python with the transforEmotion library:

library(transforEmotion)
# Setup Python
setup_miniconda()

The installations completed without any issues, and the following packages were installed:

[1] "accelerate==0.29.3"           "llama-index==0.10.30"        
[3] "nltk==3.8.1"                  "opencv-python"                 
[5] "pandas==2.1.3"                "pypdf==4.0.1"                
[7] "pytz==2024.1"                 "qdrant-client==1.8.2"        
[9] "sentencepiece==0.2.0"         "sentence-transformers==2.7.0"
[11] "tensorflow==2.14.1"           "torch==2.1.1"                
[13] "transformers==4.35.0"         "pytubefix==6.9.2"

I then ran the manual example from the transforEmotion documentation:

library(transforEmotion)
data(neo_ipip_extraversion)
text <- neo_ipip_extraversion$friendliness[1:5] # positively worded items only
# Run transformer function
transformer_scores(
  text = text,
  classes = c(
    "friendly", "gregarious", "assertive",
    "active", "excitement", "cheerful"
  )
)

After downloading various files, the following warning appeared:

Warning: could not find TensorRT
Obtaining Scores…

Shortly after, I received this error:

R session aborted.
R encountered a fatal error. The session was terminated.

I suspect the issue may be related to CUDA. Although I haven't installed CUDA explicitly, it seems that transformers is using the CUDA-enabled version of torch. Here's what I found:

# Check torch version
py_run_string("import torch; print(torch.__version__)")
# Output:
2.1.1+cu121

# Check transformers version
py_run_string("import transformers; print(transformers.__version__)")
# Output:
4.35.0

Questions:

  1. Could this issue be caused by the CUDA-enabled version of torch being used on a system without CUDA installed?
  2. Should I install a non-CUDA version of torch, and if so, how can I ensure that it integrates correctly with the transforEmotion library?
  3. Are there any specific steps I should take to resolve this error?

Any guidance would be greatly appreciated!

Update: Issue Persists with BART Model

Attempted BART Model:

I tried using the Facebook BART Large model for the transformer_scores function:

# Facebook BART Large
transformer_scores(
  text = text,
  classes = c(
    "friendly", "gregarious", "assertive",
    "active", "excitement", "cheerful"
  ),
  transformer = "facebook-bart"
)

Process:

After downloading a 1.6 GB model file, the following steps were observed:

Installing collected packages: tensorflow-cpu
Successfully installed tensorflow-cpu-2.14.1
Importing transformers and torch modules...
2024-11-19 19:36:57.807739: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Downloading config.json: 100%|██████████| 1.15k/1.15k [00:00<00:00, 4.52MB/s]
Downloading model.safetensors:  42%|████▏     | 692M/1.63G [02:41<05:04, 3.08MB/s]

Result:

The process crashed during the "Obtaining Scores" step. No additional error messages or details were provided before the crash.

Observations:

Request for Assistance:

  1. Is the crash related to system limitations (e.g., memory or CPU)?
  2. Are there specific requirements or configurations for using the facebook-bart transformer in this context?
  3. Could this issue be related to a dependency mismatch or incomplete installation of supporting packages?

Any advice on debugging or resolving this issue would be highly appreciated!

atomashevic commented 2 days ago

Hi @snvv,

Thanks for the detailed report. Regarding the first case, the error is probably related to the CUDA version of torch. We are working on this, and it is solved in the GitHub version of the package (#22), where we have CPU versions of tensorflow and torch installed by default.

For the second case, the CPU version of tensorflow is running, which is OK, but since your R session is crashing, something else is the problem. How much free RAM do you have available before running this code? Which CPU do you have on your system?

I will try to reproduce both errors on different machines.

@ibernetti63 I haven't forgotten about your issue as well. I will investigate further and try to see if there is a fix before the release of a new version of the package.