Can't create transformer pipeline because pytorch failed to be detected

[dannikay]

System Info

Ubuntu 22.04 Python 3.12.3

Who can help?

@Narsil @zucchini-nlp

Information

• [ ] The official example scripts
• [X] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [X] My own task or dataset (give details below)

Reproduction

import transformers
from transformers import is_torch_available
import torch
print(torch.__version__)
print(is_torch_available())

# Define the task that we want to use (required for proper pipeline construction)
task = "text2text-generation"

# Define the pipeline, using the task and a model instance that is applicable for our task.
generation_pipeline = transformers.pipeline(
    task=task,
    model="declare-lab/flan-alpaca-large",
)

# Define a simple input example that will be recorded with the model in MLflow, giving
# users of the model an indication of the expected input format.
input_example = ["prompt 1", "prompt 2", "prompt 3"]

# Define the parameters (and their defaults) for optional overrides at inference time.
parameters = {"max_length": 512, "do_sample": True, "temperature": 0.4}

2.3.1+cu121
False

...

RuntimeError: At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.

Expected behavior

I do not expect the above error since pytorch has been installed in my system.

[amyeroberts]

cc @ydshieh

[dannikay]

I can no longer reproduce this after restarting my notebook kernel. I suppose that the pytorch detection relies on some system environment which requires notebook kernel to restart to pickup. Feel free to close this.

[dannikay]

It seems that _torch_available is a global variable and set at initialization time: https://github.com/huggingface/transformers/blob/02300273e220932a449a47ebbe453e7789be454b/src/transformers/utils/import_utils.py#L180C1-L180C17

So it won't be re-evaluated in the same notebook kernel when it was re-evaluated (until I restart my kernel). One improvement I can think of is to reevaluate this variable in is_torch_available() but I don't know if that breaks other things or not.

[amyeroberts]

So it won't be re-evaluated in the same notebook kernel when it was re-evaluated (until I restart my kernel).

Does this mean that torch is installed in the notebook after importing transformers?

[ydshieh]

In general, for notebook (well, I usually use Google Colab), if there is/are some installation(s), it is recommended to restart the kernel.

reevaluate this variable in is_torch_available()

We make use of _torch_available is (probably) and similar ideas for many other is_xxx_available is we try to avoid re-evaluation which might slowdown a lot.

I am open to a PR that improve stuff (the mentioned issue) but keep things fast (regarding what I mentioned above)

[dannikay]

@amyeroberts torch is installed in the notebook kernel ("!pip install ...") and I re-executed the block to import transformer multiple times (with no avail).

[amyeroberts]

orch is installed in the notebook kernel ("!pip install ...") and I re-executed the block to import transformer multiple times (with no avail).

@dannikay Just to be clear, this means that transformers was already imported, torch installed, then the cell to import transformers re-executed?

As @ydshieh notes above, we have a cache which means is_torch_available is executed once and its result stored within a python session. This helps speed things up within the transformers library - we have lots of is_xxx_available flags which enable us to safely guard for different framework and modality usage e.g. PyTorch vs TensorFlow.

If you or anyone else wants to submit a PR which would make this more dynamic whilst maintaining speed, we'd be very happy to review!

[dannikay]

@amyeroberts correct.