TextToVideo tool raising name 'init_empty_weights' is not defined error

[freddyaboulton]

System Info

2023-05-15 21:13:50.400043: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/transformers/commands/env.py:63: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version. Instructions for updating: Use tf.config.list_physical_devices('GPU') instead. No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• transformers version: 4.29.1
• Platform: Linux-5.15.107+-x86_64-with-glibc2.31
• Python version: 3.10.11
• Huggingface_hub version: 0.14.1
• Safetensors version: not installed
• PyTorch version (GPU?): 2.0.0+cu118 (False)
• Tensorflow version (GPU?): 2.12.0 (False)
• Flax version (CPU?/GPU?/TPU?): 0.6.9 (cpu)
• Jax version: 0.4.8
• JaxLib version: 0.4.7
• Using GPU in script?: no
• Using distributed or parallel set-up in script?: no

Who can help?

@LysandreJik

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

from transformers import Tool, OpenAiAgent, HfAgent

agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder")
output = agent.run("Please make a video of `prompt`", prompt="a man eating spaghetti")

/usr/local/lib/python3.10/dist-packages/transformers/tools/python_interpreter.py:101 in          │
│ evaluate_ast                                                                                     │
│                                                                                                  │
│    98 │   │   return evaluate_assign(expression, state, tools)                                   │
│    99 │   elif isinstance(expression, ast.Call):                                                 │
│   100 │   │   # Function call -> we return the value of the function call                        │
│ ❱ 101 │   │   return evaluate_call(expression, state, tools)                                     │
│   102 │   elif isinstance(expression, ast.Constant):                                             │
│   103 │   │   # Constant -> just return the value                                                │
│   104 │   │   return expression.value                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/tools/python_interpreter.py:167 in          │
│ evaluate_call                                                                                    │
│                                                                                                  │
│   164 │   # Todo deal with args                                                                  │
│   165 │   args = [evaluate_ast(arg, state, tools) for arg in call.args]                          │
│   166 │   kwargs = {keyword.arg: evaluate_ast(keyword.value, state, tools) for keyword in call   │
│ ❱ 167 │   return func(*args, **kwargs)                                                           │
│   168                                                                                            │
│   169                                                                                            │
│   170 def evaluate_subscript(subscript, state, tools):                                           │
│                                                                                                  │
│ /root/.cache/huggingface/modules/transformers_modules/huggingface-tools/text-to-video/15f8f33935 │
│ f9653aa806382d1536f8a48a0c6cc0/text_to_video.py:45 in __call__                                   │
│                                                                                                  │
│   42 │                                                                                           │
│   43 │   def __call__(self, prompt, seconds=2):                                                  │
│   44 │   │   if not self.is_initialized:                                                         │
│ ❱ 45 │   │   │   self.setup()                                                                    │
│   46 │   │                                                                                       │
│   47 │   │   return self.pipeline(prompt, num_frames=8 * seconds).frames                         │
│   48                                                                                             │
│                                                                                                  │
│ /root/.cache/huggingface/modules/transformers_modules/huggingface-tools/text-to-video/15f8f33935 │
│ f9653aa806382d1536f8a48a0c6cc0/text_to_video.py:36 in setup                                      │
│                                                                                                  │
│   33 │   │   if self.device is None:                                                             │
│   34 │   │   │   self.device = get_default_device()                                              │
│   35 │   │                                                                                       │
│ ❱ 36 │   │   self.pipeline = DiffusionPipeline.from_pretrained(                                  │
│   37 │   │   │   self.default_checkpoint, variant="fp16"                                         │
│   38 │   │   )                                                                                   │
│   39 │   │   self.pipeline.to(self.device)                                                       │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/diffusers/pipelines/pipeline_utils.py:1039 in            │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   1036 │   │   │   │   loaded_sub_model = passed_class_obj[name]                                 │
│   1037 │   │   │   else:                                                                         │
│   1038 │   │   │   │   # load sub model                                                          │
│ ❱ 1039 │   │   │   │   loaded_sub_model = load_sub_model(                                        │
│   1040 │   │   │   │   │   library_name=library_name,                                            │
│   1041 │   │   │   │   │   class_name=class_name,                                                │
│   1042 │   │   │   │   │   importable_classes=importable_classes,                                │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/diffusers/pipelines/pipeline_utils.py:445 in             │
│ load_sub_model                                                                                   │
│                                                                                                  │
│    442 │                                                                                         │
│    443 │   # check if the module is in a subdirectory                                            │
│    444 │   if os.path.isdir(os.path.join(cached_folder, name)):                                  │
│ ❱  445 │   │   loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwar  │
│    446 │   else:                                                                                 │
│    447 │   │   # else load from the root directory                                               │
│    448 │   │   loaded_sub_model = load_method(cached_folder, **loading_kwargs)                   │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:2608 in from_pretrained   │
│                                                                                                  │
│   2605 │   │   │   logger.info("Detected DeepSpeed ZeRO-3: activating zero.init() for this mode  │
│   2606 │   │   │   init_contexts = [deepspeed.zero.Init(config_dict_or_path=deepspeed_config())  │
│   2607 │   │   elif load_in_8bit or low_cpu_mem_usage:                                           │
│ ❱ 2608 │   │   │   init_contexts.append(init_empty_weights())                                    │
│   2609 │   │                                                                                     │
│   2610 │   │   with ContextManagers(init_contexts):                                              │
│   2611 │   │   │   model = cls(config, *model_args, **model_kwargs)                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
NameError: name 'init_empty_weights' is not defined

Same behavior calling the tool directly,

from transformers.tools import load_tool
tool = load_tool("huggingface-tools/text-to-video")
tool(prompt="a man eating spaghetti")

Expected behavior

The tool does not error

[LysandreJik]

Thanks for reporting @freddyaboulton! Do you have accelerate installed? If so, which version?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.