Error with simple generate command

[DavorJordacevic]

Does anyone know what is the cause?

I want to run the service in low memory mode.

Command:

python generate.py --base_model=h2oai/h2ogpt-gm-oasst1-en-2048-falcon-7b-v3 --hf_embedding_model=sentence-transformers/all-MiniLM-L6-v2 --score_model=None --load_4bit=True --langchain_mode='MyData' --gpu_id=1

Output:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/djordacevic/h2ogpt/generate.py:16 in <module>                                              │
│                                                                                                  │
│   13                                                                                             │
│   14                                                                                             │
│   15 if __name__ == "__main__":                                                                  │
│ ❱ 16 │   entrypoint_main()                                                                       │
│   17                                                                                             │
│                                                                                                  │
│ /home/djordacevic/h2ogpt/generate.py:12 in entrypoint_main                                       │
│                                                                                                  │
│    9                                                                                             │
│   10                                                                                             │
│   11 def entrypoint_main():                                                                      │
│ ❱ 12 │   fire.Fire(main)                                                                         │
│   13                                                                                             │
│   14                                                                                             │
│   15 if __name__ == "__main__":                                                                  │
│                                                                                                  │
│ /home/djordacevic/.virtualenvs/h2o/lib/python3.8/site-packages/fire/core.py:141 in Fire          │
│                                                                                                  │
│   138 │   context.update(caller_globals)                                                         │
│   139 │   context.update(caller_locals)                                                          │
│   140                                                                                            │
│ ❱ 141   component_trace = _Fire(component, args, parsed_flag_args, context, name)                │
│   142                                                                                            │
│   143   if component_trace.HasError():                                                           │
│   144 │   _DisplayError(component_trace)                                                         │
│                                                                                                  │
│ /home/djordacevic/.virtualenvs/h2o/lib/python3.8/site-packages/fire/core.py:475 in _Fire         │
│                                                                                                  │
│   472 │     is_class = inspect.isclass(component)                                                │
│   473 │                                                                                          │
│   474 │     try:                                                                                 │
│ ❱ 475 │   │   component, remaining_args = _CallAndUpdateTrace(                                   │
│   476 │   │   │   component,                                                                     │
│   477 │   │   │   remaining_args,                                                                │
│   478 │   │   │   component_trace,                                                               │
│                                                                                                  │
│ /home/djordacevic/.virtualenvs/h2o/lib/python3.8/site-packages/fire/core.py:691 in               │
│ _CallAndUpdateTrace                                                                              │
│                                                                                                  │
│   688 │   loop = asyncio.get_event_loop()                                                        │
│   689 │   component = loop.run_until_complete(fn(*varargs, **kwargs))                            │
│   690   else:                                                                                    │
│ ❱ 691 │   component = fn(*varargs, **kwargs)                                                     │
│   692                                                                                            │
│   693   if treatment == 'class':                                                                 │
│   694 │   action = trace.INSTANTIATED_CLASS                                                      │
│                                                                                                  │
│ /home/djordacevic/h2ogpt/src/gen.py:719 in main                                                  │
│                                                                                                  │
│    716 │   │   │   caption_loader = False                                                        │
│    717 │   │                                                                                     │
│    718 │   │   # assume gradio needs everything                                                  │
│ ❱  719 │   │   go_gradio(**locals())                                                             │
│    720                                                                                           │
│    721                                                                                           │
│    722 def get_config(base_model,                                                                │
│                                                                                                  │
│ /home/djordacevic/h2ogpt/src/gradio_runner.py:360 in go_gradio                                   │
│                                                                                                  │
│    357 │   │   │   │   │   │   interactive=True,                                                 │
│    358 │   │   │   │   │   │   visible=False)  # WIP                                             │
│    359 │   │   │   col_tabs = gr.Column(elem_id="col_container", scale=10)                       │
│ ❱  360 │   │   │   with (col_tabs, gr.Tabs()):                                                   │
│    361 │   │   │   │   with gr.TabItem("Chat"):                                                  │
│    362 │   │   │   │   │   if kwargs['langchain_mode'] == 'Disabled':                            │
│    363 │   │   │   │   │   │   text_output_nochat = gr.Textbox(lines=5, label=output_label0, sh  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: __enter__

[pseudotensor]

Please see this. Perhaps you also used torch v1?

https://github.com/h2oai/h2ogpt/issues/461#issuecomment-1636886554