Cannot run as nonroot

[Rudd-O]

Describe the bug

After typing the first prompt, my own prompt and the assistant's response (onscreen says Typing...) vanish.

Traceback:

Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:37<00:00, 18.99s/it]
Loaded the model in 38.28 seconds.
Loading the extension "gallery"... Ok.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Traceback (most recent call last):
  File "/opt/oobabooga_linux/text-generation-webui/modules/callbacks.py", line 66, in gentask
    ret = self.mfunc(callback=_callback, **self.kwargs)
  File "/opt/oobabooga_linux/text-generation-webui/modules/text_generation.py", line 252, in generate_with_callback
    shared.model.generate(**kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py", line 1485, in generate
    return self.sample(
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/transformers/generation/utils.py", line 2524, in sample
    outputs = self(
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py", line 662, in forward
    outputs = self.gpt_neox(
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py", line 553, in forward
    outputs = layer(
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py", line 320, in forward
    attention_layer_outputs = self.attention(
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/transformers/models/gpt_neox/modeling_gpt_neox.py", line 116, in forward
    qkv = self.query_key_value(hidden_states)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`
Output generated in 1.51 seconds (0.00 tokens/s, 0 tokens, context 44, seed 2096995086)

There should be at least a check that says "Cannot access NVIDIA card" or something to that effect.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Run the start script as nonroot on a system with an NVIDIA card where oobabooga has been set up to use the NVIDIA card at setup time.

Screenshot

No response

Logs

The traceback is above.

System Info

Fedora 37 x86_64, 2080Ti card, 6.9b Pythia model.

[mcmonkey4eva]

I'm running as not-root on a Ubuntu install without issue. I have seen this error before, but it's usually resolved by closing my terminal window and opening a new one, usually only happens after I've done something funky with conda that breaks the state.

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.