Puzzling TB when insufficient memory

From Slack:

Hi folks, I have installed on my Fedora-39 logdetective by pip3 install logdetective. It works. By the next step I run command

$ logdetective /var/tmp/daily_scl_tests/rhel8-test/log.txt
mistral-7b-instruct-v0.2.Q4_K_S.gguf: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.14G/4.14G [00:23<00:00, 175MB/s]
ggml_backend_cpu_buffer_type_alloc_buffer: failed to allocate buffer of size 4294967328
llama_kv_cache_init: failed to allocate buffer for kv cache
llama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache
Traceback (most recent call last):
  File "/usr/local/bin/logdetective", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/fedora/.local/lib/python3.12/site-packages/logdetective/logdetective.py", line 52, in main
    model = initialize_model(args.model, filename_suffix=args.filename_suffix,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fedora/.local/lib/python3.12/site-packages/logdetective/utils.py", line 62, in initialize_model
    model = Llama.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fedora/.local/lib/python3.12/site-packages/llama_cpp/llama.py", line 2091, in from_pretrained
    return cls(
           ^^^^
  File "/home/fedora/.local/lib/python3.12/site-packages/llama_cpp/llama.py", line 372, in __init__
    self._ctx = self._stack.enter_context(contextlib.closing(_LlamaContext(
                                                             ^^^^^^^^^^^^^^
  File "/home/fedora/.local/lib/python3.12/site-packages/llama_cpp/_internals.py", line 285, in __init__
    raise ValueError("Failed to create llama_context")
ValueError: Failed to create llama_context

What’s is wrong? I would like to use logdetective to analyze container logs. Some of them were already uploaded (edited)

The cause is insufficient memory. We should catch this error and print friendly error to user.

fedora-copr / logdetective

Puzzling TB when insufficient memory #42