search

Jeffser / Alpaca

An Ollama client made with GTK4 and Adwaita
https://jeffser.com/alpaca
GNU General Public License v3.0
390 stars 41 forks source link

Accessibility: Have Alpacca announce messages from the LLM #296

Open devinprater opened 1 month ago

devinprater commented 1 month ago

Describe the bug I'm sorry if this is a duplicate, I searched for accessibility and didn't find anything like this. When using Alpacca with the Orca screen reader, Orca does not announce messages from the LLM.

Expected behavior When the LLM is done writing the message, or if the messages stream, when a sentence or so is finished being written, Orca should be announcing the output.

For ways of doing this depending on which GTK you're using, this may help. If this isn't possible, maybe play a sound when generation is done.

Screenshots N/a

Debugging information

INFO    [main.py | main] Alpaca version: 2.0.0
INFO    [connection_handler.py | start] Starting Alpaca's Ollama instance...
INFO    [connection_handler.py | start] Started Alpaca's Ollama instance
2024/09/11 08:08:50 routes.go:1125: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11435 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/devin/.var/app/com.jeffser.Alpaca/data/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-09-11T08:08:50.340-05:00 level=INFO source=images.go:753 msg="total blobs: 5"
time=2024-09-11T08:08:50.340-05:00 level=INFO source=images.go:760 msg="total unused blobs removed: 0"
time=2024-09-11T08:08:50.340-05:00 level=INFO source=routes.go:1172 msg="Listening on 127.0.0.1:11435 (version 0.3.9)"
time=2024-09-11T08:08:50.340-05:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners
time=2024-09-11T08:08:50.340-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/libggml.so.gz
time=2024-09-11T08:08:50.340-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/libllama.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/libggml.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/libllama.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/libggml.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/libllama.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libggml.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libllama.so.gz
INFO    [connection_handler.py | start] Ollama version: 0.3.9
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v12 file=build/linux/x86_64/cuda_v12/bin/libggml.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v12 file=build/linux/x86_64/cuda_v12/bin/libllama.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v12 file=build/linux/x86_64/cuda_v12/bin/ollama_llama_server.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/libggml.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/libllama.so.gz
time=2024-09-11T08:08:50.341-05:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/ollama_llama_server.gz
INFO    [connection_handler.py | request] GET : http://127.0.0.1:11435/api/tags
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/cpu/ollama_llama_server
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/cpu_avx/ollama_llama_server
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/cpu_avx2/ollama_llama_server
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/cuda_v11/ollama_llama_server
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/cuda_v12/ollama_llama_server
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/rocm_v60102/ollama_llama_server
time=2024-09-11T08:08:57.319-05:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu_avx cpu_avx2 cuda_v11 cuda_v12 rocm_v60102 cpu]"
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=sched.go:105 msg="starting llm scheduler"
time=2024-09-11T08:08:57.319-05:00 level=INFO source=gpu.go:200 msg="looking for compatible GPUs"
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=gpu.go:86 msg="searching for GPU discovery libraries for NVIDIA"
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcuda.so*
time=2024-09-11T08:08:57.319-05:00 level=DEBUG source=gpu.go:491 msg="gpu library search" globs="[/app/lib/ollama/libcuda.so* /app/lib/libcuda.so* /usr/lib/x86_64-linux-gnu/GL/default/lib/libcuda.so* /usr/lib/x86_64-linux-gnu/openh264/extra/libcuda.so* /usr/lib/x86_64-linux-gnu/openh264/extra/libcuda.so* /usr/lib/sdk/llvm15/lib/libcuda.so* /usr/lib/x86_64-linux-gnu/GL/default/lib/libcuda.so* /usr/lib/ollama/libcuda.so* /app/plugins/AMD/lib/ollama/libcuda.so* /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
time=2024-09-11T08:08:57.321-05:00 level=DEBUG source=gpu.go:525 msg="discovered GPU libraries" paths=[]
time=2024-09-11T08:08:57.321-05:00 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcudart.so*
time=2024-09-11T08:08:57.321-05:00 level=DEBUG source=gpu.go:491 msg="gpu library search" globs="[/app/lib/ollama/libcudart.so* /app/lib/libcudart.so* /usr/lib/x86_64-linux-gnu/GL/default/lib/libcudart.so* /usr/lib/x86_64-linux-gnu/openh264/extra/libcudart.so* /usr/lib/x86_64-linux-gnu/openh264/extra/libcudart.so* /usr/lib/sdk/llvm15/lib/libcudart.so* /usr/lib/x86_64-linux-gnu/GL/default/lib/libcudart.so* /usr/lib/ollama/libcudart.so* /app/plugins/AMD/lib/ollama/libcudart.so* /home/devin/.var/app/com.jeffser.Alpaca/cache/tmp/ollama/ollama1166176636/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
time=2024-09-11T08:08:57.321-05:00 level=DEBUG source=gpu.go:525 msg="discovered GPU libraries" paths="[/app/lib/ollama/libcudart.so.12.4.99 /app/lib/ollama/libcudart.so.11.3.109]"
cudaSetDevice err: 35
time=2024-09-11T08:08:57.322-05:00 level=DEBUG source=gpu.go:537 msg="Unable to load cudart" library=/app/lib/ollama/libcudart.so.12.4.99 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
cudaSetDevice err: 35
time=2024-09-11T08:08:57.324-05:00 level=DEBUG source=gpu.go:537 msg="Unable to load cudart" library=/app/lib/ollama/libcudart.so.11.3.109 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
time=2024-09-11T08:08:57.324-05:00 level=DEBUG source=amd_linux.go:371 msg="amdgpu driver not detected /sys/module/amdgpu"
time=2024-09-11T08:08:57.324-05:00 level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered"
time=2024-09-11T08:08:57.324-05:00 level=INFO source=types.go:107 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="15.3 GiB" available="9.6 GiB"
[GIN] 2024/09/11 - 08:08:57 | 200 |     264.832µs |       127.0.0.1 | GET      "/api/tags"
Jeffser commented 1 month ago

Hi, thanks for the report, Alpaca is supposed to announce the messages when the user tabs into them, I'll look into this

Jeffser commented 1 month ago

Oh I understand, you mean when a new message is received, yeah it doesn't read the message because the focus is on the text field at the bottom, I'll see what I can do