unknown flag: --device

[amd1890]

I am getting an error when I run this in the terminal showing

Error: unknown flag: --device

I am still trying to figure out if this is running correctly and probably need to install a web-ui interface to see if it works at all. It's possible this error is trivial.

[hqnicolas]

@amd1890 It's not updated, I will fix this REPO tomorrow...

this error looks strage to me: Docker Docs

docker run  --device=/dev/dri/card0 --device=/dev/kfd -d -v /DATA/Downloads:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

I thin It's an issue with updated version of docker

try to import this "ollama.yaml" to casaos

name: recursing_werner
services:
  ollama:
    cpu_shares: 90
    command: []
    container_name: ollama
    deploy:
      resources:
        limits:
          memory: 31857M
    devices:
      - /dev/dri/card0:/dev/dri/card0
      - /dev/kfd:/dev/kfd
    environment:
      - OLLAMA_WEBAPI_PORT-11434=11434
    hostname: ollama
    image: ollama/ollama:0.4.1-rc0-rocm
    labels:
      icon: https://icon.casaos.io/main/all/ollama.png
    ports:
      - target: 11434
        published: "11434"
        protocol: tcp
    privileged: true
    restart: unless-stopped
    volumes:
      - type: bind
        source: /DATA/Downloads
        target: /root/.ollama
    cap_add: []
    network_mode: bridge
x-casaos:
  author: self
  category: self
  hostname: ""
  icon: https://icon.casaos.io/main/all/ollama.png
  index: /
  is_uncontrolled: false
  port_map: ""
  scheme: http
  store_app_id: recursing_werner
  title:
    custom: ollama

webui.yaml

name: ollama-webui
services:
  ollama-webui:
    cpu_shares: 50
    command:
      - bash
      - start.sh
    container_name: ollama-webui
    deploy:
      resources:
        limits:
          memory: 4096M
    environment:
      - ENV=prod
      - GPG_KEY=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAa
      - LANG=C.UTF-8
      - OLLAMA_BASE_URL=http://192.168.1.15:11434
      - OPENAI_API_BASE_URL=http://192.168.1.15:11434/v1/chat/completions
      - OPENAI_API_KEY=
      - PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
      - PORT=
      - PYTHON_GET_PIP_SHA256=7cfd4bdc4d475ea971f1c0710a5953bcc704d171f83c797b9529d9974502fcc6
      - PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/049c52c665e8c5fd1751f942316e0a5c777d304f/public/get-pip.py
      - PYTHON_PIP_VERSION=23.2.1
      - PYTHON_SETUPTOOLS_VERSION=65.5.1
      - PYTHON_VERSION=3.11.7
      - WEBUI_SECRET_KEY=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    hostname: ollama-webui
    image: ghcr.io/open-webui/open-webui:git-09935d1
    ports:
      - target: 8080
        published: "3003"
        protocol: tcp
    restart: always
    volumes:
      - type: bind
        source: /DATA/Downloads/ollama-ui
        target: /app/backend/data
    x-casaos:
      envs:
        - container: OLLAMA_API_BASE_URL
          description:
            en_us: ""
        - container: PATH
          description:
            en_us: ""
        - container: LANG
          description:
            en_us: ""
        - container: GPG_KEY
          description:
            en_us: ""
        - container: PYTHON_VERSION
          description:
            en_us: ""
        - container: PYTHON_PIP_VERSION
          description:
            en_us: ""
        - container: PYTHON_SETUPTOOLS_VERSION
          description:
            en_us: ""
        - container: PYTHON_GET_PIP_URL
          description:
            en_us: ""
        - container: PYTHON_GET_PIP_SHA256
          description:
            en_us: ""
        - container: ENV
          description:
            en_us: ""
        - container: PORT
          description:
            en_us: ""
        - container: OPENAI_API_BASE_URL
          description:
            en_us: ""
        - container: OPENAI_API_KEY
          description:
            en_us: ""
        - container: WEBUI_SECRET_KEY
          description:
            en_us: ""
      volumes:
        - container: /app/backend/data
          description:
            en_us: ""
    devices: []
    cap_add: []
    network_mode: bridge
    privileged: false
x-casaos:
  architectures:
    - arm64
  author: CasaOS User
  category: unknown
  description:
    en_us: ""
  developer: unknown
  hostname: ""
  icon: ""
  index: /
  is_uncontrolled: false
  main: ollama-webui
  port_map: "3003"
  scheme: http
  store_app_id: ollama-webui
  tagline:
    en_us: OpenWebUi
  thumbnail: ""
  tips:
    custom: OpenWebUi
  title:
    custom: ollama-webui
    en_us: ollama-webui

you can Install new AMD drivers using this repo: https://github.com/hqnicolas/bmaltaisKohya_ssROCm

[hqnicolas]

@amd1890 did you manage to install Ollama?

[amd1890]

My skill level is very low compared with a typical github user. I'm not a developer and have taken 1 computer programming class years ago and don't read all the documentation fully for every program I use due to a lack of time.

I don't know how to import a yaml into casaos or docker or really understand what a yaml is. I am very new to docker.

The main reason I am trying to get this to work is that I would like to run Perplexica, a github open-source program, but I want to use it with Vulkan enabled. When I use GPT4ALL, a local AI program, it utilizes Vulkan. When I am using Perplexica, it only uses 1 token at a time using regular ollama in a docker.

Is this updated now so I can just follow the instructions again and this error will not happen? I'll try to follow the instructions again but am not sure how to implement the Yaml stuff you wrote back about.

[hqnicolas]

Let's start from the basics: you are on a Windows machine with a ROCm video card enabled? you are on linux with a ROCm board? What is your card series and what version of drivers are you using? did you manage to install rocm 5.6, 6.0 or 6.2? https://ollama.com/ https://github.com/ItzCrazyKns/Perplexica

[amd1890]

AMD® Ryzen 7 8840u w/ radeon 780m graphics × 16 AMD® Radeon graphics Pop!_OS 22.04 LTS 42.9 Wayland

I am on linux with a Ryzen processor

I don't know what a card series is. I have an AMD GPU that has a 780 radeon built into it

dpkg -l rocm ii rocm 6.1.0.60100-82~22.04 amd64 Radeon Open Compute (ROCm) software stack meta package

i have installed perplexica and ollama before and they run but they are too slow and in CPU mode.

[hqnicolas]

@amd1890 access: https://hub.docker.com/r/ollama/ollama Look at the "AMD GPU" section

docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm