DockerHub Command Block Listed as "Experimental", But Latest Version Uses It

[F1zzyD]

Running the compose.yml file only results in this error:

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "--model": executable file not found in $PATH: unknown

I have used the .env file with no success, and I'm not sure the compose.yml file even knows to reference it; there's no mention of it anywhere. Manually adding it via environment: -.env or env_file: .env does nothing to remedy the error. I've manually specified the model and model directory in the compose.yml, yet I get the same error. This is only from using the given compose.yml.

Here is my compose.yml:

services:
  koboldcpp:
    container_name: koboldcpp
    restart: always
    volumes:
      - '/home/user/models:/app/models'
    ports:
      - '4040:80'
    image: 'noneabove1182/koboldcpp-gpu:latest'
    ulimits:
      memlock: -1
    mem_limit: 30gb
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    command:
      [
        "--model",
        "/app/models/violet_twilight_v2.gguf",
        "--port",
        "80",
        "--threads",
        "1",
        "--usecublas",
        "normal",
        "--gpulayers",
        "43"
      ]

[F1zzyD]

Wow, got it fixed... I'll leave this up as this wasn't very clear for me.

On the prebuilt page for DockerHub, it says "Experimental tag (and likely all future tags) requires now launching python3 koboldcpp.py, so my command block now looks like so". My understanding of this was that the new command block had yet to be implemented and was exclusive to the "experimental" tag. Apparently the latest image requires this new command block. I feel like that should be worded differently...

Anywho, if you're having the same issue as I was, try this one:

    command:
      [
        "python3",
        "koboldcpp.py",
        "--model",
        "/app/models/violet_twilight_v2.gguf",
        "--port",
        "80",
        "--threads",
        "1",
        "--usemlock",
        "--usecublas",
        "normal",
        "--gpulayers",
        "60"
      ]

Note: I changed lowvram to normal.

[bartowski1182]

Ha sorry.. I definitely let these repos fester, I'll try to update them!