Variable Template system

[genefyx]

Can we have a standardized template system. Reference: https://github.com/SelfhostedPro/Yacht

This will allow us to use standardized templates Ex: https://raw.githubusercontent.com/silopolis/generate-lsio/yacht_templates_generator/templates-yacht.json https://raw.githubusercontent.com/SelfhostedPro/selfhosted_templates/yacht/Template/template.json

These templates are both compatible with Portainer and Yacht and will allow standardized importing & conversion of apps to CasaOS who have already been using this system.

[WisdomSky]

Would be nice if we have an environment variable that assigns us an available port in host so that the user would have a more seamless installation rather than being prompted with an error if the installation failed due to already occupied port.

I'm picturing something like...

name: Heimdall
services:
  heimdall:
    image: linuxserver/heimdall:2.5.6
    restart: unless-stopped
    environment:
      PUID: $PUID
      PGID: $PGID
      TZ: $TZ
    ports:
      - target: 80
        published: $AVAIL_TCP_PORT
        protocol: tcp
....
x-casaos:
  main: heimdall
 ...
  title:
    en_us: Heimdall
  category: Web
  port_map: $AVAIL_TCP_PORT

Or since this will be mostly used for WebUI ports. Then why not just name it $WEBUI_PORT

name: Heimdall
services:
  heimdall:
    image: linuxserver/heimdall:2.5.6
    restart: unless-stopped
    environment:
      PUID: $PUID
      PGID: $PGID
      TZ: $TZ
    ports:
      - target: 80
        published: $WEBUI_PORT
        protocol: tcp
....
x-casaos:
  main: heimdall
 ...
  title:
    en_us: Heimdall
  category: Web
  port_map: $WEBUI_PORT

[tigerinus]

Good idea!

[tigerinus]

I think the challenge is expanding variable in the ports property of Docker Compose, which I don't think is supported by Docker Compose engine.

Currently var expansion is only supported in environments if I remember correctly.

CasaOS tries not to interfere how Docker Compose file is interpreted.

[tigerinus]

OR, I could be wrong:

https://docs.docker.com/compose/compose-file/12-interpolation/

[WisdomSky]

I think it applies to every part of the compose file. We regularly use this to make the published port configurable by the user by utilizing an .env file (docker supports env files for further configurations).

One popular usage is by Laravel Sail: https://github.com/laravel/sail/blob/1.x/stubs/docker-compose.stub#L14

[WisdomSky]

Another example is one of my private repo:

Passing environment variables to the command, in docker-compose

[WisdomSky]

Docker also allows us to even make the image tag/version configurable as seen in this example in their documentation.

It would be even great if CasaOS will allow casaos app developers to somehow make versions configurable by the user. I mean not by letting the user put their own version per se, but allow us to specify versions that the user can select from.

for example:

name: MongoDB
services:
  mongo:
    image: mongo:${MONGO_VERSION}
    restart: unless-stopped
    environment:
      PUID: $PUID
      PGID: $PGID
      TZ: $TZ
    ...
x-casaos:
...
  main: mongo
...
  title:
    en_us: MongoDB
  install_vars:
     - name: MONGO_VERSION
       label:
         en_us:  "Select MongoDB version to use:"
       type: select
       default: 6.0.6
       options:
         - label: 
             en_us: "4"
           value: 4.4.6
         - label:
             en_us: "5"
           value: 5.3
         - label:
             en_us: "6"
           value: 6.0.6

Then during installation, the user will be prompted with a dropdown menu with those following options.

This way, we can let the user decide which major version he/she wants to install while also only specifying the version. This will also solve the issue of other users who can't use the newer (major) versions of an app.

It could also be structured this way, so there'll be less to write:

name: MongoDB
services:
  mongo:
    image: mongo:${MONGO_VERSION}
    ...
x-casaos:
...
  main: mongo
...
  title:
    en_us: MongoDB
  install_vars:
     MONGO_VERSION:
       label: 
         en_us: "Select MongoDB version to use:"
       type: select # I'm thinking having select and text as available types would suffice. 
       default: 6.0.6
       options:
         - 4: 4.4.6
         - 5: 5.3
         - 6: 6.0.6

Example for text type:

name: Cloudflare-Tunnel
services:
  cloudflare:
    image: cloudflare/cloudflared
    command: tunnel run
    environment:
      TUNNEL_TOKEN: $TUNNEL_TOKEN
    ...
x-casaos:
...
  main: cloudflare
...
  title:
    en_us: CloudFlare Tunnel
  install_vars:
     TUNNEL_TOKEN:
       label:
         en_us:  "Cloudflare Tunnel Token:"
       type: text
       default: "some value"

[tigerinus]

This is great.

It's just that I am always obsessed with this idea that even without CasaOS, anyone with docker installed can bring up and run any CasaOS App.

Guess this idea has to change due to introducing CasaOS specific vars.

[WisdomSky]

Actually they can,

We (the one who made the docker-compose file), will just include an env_file attribute into our docker-compose.yaml together with an .env as a fallback. so running the image without casaos is still possible and sails smoothly for other non-casaos users.

docker-compose.yml file:

name: Cloudflare-Tunnel
services:
  cloudflare:
    image: cloudflare/cloudflared
    env_file: 
      - config.env
    command: tunnel run
    environment:
      TUNNEL_TOKEN: $TUNNEL_TOKEN
    ...
x-casaos:
...
  main: cloudflare
...
  title:
    en_us: CloudFlare Tunnel
  install_vars:
     TUNNEL_TOKEN:
       label:
         en_us:  "Cloudflare Tunnel Token:"
       type: text
       default: "some value"

config.env file:

TUNNEL_TOKEN=XXXXXXX

and the best thing is, docker is smart in handling environment variables. the env_file will take the least precedence, so there's no worry that the env file will over write whatever environment variable you pass to it.

[tigerinus]

Cool.

For next step, can you list some CasaOS specific variables in your mind?

I might not be able to get to this anytime soon, but I really like this idea.

[WisdomSky]

Few variable I have in mind that would be really useful is:

$WEBUI_PORT A special variable that passes an available port to the compose file. so that we can avoid these multiple apps fighting over port 80 or 8080.

$APPDATA A variable that contains a CasaOS-generated path for the following app. instead of giving the job of specifying the path to the developer. Right now, we have to manually write /DATA/AppData/$appId, which for uninformed developers can be problematic. Basically $APPDATA is just an alias to /DATA/AppData/$appId

I think these two are what I have on mind right now.

[tigerinus]

Currently the existing vars are defined at https://github.com/IceWhaleTech/CasaOS-AppManagement/blob/918ded92c20738eb070b52787ca68d6fde8320b5/service/compose_service.go#L190-L198

They are all static.

Someone could update this and relevant logic to include the dynamic vars you mentioned, before I am freed from current swamp.

[WisdomSky]

As much as I would love to work on this one, I practically have zero experience in Golang. :/ That's why what I can only do is suggest. 😅 I would have to learn a bit of go before I can even start to dive down and help.

But I do try to contribute on the UI side of things.

[CorrectRoadH]

Actually they can,

We (the one who made the docker-compose file), will just include an env_file attribute into our docker-compose.yaml together with an .env as a fallback. so running the image without casaos is still possible and sails smoothly for other non-casaos users.

docker-compose.yml file:

name: Cloudflare-Tunnel
services:
  cloudflare:
    image: cloudflare/cloudflared
    env_file: 
      - config.env
    command: tunnel run
    environment:
      TUNNEL_TOKEN: $TUNNEL_TOKEN
    ...
x-casaos:
...
  main: cloudflare
...
  title:
    en_us: CloudFlare Tunnel
  install_vars:
     TUNNEL_TOKEN:
       label:
         en_us:  "Cloudflare Tunnel Token:"
       type: text
       default: "some value"

config.env file:

TUNNEL_TOKEN=XXXXXXX

and the best thing is, docker is smart in handling environment variables. the env_file will take the least precedence, so there's no worry that the env file will over write whatever environment variable you pass to it.

May I have the document or tutorial source of config.env? I can't reproduce it😭 I create a config.env and run docker compose up a docker compose. but the env of config.env didn't have any effect.

[WisdomSky]

@CorrectRoadH what docker image are you trying tocrun? can you post your whole docker-compose.yanl here also the contents of your config.env?

I already linked the docker documentation above regarding using of env files. Just check my previous comments.

[CorrectRoadH]

@CorrectRoadH what docker image are you trying tocrun? can you post your whole docker-compose.yanl here also the contents of your config.env?

I already linked the docker documentation above regarding using of env files. Just check my previous comments.

memos

// docker-compose.yml
name: memos
services:
  memos:
    environment:
      PGID: $PGID
      PUID: $PUID
      TZ: $TZ
    image: neosmemo/memos:0.13.1
    deploy:
      resources:
        reservations:
          memory: 64M
    network_mode: bridge
    ports:
      - target: 5230
        published: $WEBUI_PORT
        protocol: tcp
    restart: unless-stopped
    volumes:
      - type: bind
        source: [you need to change this]
        target: /var/opt/memos
    x-casaos:
      envs:
        - container: PUID
          description:
            en_us: ""
        - container: PGID
          description:
            en_us: ""
        - container: TZ
          description:
            en_us: "timezone"
            zh_cn: "时区"
      ports:
        - container: "5230"
          description:
            en_us: WebUI HTTP Port
            zh_cn: WebUI HTTP 端口
      volumes:
        - container: /var/opt/memos
          description:
            en_us: memos data directory.
            zh_cn: memos 数据目录

x-casaos:
  architectures:
    - amd64
    - arm64
    - arm
  main: memos
  author: usememos Team
  category: Notes
  description:
    en_us: Memos is a lightweight, self-hosted memo hub. Open Source and Free forever.
    zh_cn: Memos 是一个轻量级的自托管Memos中心。 开源且永远免费。
  developer: usememos Team
  icon: https://cdn.jsdelivr.net/gh/IceWhaleTech/CasaOS-AppStore@main/Apps/Memos/icon.png
  screenshot_link:
    - https://cdn.jsdelivr.net/gh/IceWhaleTech/CasaOS-AppStore@main/Apps/Memos/screenshot-1.png
    - https://cdn.jsdelivr.net/gh/IceWhaleTech/CasaOS-AppStore@main/Apps/Memos/screenshot-2.png
  tagline:
    en_us: Memos is a lightweight, self-hosted memo hub. Open Source and Free forever.
    zh_cn: Memos 是一个轻量级的自托管Memos中心。 开源且永远免费。
  thumbnail: https://cdn.jsdelivr.net/gh/IceWhaleTech/CasaOS-AppStore@main/Apps/Memos/thumbnail.png
  tips: {}
  title:
    en_us: Memos
  port_map: "5230"

//config.env
WEBUI_PORT=7789

shell

[WisdomSky]

You forgot to add the env_file attribute in your docker-compose.yaml file.

[WisdomSky]

[CorrectRoadH]

It seems didn't work in my computer🥲

[WisdomSky]

try using the default env file name. Rename it into: .env instead of config.env

[CorrectRoadH]

try using the default env file name. Rename it into: .env instead of config.env

It is work now! 😋 Thanks you

[WisdomSky]

No problem. ☺️

[tigerinus]

@CorrectRoadH is working on the $WEBUI_PORT feature:

• https://github.com/IceWhaleTech/CasaOS-AppManagement/pull/113

For the .env support (or any other supporting files in addition to docker-compose.yml), I'd like to support apps in .zip format, which includes all of docker-compose.yml and other files, in future.

[genefyx]

Thank you for adding the template system. In testing this I saw that I forgot to mention the main feature of a templating system outside of compatibility. The ability to set custom defaults. I have multiple drives merged drives & custom default locations, and cannot easily rebuild due to the CasaOS' variables already being pre-defined. I included a screenshot below for yacht default customization.