1.10.1: API_ENDPOINT doesn't seem to be respected in web client

[samip5]

Describe the bug

It seems that the API_ENDPOINT env variable is not being respected by the web client for unknown reasons?

Expected behavior

I would have expected it to be enough to set the API_ENDPOINT to the one the API is reacheable at without port, so eg https://s-api.skylab.fi but it still tries to use localhost instead.

Additional context

k exec -it -n default your-spotify-web-c854ddd8-skclb -- sh
/app $ env
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://10.96.0.1:443
NODE_VERSION=20.11.1
HOSTNAME=your-spotify-web-c854ddd8-skclb
YARN_VERSION=1.22.19
SHLVL=1
HOME=/home/node
API_ENDPOINT=https://s-api.skylab.fi
TIMEZONE=Europe/Helsinki
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_HOST=10.96.0.1
PWD=/app

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: your-spotify
spec:
  interval: 15m
  chart:
    spec:
      chart: app-template
      version: 3.3.2
      sourceRef:
        kind: HelmRepository
        name: bjw-s
        namespace: flux-system

  driftDetection:
    mode: enabled
  maxHistory: 3
  install:
    remediation:
      retries: 3
  upgrade:
    cleanupOnFail: true
    remediation:
      strategy: rollback
      retries: 3
  values:
    defaultPodOptions:
      automountServiceAccountToken: false
      securityContext:
        runAsUser: ${APP_UID}
        runAsGroup: ${APP_GID}
        fsGroup: ${APP_GID}
        fsGroupChangePolicy: OnRootMismatch
      nodeSelector:
        kubernetes.io/arch: amd64
    controllers:
      server:
        strategy: &strat RollingUpdate
        rollingUpdate: &ru
          unavailable: 0

        pod: &pod
          annotations:
            reloader.stakater.com/auto: 'true'

        containers:
          app:
            image:
              repository: yooooomi/your_spotify_server
              tag: 1.10.1
            env:
              TIMEZONE: "Europe/Helsinki"

              # Caveat if it includes Spotify in name: https://github.com/Yooooomi/your_spotify/pull/254
              API_ENDPOINT: &api_endpoint https://s-api.skylab.fi
              CLIENT_ENDPOINT: &client https://s.skylab.fi
              CORS: *client
              MONGO_ENDPOINT: mongodb://your-spotify-mongodb:27017/your_spotify

            envFrom:
              - secretRef:
                  name: your-spotify-secret

            probes: &probes
              liveness:
                enabled: true
              readiness:
                enabled: true
              startup:
                enabled: true

            securityContext: &securityContext
              allowPrivilegeEscalation: false
              readOnlyRootFilesystem: false
              capabilities: { drop: [ 'ALL' ] }

      web:
        strategy: *strat
        rollingUpdate: *ru
        pod: *pod

        containers:
          app:
            image:
              repository: yooooomi/your_spotify_client
              tag: 1.10.1

            env:
              TIMEZONE: "Europe/Helsinki"
              API_ENDPOINT: *api_endpoint
            probes: *probes
            securityContext: *securityContext

      mongodb:
        type: statefulset
        rollingUpdate: *ru
        statefulset:
          podManagementPolicy: OrderedReady

        containers:
          app:
            image:
              repository: docker.io/library/mongo
              tag: 6.0.4-focal

            env:
              TZ: ${TIMEZONE}

            probes: *probes

    service:
      server:
        controller: server
        primary: true
        ports:
          http:
            port: 8080
      web:
        controller: web
        ports:
          http:
            port: 3000
      mongodb:
        controller: mongodb
        ports:
          http:
            port: 27017

    ingress:
      server-int:
        className: internal-nginx
        hosts:
          - host: s-api.skylab.fi
            paths:
              - path: /
                pathType: Prefix
                service:
                  identifier: server
                  port: http

      web-int:
        className: internal-nginx
        hosts:
          - host: s.skylab.fi
            paths:
              - path: /
                pathType: Prefix
                service:
                  identifier: web
                  port: http

    persistence:
      mongo:
        existingClaim: "${VOLSYNC_CLAIM}"
        advancedMounts:
          mongodb:
            app:
              - path: /data/db

Screenshots

[samip5]

It seems relevant:

k logs your-spotify-web-c854ddd8-skclb
cp: can't create '/app/apps/client/build/variables.js': File exists
Setting API Endpoint to 'https://s-api.skylab.fi'
sed: can't create temp file '/app/apps/client/build/variables.jsXXXXXX': Permission denied
sed: can't create temp file '/app/apps/client/build/index.htmlXXXXXX': Permission denied
sed: can't create temp file '/app/apps/client/build/index.htmlXXXXXX': Permission denied
sed: can't create temp file '/app/apps/client/scripts/run/serve.jsonXXXXXX': Permission denied

ls -al /app/apps/client/
total 4
drwxr-xr-x    1 root     root            19 Mar 24 00:10 .
drwxr-xr-x    1 root     root            20 Mar 24 00:10 ..
drwxr-xr-x    3 root     root          4096 Mar 24 00:10 build
drwxr-xr-x    4 root     root            30 Mar 24 00:07 scripts

I'm running it as unprivileged, so that's a problem.

[Yooooomi]

Hello, before starting the client server, it overrides some of the built files according to your environment. It seems like it gets permission denied in your case, which is why the files stay untouched and the client gets the default value of localhost.

[samip5]

Hello, before starting the client server, it overrides some of the built files according to your environment. It seems like it gets permission denied in your case, which is why the files stay untouched and the client gets the default value of localhost.

Which should be a) documented and b) not require root permissions so it should probably use an unprivileged user instead of eg node aka uid 1000 which seems to exist in the container.

[RagingCactus]

Using a non-root user in the container images would be nice, I agree. I assume if you submit a pull request implementing that, it would have a good chance of being merged.

However, I disagree with your notion that you expect a container image to work when you change the UID that is being used. In general, you will have trouble finding container images that still work after you change the UID from the outside like that. Many images (including typical web servers and "official" library images) rewrite configuration files in the entrypoint script when they start up. When you change the user ID from the outside, that will fail in almost all cases.

[samip5]

rewrite configuration files in the entrypoint script

Which is not really great, same goes for s6-overlay (as a lot of images use either of these things). I even tried to mount just that specific path properly but as the whole app lives there, it's not really workable.

[Yooooomi]

I'm afraid if I change the UID of the user in the container now, all the current instances of YourSpotify will break. If you have any idea on how to proceed I would be more than happy to implement.

[samip5]

I'm afraid if I change the UID of the user in the container now, all the current instances of YourSpotify will break. If you have any idea on how to proceed I would be more than happy to implement.

It should be enough to just change the group of the files/folders so that root AND eg group users or node or something can write.

[RagingCactus]

I'm afraid if I change the UID of the user in the container now, all the current instances of YourSpotify will break.

Neither the backend nor the frontend containers should have persistent data volumes in most deployments. The sample compose files don't define volumes for these containers, and I don't think any user has reason to define volumes for them, as the MongoDB handles all data persistence.

Can you elaborate on how changing the user in the Dockerfile would break existing installations on update?

[Yooooomi]

You're right. I was thinking about changing the UID of the mongo too but it is not needed so I think we could change the UID yeah. Might try that when I have a bit of time.

[RagingCactus]

The mongodb container image starts as root, but drops its permissions to a different user in the entrypoint script, so that's already handled by the MongoDB maintainers. (Which, by the way, is great example why forcing a different UID onto a container in the compose file/kubernetes deployment breaks existing images in 99% of cases.)

[samip5]

The mongodb container image starts as root, but drops its permissions to a different user in the entrypoint script, so that's already handled by the MongoDB maintainers.

(Which, by the way, is great example why forcing a different UID onto a container in the compose file/kubernetes deployment breaks existing images in 99% of cases.)

Which is an example of an s6-overlay when it does that.