Entrypoint script not working even though present.

[Limpan]

Issue Description

The following error is thrown when trying to start container.

{"msg":"exec container process (missing dynamic library?) `/app/entrypoint.sh`: No such file or directory","level":"error","time":"2024-05-22T09:49:15.605528Z"}

Steps to reproduce the issue

Steps to reproduce the issue

1. Clone repo

   git clone https://gitlab.com/arbetsformedlingen/joblinks/data-aggregator.git

2. Check out commit

   git reset --hard 0002619f19d973cb5311a3a317440613ed04131c

3. Build container

   podman build -t data_aggregator:latest .

4. Run container

   podman run --rm data_aggregator:latest

Describe the results you received

The container would not start.

Describe the results you expected

Expected the container to start.

podman info output

host:
  arch: amd64
  buildahVersion: 1.35.3
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.10-1.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 99.6
    systemPercent: 0.21
    userPercent: 0.2
  cpus: 8
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: container
    version: "40"
  eventLogger: journald
  freeLocks: 2048
  hostname: 98-31465
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 5.15.146.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 10061709312
  memTotal: 16609513472
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.10.0
    package: netavark-1.10.3-3.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.10.3
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.14.4
      commit: a220ca661ce078f2c37b38c92e66cf66c012d9c1
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240510.g7288448-1.fc40.x86_64
    version: |
      pasta 0^20240510.g7288448-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 4294967296
  swapTotal: 4294967296
  uptime: 42h 58m 56.00s (Approximately 1.75 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 1081101176832
  graphRootUsed: 4802510848
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 234
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 5.0.2
  Built: 1713312000
  BuiltTime: Wed Apr 17 02:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.1
  Os: linux
  OsArch: linux/amd64
  Version: 5.0.2

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

Yes

Additional environment details

Using Podman Desktop on Windows 10 Enterprise, 22H2.

Additional information

$ podman run --rm --entrypoint "/bin/sh" data_aggregator:latest -c "ls -la /app && cat /app/entrypoint.sh"
total 16
drwxr-xr-x 1 root root 4096 May 22 09:49 .
dr-xr-xr-x 1 root root 4096 May 22 10:00 ..
drwxr-xr-x 4 root root 4096 May 22 09:49 .venv
-rwxrwxr-x 1 root root  117 May 22 09:48 entrypoint.sh
#!/bin/sh

set -e

# Activate virtual environment
. /app/.venv/bin/activate

# Run passed command
exec "$@"

[Luap99]

Please run podman machine ssh and then install strace and then podman under strace strace -f - o log podman run ...

The log file should tell us more what exactly is missing.

[Limpan]

Ran strace -f -o log.txt podman run --rm data_aggregator:latest from within the podman machine and got the following output and strace.log.

WARN[0000] Using cgroups-v1 which is deprecated in favor of cgroups-v2 with Podman v5 and will be removed in a future version. Set environment variable `PODMAN_IGNORE_CGROUPSV1_WARNING` to hide this warning.
{"msg":"exec container process (missing dynamic library?) `/app/entrypoint.sh`: No such file or directory","level":"error","time":"2024-06-17T11:03:51.653143Z"}

[Luap99]

strange, your reproducer works fine for me on linux as well. The strace doesn't show us much more either other than it cannot find /app/entrypoint.sh which the error already shows.

Do you store you files with windows line endings? I think it is possible that you entrypoint file has CR+LF windows line endings instead of LF linux line endings, as such the shebang which is execute when the script is loaded is not the normal /bin/sh but rather /bin/sh\r which of course does not exists.

[Limpan]

No carriage returns in entrypoint.sh, only line feed.

[Luap99]

Did you run something like cat -e /app/entrypoint.sh in the container to confirm? As I said it works fine on linux so I don't really see another explanation and when I changed the file to CR+LF it failed exactly in the way it does for you.

[Limpan]

Interesting. I think I understand what is happening. I did the check in my linux vm and there I have LF endings but the container I built from Windows I have CR+LF endings.

C:\Users\torlv>podman run --rm -it --entrypoint "" data_aggregator:latest /bin/sh -c "cat -e /app/entrypoint.sh"
#!/bin/sh^M$
^M$
set -e^M$
^M$
# Activate virtual environment^M$
. /app/.venv/bin/activate^M$
^M$
# Run passed command^M$
exec "$@"^M$

The reason seems to be the autocrlf feature of git. It changed line endings and then the shebang of my entrypoint script broke.

C:\Users\torlv>git config core.autocrlf
true

Turning off gits autocrlf so called feature with git config --global core.autocrlf false, cloning the repo again and building it works as expected.

Thank you for your time! Sorry that the issue wasn't related to podman.