darmentrout commented 10 months ago

Describe the bug pelias/api:master container stopped running, will not restart, throws exit code 126. No configuration changes or updates were made recently before this issue started. I deleted the contents of the data folder and redownloaded everything.

Steps to Reproduce Run the Pelias start-up script:

mkdir ./data
sed -i '/DATA_DIR/d' .env
echo 'DATA_DIR=./data' >> .env
pelias compose pull
pelias elastic start
pelias elastic wait
pelias elastic create
pelias download all
pelias prepare all
pelias import all
pelias compose up

Expected behavior The pelias api service runs successfully like the others

Environment (please complete the following information):

$ docker -v Docker version 24.0.4, build 3713ee1

$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES fb620b98b9f4 pelias/api:master "./bin/start" 35 seconds ago Restarting (126) 10 seconds ago pelias_api 99a006737abc pelias/placeholder:master "./cmd/server.sh" 35 seconds ago Up 32 seconds 0.0.0.0:4100->4100/tcp, :::4100->4100/tcp pelias_placeholder 5be98070602b pelias/pip-service:master "./bin/start" 35 seconds ago Up 32 seconds 0.0.0.0:4200->4200/tcp, :::4200->4200/tcp pelias_pip-service cb9c1671b4ea pelias/libpostal-service "/bin/wof-libpostal-…" 35 seconds ago Up 32 seconds 0.0.0.0:4400->4400/tcp, :::4400->4400/tcp pelias_libpostal 527685dd9b51 pelias/interpolation:master "./interpolate serve…" 35 seconds ago Up 32 seconds 0.0.0.0:4300->4300/tcp, :::4300->4300/tcp pelias_interpolation 73678339907b pelias/elasticsearch:7.17.9 "/bin/tini -- /usr/l…" 9 minutes ago Up 9 minutes 0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp pelias_elasticsearch

$ docker logs --tail 3 --follow --timestamps pelias_api 2023-08-08T17:38:09.136068517Z /bin/bash: ./bin/start: Permission denied 2023-08-08T17:39:09.582053068Z /bin/bash: ./bin/start: Permission denied 2023-08-08T17:40:10.074395564Z /bin/bash: ./bin/start: Permission denied

docker-compose.yml api: image: pelias/api:master container_name: pelias_api user: "${DOCKER_USER}" restart: always environment: [ "PORT=4000" ] ports: [ "4000:4000" ] volumes:

"./pelias.json:/code/pelias.json"

$ docker exec -d pelias_api ls / Error response from daemon: Container fb620b98b9f4aa1728bbfc3ef5f5581f83e7ec7a3fd201de33e36540df64ad28 is restarting, wait until the container is running

$ pelias compose top ERROR: Container fb620b98b9f4aa1728bbfc3ef5f5581f83e7ec7a3fd201de33e36540df64ad28 is restarting, wait until the container is running

missinglink commented 10 months ago

I suspect the DOCKER_USER doesn't have sufficient permissions to execute the script.

Is this something which you've been running for a long time without modification and it stopped working or is it something you've been tinkering with recently?

missinglink commented 10 months ago

@orangejulius is it possible recent changes to the docker baseimage might be affecting which users can exec scripts?

https://github.com/pelias/docker-baseimage/commits/master

missinglink commented 10 months ago

Assuming DOCKER_USER is empty/undefined in your .env file?

darmentrout commented 10 months ago

Is this something which you've been running for a long time without modification and it stopped working or is it something you've been tinkering with recently? This is something that was running uninterrupted for a long time and suddenly stopped working.

DOCKER_USER is not defined in the .env file.

missinglink commented 10 months ago

Can you please provide the image hashes you're running using docker images and posting the ones which say pelias?

darmentrout commented 10 months ago

Sure can. Thanks for your help!

$ docker images

REPOSITORY                 TAG       IMAGE ID       CREATED         SIZE
pelias/csv-importer        master    84e5e7f6eb0e   9 days ago      398MB
pelias/placeholder         master    36cf451f239f   2 weeks ago     436MB
pelias/whosonfirst         master    581acf99a630   2 weeks ago     334MB
pelias/interpolation       master    e8c115d4a542   2 weeks ago     2.61GB
pelias/polylines           master    a514b19a2c4d   2 weeks ago     405MB
pelias/pip-service         master    f1f8b366822e   2 weeks ago     392MB
pelias/schema              master    9ec159e22f19   2 weeks ago     309MB
pelias/api                 master    25c1cf7efee6   2 weeks ago     373MB
pelias/openaddresses       master    e1aae0e8c277   2 weeks ago     501MB
pelias/openstreetmap       master    dbfc617695a4   2 weeks ago     428MB
pelias/libpostal-service   latest    846cd5bdb6db   2 weeks ago     2.3GB
pelias/transit             master    f3f8bd680fc1   2 weeks ago     396MB
pelias/elasticsearch       7.17.9    a0c6e9efe3fb   4 months ago    1.19GB
pelias/csv-importer        <none>    52d0a9313ac3   7 months ago    368MB
pelias/openaddresses       <none>    644d0d053db3   7 months ago    444MB
pelias/api                 <none>    88c7684115f5   9 months ago    344MB
pelias/api                 <none>    0ab14347db69   10 months ago   343MB
pelias/polylines           <none>    fb17a11a5014   10 months ago   376MB
pelias/csv-importer        <none>    be9f57d8db01   10 months ago   367MB
pelias/openaddresses       <none>    7ac705b43c0a   10 months ago   604MB
pelias/openstreetmap       <none>    a90ee5d4bb8b   10 months ago   400MB
pelias/schema              <none>    380146b3c373   13 months ago   280MB
pelias/pip-service         <none>    43646af02db5   14 months ago   360MB
pelias/transit             <none>    9a9aa188dad6   14 months ago   365MB
pelias/geonames            master    3fd846d539c7   15 months ago   369MB
pelias/openaddresses       <none>    c4219f945484   16 months ago   601MB
pelias/api                 <none>    21346debfa70   16 months ago   340MB
pelias/placeholder         <none>    cb4069478d97   16 months ago   309MB
pelias/openstreetmap       <none>    a9a22a08a9e0   16 months ago   396MB
pelias/fuzzy-tester        master    725511c299c8   17 months ago   386MB
pelias/geonames            <none>    f87199a3d012   17 months ago   369MB
pelias/api                 <none>    2215590f6046   18 months ago   340MB
pelias/placeholder         <none>    fcad458a500f   18 months ago   309MB
pelias/openaddresses       <none>    d8ac713da6ff   18 months ago   601MB
pelias/interpolation       <none>    ee178f896646   18 months ago   2.46GB
pelias/openstreetmap       <none>    d45cdf1b5266   19 months ago   384MB
pelias/polylines           <none>    f36d8475da3c   19 months ago   360MB
pelias/csv-importer        <none>    cc5571ed64c2   19 months ago   351MB
pelias/api                 <none>    a7f23d04298b   20 months ago   340MB
pelias/elasticsearch       7.16.1    d5c80585f37b   20 months ago   660MB
pelias/elasticsearch       7.5.1     acf1e33baa9b   20 months ago   797MB
pelias/placeholder         <none>    fa680b4e2b67   20 months ago   303MB
pelias/openaddresses       <none>    9aa213d53fdc   20 months ago   589MB
pelias/csv-importer        <none>    b493109ce313   20 months ago   351MB
pelias/interpolation       <none>    084a15ea8308   21 months ago   2.46GB
pelias/geonames            <none>    3ee7679efa83   21 months ago   355MB
pelias/openstreetmap       <none>    14bc3ce1c3d6   21 months ago   384MB
pelias/transit             <none>    6578e1425149   21 months ago   350MB
pelias/libpostal-service   <none>    36f3bcdb1dc1   21 months ago   2.28GB
pelias/schema              <none>    771ce6fb70b5   21 months ago   517MB
pelias/whosonfirst         <none>    605e6965ad0d   21 months ago   535MB
pelias/polylines           <none>    1cb39cdb1b3b   21 months ago   599MB
pelias/fuzzy-tester        <none>    aa80f3e8c0fb   22 months ago   617MB
pelias/pip-service         <none>    7c0035aa5ca2   22 months ago   586MB
pelias/api                 <none>    d075ec521e73   2 years ago     582MB
pelias/openstreetmap       <none>    367ad8773f42   2 years ago     645MB
pelias/whosonfirst         <none>    b036047769af   2 years ago     535MB
pelias/polylines           <none>    ead6830d8490   2 years ago     1.02GB
pelias/openaddresses       <none>    88b8b7ff0518   2 years ago     781MB
pelias/csv-importer        <none>    e0f97ea3f2c7   2 years ago     622MB
pelias/fuzzy-tester        <none>    f734149727e5   2 years ago     625MB
pelias/geonames            <none>    fc4725903c47   2 years ago     609MB
pelias/schema              <none>    f199fc555e26   2 years ago     530MB
pelias/api                 <none>    91d054047884   2 years ago     597MB
pelias/interpolation       <none>    d57db995e2a7   2 years ago     3.45GB
pelias/csv-importer        <none>    0d58697384e2   2 years ago     620MB
pelias/whosonfirst         <none>    fa9cd943e390   2 years ago     573MB
pelias/openstreetmap       <none>    9b0cfca403fe   2 years ago     685MB
pelias/transit             <none>    92b62654c66f   2 years ago     625MB
pelias/schema              <none>    3303364c130f   2 years ago     533MB
pelias/openaddresses       <none>    5f9201183e3e   2 years ago     815MB
pelias/polylines           <none>    ad9d5b27e07e   2 years ago     1.03GB
pelias/placeholder         <none>    3d1500c9237d   2 years ago     554MB
pelias/fuzzy-tester        <none>    9efa10dcef81   2 years ago     615MB
pelias/pip-service         <none>    e79a1d5f5c21   3 years ago     598MB
pelias/geonames            <none>    476e62c619e1   3 years ago     607MB
pelias/libpostal-service   <none>    4c4af01d0797   3 years ago     3.19GB
pelias/elasticsearch       <none>    6f90d70df8ee   3 years ago     1.32GB

missinglink commented 10 months ago

Could you please roll back and check that the previous version works? (ie. instead of running 25c1cf7efee6 can you please try 88c7684115f5)

pelias/api                 master    25c1cf7efee6   2 weeks ago
pelias/api                 <none>    88c7684115f5   9 months ago
pelias/api                 <none>    0ab14347db69   10 months ago
pelias/api                 <none>    21346debfa70   16 months ago
pelias/api                 <none>    2215590f6046   18 months ago
pelias/api                 <none>    a7f23d04298b   20 months ago
pelias/api                 <none>    d075ec521e73   2 years ago
pelias/api                 <none>    91d054047884   2 years ago

I think you can 'pin' the version with the following syntax (not tested):

@@ -16,7 +16,7 @@ services:
     volumes:
       - "./pelias.json:/code/pelias.json"
   api:
-    image: pelias/api:master
+    image: pelias/api:88c7684115f5
     container_name: pelias_api
     user: "${DOCKER_USER}"
     restart: always

darmentrout commented 10 months ago

I tried replacing "master" with the image ID but I get errors:

$ docker pull pelias/api:88c7684115f5
Error response from daemon: manifest for pelias/api:88c7684115f5 not found: manifest unknown: manifest unknown

ERROR: for api  manifest for pelias/api:88c7684115f5 not found: manifest unknown: manifest unknown
ERROR: manifest for pelias/api:88c7684115f5 not found: manifest unknown: manifest unknown

I tried a different image and got similar results:

$ docker pull pelias/api:0a2350339fbd1aee8ede99c95dd8e3bf4f1b2ff6125a4467b85683f0b1037689
Error response from daemon: manifest for pelias/api:0a2350339fbd1aee8ede99c95dd8e3bf4f1b2ff6125a4467b85683f0b1037689 not found: manifest unknown: manifest unknown

ERROR: for api  manifest for pelias/api:0a2350339fbd1aee8ede99c95dd8e3bf4f1b2ff6125a4467b85683f0b1037689 not found: manifest unknown: manifest unknown
ERROR: manifest for pelias/api:0a2350339fbd1aee8ede99c95dd8e3bf4f1b2ff6125a4467b85683f0b1037689 not found: manifest unknown: manifest unknown

https://hub.docker.com/r/pelias/api/tags

https://docs.docker.com/engine/reference/commandline/pull/#pull-an-image-by-digest-immutable-identifier

darmentrout commented 10 months ago

I just tried pelias/api:v5.55.0 and the API container is up and I can reach the API's default web page in the browser.

58c34cea8b4d   pelias/api:v5.55.0            "./bin/start"            55 seconds ago   Up 49 seconds   0.0.0.0:4000->4000/tcp, :::4000->4000/tcp                                              pelias_api

missinglink commented 10 months ago

Not much has changed except the Ubuntu version.

https://github.com/pelias/api/compare/v5.55.0...v5.57.0

fleet-louisp commented 10 months ago

I was having the same issue v5.55.0 works fine but no luck with v5.57.0.

When I do a pelias compose up I get: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./bin/start": stat ./bin/start: permission denied: unknown

orangejulius commented 10 months ago

I just tried to reproduce this and wasn't able to, using both the v5.57.0 and master image, the API starts and responds to queries just fine.

Can someone who is currently experiencing this issue please share:

their Docker version (`docker --version)
Which pelias project they are using (planet? portland-metro?)
Operating system and CPU architecture (Mac? Linux (include the distro), Intel 64 bit, Apple M1, other ARM, etc)

LiFiCode commented 10 months ago

I have the exact same issue. Running completly fine until it suddenly stopped. Also on ubuntu.

The logs just fill up with /bin/bash: ./bin/start: Permission denied otherwise no error.

Docker version 24.0.5, build ced0996
Project "germany"
Linux openroute 5.15.0-79-generic #86-Ubuntu SMP Mon Jul 10 16:07:21 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
- Distributor ID: Ubuntu
- Description: Ubuntu 22.04.3 LTS
- Release: 22.04
- Codename: jammy

fleet-louisp commented 10 months ago

I was using Australia.

On Tue, 29 Aug 2023, 7:11 am LiFiCode, @.***> wrote:

I have the exact same issue. Running completly fine until it suddenly stopped. Also on ubuntu.

The logs just fill up with /bin/bash: ./bin/start: Permission denied otherwise no error.

Docker version 24.0.5, build ced0996

Project "germany"

Linux openroute 5.15.0-79-generic #86-Ubuntu SMP Mon Jul 10 16:07:21 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Distributor ID: Ubuntu

Description: Ubuntu 22.04.3 LTS

Release: 22.04

Codename: jammy

— Reply to this email directly, view it on GitHub https://github.com/pelias/docker/issues/318#issuecomment-1696417539, or unsubscribe https://github.com/notifications/unsubscribe-auth/BA77RQST4DN2QZTF376X62TXXUCQ7ANCNFSM6AAAAAA3I2UBCY . You are receiving this because you commented.Message ID: @.***>

vctqs1 commented 7 months ago

I'm using Singapore project and also encountered error /bin/bash: ./bin/start: Permission denied. Have everyone had a solution to resolve it yet?

vctqs1 commented 7 months ago

I tried this https://github.com/pelias/docker/issues/318#issuecomment-1671416078 and it works now, thanks @darmentrout you saved me

darmentrout commented 7 months ago

I tried again with the most current version of the API. Here's what I observed.

MODIFIED "image: pelias/api" in docker-compose.yml from "v5.55.0" to "master"

ATTEMPT 1: NO USER VALUE SET (DOCKER_USER NOT IN .env)

docker-compose.yml api: image: pelias/api:master container_name: pelias_api user: "${DOCKER_USER}" restart: always environment: [ "PORT=4000" ] ports: [ "4000:4000" ] volumes:

"./pelias.json:/code/pelias.json"

$ pelias compose logs pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied

ATTEMPT 2: USER VALUE SET TO "1000"

docker-compose.yml api: image: pelias/api:master container_name: pelias_api user: "1000" # "${DOCKER_USER}" restart: always environment: [ "PORT=4000" ] ports: [ "4000:4000" ] volumes:

"./pelias.json:/code/pelias.json"

$ pelias compose logs pelias_api | info: [pip] using pip service at http://pip:4200/ pelias_api | info: [placeholder] using placeholder service at http://placeholder:4100/ pelias_api | info: [language] using language service at http://placeholder:4100/ pelias_api | info: [interpolation] using interpolation service at http://interpolation:4300/ pelias_api | info: [libpostal] using libpostal service at http://libpostal:4400/ pelias_api | info: [libpostal] using libpostal service at http://libpostal:4400/ pelias_api | info: [api:type_mapping_discovery] total hits 10000 pelias_api | info: [api:type_mapping_discovery] total sources 4 pelias_api | info: [api:type_mapping_discovery] successfully discovered type mapping from elasticsearch pelias_api | info: [api] pelias is now running on http://:::4000 pelias_api | info: [api] ::ffff:172.30.0.1 - - [20/Nov/2023:18:51:17 +0000] "GET /v1 HTTP/1.1" 200 389 pelias_api | info: [api] ::ffff:172.30.0.1 - - [20/Nov/2023:18:51:25 +0000] "GET /v1 HTTP/1.1" 304 -

missinglink commented 7 months ago

What is the output of this command on your system?

id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}

darmentrout commented 7 months ago

$ id -u ${SUDO_USER-${USER}}:$(id -g ${SUDO_USER-${USER}}) (I fixed the parentheses) id: ‘username:1003’: no such user

missinglink commented 7 months ago

uuugh... ok, then this ^ is definitely an issue, I assumed that having USER in your env was as old as the telefax. Surely this isn't the same error for people running Ubuntu, I've been using Ubuntu since 2006 and USER was always set.

@darmentrout could you please let me know what OS and shell you're running. Could you also please run env and tell me which variable name is used for your user, maybe USERNAME or LOGNAME?

this is the relevant line if you'd like to experiment with the syntax: https://github.com/pelias/docker/blob/master/lib/env.sh#L9

missinglink commented 7 months ago

Agh no, scratch that @darmentrout I think there was a copy paste error, your syntax also fails for me, try this:

CURRENT_USER=$(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}})

echo $CURRENT_USER
501:20

darmentrout commented 7 months ago

$ CURRENT_USER=$(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) $ echo $CURRENT_USER 1001:1001

Thanks for your ongoing help!

missinglink commented 7 months ago

So in your case the ID:GROUP of 1001:1001 should be passed to docker by setting the DOCKER_USER env var.

This variable is referenced many times in the docker-compose.yml file.

We do this because otherwise Docker can run as root or another user such as docker and cause all sorts of permissions errors which are difficult for new linux users to diagnose and fix.

It also detects users running as sudo, which is a bad habit we want people to avoid, in this case we allow it but then 'de-escalate' permissions back down to the actual user, since no root privileges are required.

What should be happening is that Docker runs as you and creates all the directories as you, downloads all the files as you and runs all the processes as you.

I'm still not clear what the issue is here because Permission denied seems to indicate that an executable file is being run by a user who doesn't have privileges to do so.

missinglink commented 7 months ago

@darmentrout in your comment from two days ago https://github.com/pelias/docker/issues/318#issuecomment-1819642739 you said that manually setting your user to 1000 fixes the issue, but today you said your user is 1001, what's up with that?

[edit] there's also this other comment ‘username:1003’

darmentrout commented 7 months ago

The 1000 is set in the YML file. 1001 comes when I run the command at the prompt.

On Wed, Nov 22, 2023, 12:19 PM Peter Johnson @.***> wrote:

@darmentrout https://github.com/darmentrout in your comment from two days ago #318 (comment) https://github.com/pelias/docker/issues/318#issuecomment-1819642739 you said that manually setting your user to 1000 fixes the issue, but today you said your user is 1001, what's up with that?

— Reply to this email directly, view it on GitHub https://github.com/pelias/docker/issues/318#issuecomment-1823184505, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF7ZSWQ2YPAMHTYWDVO2UWTYFYX3XAVCNFSM6AAAAAA3I2UBC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRTGE4DINJQGU . You are receiving this because you were mentioned.Message ID: @.***>

darmentrout commented 7 months ago

I think that variable was deprecated.

https://github.com/pelias/docker/commit/9dffa863219a6ff5fc216837ad58d005940652d1

On Wed, Nov 22, 2023, 12:19 PM Peter Johnson @.***> wrote:

@darmentrout https://github.com/darmentrout in your comment from two days ago #318 (comment) https://github.com/pelias/docker/issues/318#issuecomment-1819642739 you said that manually setting your user to 1000 fixes the issue, but today you said your user is 1001, what's up with that?

— Reply to this email directly, view it on GitHub https://github.com/pelias/docker/issues/318#issuecomment-1823184505, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF7ZSWQ2YPAMHTYWDVO2UWTYFYX3XAVCNFSM6AAAAAA3I2UBC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRTGE4DINJQGU . You are receiving this because you were mentioned.Message ID: @.***>

missinglink commented 7 months ago

I'm confused how executing bin/start can fail with Permission denied since the executable bit is set for all users:

missinglink commented 7 months ago

I think that variable was deprecated.

We just deprecated setting it manually. Previously we would allow users to set it themselves, but that also caused issues in some cases, so instead we deprecated the ability to set it yourself in favour of automatically detecting it using the CURRENT_USER method we discussed above.

missinglink commented 7 months ago

The PR has more detail https://github.com/pelias/docker/pull/220

ikaroz commented 7 months ago

I was having the same issue on my custom server when trying to update the database, so I have tried a fresh install following the documentation tutorial on a clean Debian 12 image, but got the same permission error, even for portland-metro.

Debian GNU/Linux 12 (bookworm) Kernel: Linux 6.1.0-13-cloud-amd64

docker version 20.10.24+dfsg1, build 297e128 docker-compose version 1.29.2

Projects tested: Brazil (custom), Portland-Metro

echo $CURRENT_USER 1001:1001

darmentrout commented 7 months ago

Here are some more results.

$ pelias system env USER=LOCAL USERNAME DOCKER_USER=1003:1003 LOGNAME=LOCAL USERNAME DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1003/bus XDG_RUNTIME_DIR=/run/user/1003

1: SUDO User 1000

not logged in

docker-compose.yml:

"./pelias.json:/code/pelias.json" api: image: pelias/api:master container_name: pelias_api user: "1000" # "${DOCKER_USER}" restart: always environment: [ "PORT=4000" ] ports: [ "4000:4000" ] volumes:
"./pelias.json:/code/pelias.json"

$ pelias compose logs api Attaching to pelias_api pelias_api | info: [pip] using pip service at http://pip:4200/ pelias_api | info: [placeholder] using placeholder service at http://placeholder:4100/ pelias_api | info: [language] using language service at http://placeholder:4100/ pelias_api | info: [interpolation] using interpolation service at http://interpolation:4300/ pelias_api | info: [libpostal] using libpostal service at http://libpostal:4400/ pelias_api | info: [libpostal] using libpostal service at http://libpostal:4400/ pelias_api | info: [api:type_mapping_discovery] total hits 10000 pelias_api | info: [api:type_mapping_discovery] total sources 4 pelias_api | info: [api:type_mapping_discovery] successfully discovered type mapping from elasticsearch pelias_api | info: [api] pelias is now running on http://:::4000

$ echo $(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) 1003:1003

$ docker exec 8c8c781ba6a9 echo $(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) 1003:1003

$ docker exec 8c8c781ba6a9 ls -la /code total 4 drwxr-xr-x 1 pelias pelias 39 Jul 23 14:14 . drwxr-xr-x 1 root root 6 Nov 27 18:12 .. drwxr-xr-x 1 pelias pelias 6 Jul 23 14:13 pelias -rwxr-xr-x. 1 1003 1003 2595 Oct 13 2022 pelias.json

$ docker exec 8c8c781ba6a9 ls -la /code/pelias total 0 drwxr-xr-x 1 pelias pelias 6 Jul 23 14:13 . drwxr-xr-x 1 pelias pelias 39 Jul 23 14:14 ..

2: Non-SUDO User 1003

logged in

api: image: pelias/api:master container_name: pelias_api user: "1003" # "${DOCKER_USER}" restart: always environment: [ "PORT=4000" ] ports: [ "4000:4000" ] volumes:

"./pelias.json:/code/pelias.json"

$ pelias compose logs api Attaching to pelias_api pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied

$ echo $(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) 1003:1003

$ docker exec 6af221cd25a5 echo $(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) Error response from daemon: Container 6af221cd25a515e7df610e2793e79b80256a707f5d183b4d6073d694bbe47d32 is restarting, wait until the container is running

3: ${DOCKER_USER}

default

api: image: pelias/api:master container_name: pelias_api user: "${DOCKER_USER}" restart: always environment: [ "PORT=4000" ] ports: [ "4000:4000" ] volumes:

"./pelias.json:/code/pelias.json"

$ pelias compose logs api Attaching to pelias_api pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied pelias_api | /bin/bash: ./bin/start: Permission denied

$ echo $(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) 1003:1003

$ docker exec 24c3b429970e echo $(id -u ${SUDO_USER-${USER}}):$(id -g ${SUDO_USER-${USER}}) Error response from daemon: Container 24c3b429970ec9fe9dd578090694dc2e4db62a7c4cbd0d8379e5d325e5271393 is restarting, wait until the container is running

missinglink commented 7 months ago

Thanks for the feedback, I spent several hours digging into it today and have opened a PR to fix it: https://github.com/pelias/api/pull/1663

I'm still not clear which commit introduced it 🤷

darmentrout commented 7 months ago

@missinglink Thanks for staying on top of this! Your software helps thousands of people in our community every day. I really appreciate all of your work.

missinglink commented 7 months ago

A new docker image has been published under the tag latest (also known as master-2023-11-28-9a0e94f70f10f2557d75ab187b939cc72df640ec)

Please pull the latest image and try again.

note: this issue was closed automatically, please comment if it isn't resolved and I'll reopen it.

missinglink commented 7 months ago

Thanks for all your reports and patience, it was a difficult one to track down because it only affected users with a UID != 1000, which is the first UID assigned to 'normal users' on most common Linux systems.

pelias / docker

pelias_api stopped running, exit code 126 #318

1: SUDO User 1000

2: Non-SUDO User 1003

3: ${DOCKER_USER}