ahmet2mir commented 9 years ago

Hi,

It will be very helpful to have something like "onrun" in the YAML to be able to run commands after the run. Similar to https://github.com/docker/docker/issues/8860

mongodb:
    image: mongo:3.0.2
    hostname: myhostname
    domainname: domain.lan
    volumes:
        - /data/mongodb:/data
    ports:
        - "27017:27017" 
    onrun:
        - mongodump --host db2dump.domain.lan --port 27017 --out /data/mongodb/dumps/latest
        - mongorestore -d database /data/mongodb/dumps/latest/database

After the mongodb start, It will dump db2dump.domain.lan and restore it.

When I will stop and then start the container, onrun part will no be executed to preserve idempotency.

EDIT 15 June 2020

5 years later, Compose wan't to "standardize" specifications, please check https://github.com/compose-spec/compose-spec/issues/84

dopry commented 5 years ago

@victor-perov create another container for the roll-up task and execute it as separate service

Here are some snippets from one of our projects to show a task service to run a database migration.

x-task: &task
  # run once deploy policy for tasks
  deploy:
    restart_policy: 
      condition: none
    replicas: 1

service:
  automata-auth-migrate:
    <<: *automata-auth
    <<: *task
    # without the sleep it can't lookup the host postgres.  maybe the command is ran before the network set is complete.
    command: sleep 5 && python /code/manage.py migrate --noinput

annjawn commented 5 years ago

Well, this is the fourth year this discussion has been stretched to. So let me add my +1 to this use case of a need for onrun. P.S.: I should've bought popcorn for the whole thread.

rcarmo commented 5 years ago

I, too, would think onrun or equivalent (post-run?) is a must. Adding a wrapper script and doing docker exec into the container is just... ugly.

dkrieger commented 5 years ago

IMO docker compose was a great container orchestration MVP to convince people that managing containers can be easy. Maybe we, the community, should consider it to be in "maintenance mode" as production-ready orchestration solutions (i.e. kubernetes) have proliferated. When you have advanced features like container dependencies, combined with absent features such as "exec this thing after the container is up", it seems to fit the narrative that the pace of development has simply plateaued. At the very least, it is not obvious that this feature should be considered out of scope.

rahafrouz commented 5 years ago

You cannot do everything easily with Dockerfile. Let's say you want to add your own script to a container. For example take the mysql container and try to add a simple script to call an API in case of some event. You can do it either by:

Changing the Dockerfile of mysql and add your own script to the container before the entrypoint. You cannot add a CMD in the Dockerfile, since it would be an argument to the ENTRYPOINT.
You can run the container and then copy your script to the running container and run it [docker cp, docker exec].

So that's why I also think a feature like onrun is beneficial since changing the Dockerfile is not always enough.

Somebi commented 5 years ago

Dump, why this is closed? Consider situation, when you are using official docker image, like Cassandra and you need to load schema after it's started... Have to implement your own bash script solution for this... ugh, this is ugly

dantebarba commented 5 years ago

@somebi looks like compose is closed...

aManNamedJed commented 5 years ago

Just my two cents: I landed here because I am currently having to enable Apache modules manually every time I start the container (SSL isn't enabled by default in the Docker Hub wordpress image). Not the end of the world but was hoping to run a couple of commands whenever it goes up so I can just seamlessly take the containers up and down without having to bash in.

ghost commented 5 years ago

Just my two cents: I landed here because I am currently having to enable Apache modules manually every time I start the container (SSL isn't enabled by default in the Docker Hub wordpress image). Not the end of the world but was hoping to run a couple of commands whenever it goes up so I can just seamlessly take the containers up and down without having to bash in.

Well this could be easily resolved if you build a new image based on the wordpress image, that has the modules you need enabled. Then use that instead for e.g. a dockerfile:

FROM wordpress:php7.1 RUN a2enmod ssl

Another solution would be to download the wordpress Dockerfile and add the module activation in it. Then produce a new image for yourself using docker build. For e.g. this is the Dockerfile for wordpress 5.2 with php 7.1:

wordpress dockerfile

you may enable more modules in line 63 or run ssl genaration.

All this is not the case that I think we are discussing here. The problem is creating dynamic hooks in the container lifecycle like when it starts ends etc.

fairmonk commented 5 years ago

This would be a nice addition to docker-compose !

Ordiel commented 4 years ago

Answers like the ones on this thread are the reason Kubernetes is keeping "all" the money Docker (technology) is producing, and it's not a bad thing hopefully someone will buy Docker (company) soon and change the way community proposals/request are welcome/analysed...

webpolis commented 4 years ago

Answers like the ones on this thread are the reason Kubernetes is keeping "all" the mony Docker (technology) is producing, and it's not a bad thing hopefully someone will buy Docker (company) soon and change the way community proposals/request are welcome/analysed...

I wrote a similar critic, without any offensive statement (it was along the lines of open source projects which are not entirely open source whose maintainers defiantly ignore arguments without any other reason than showing how much technical argon they possess) , it got plenty of support, and the message was removed.

That shows what kind of arrogant persons are behind this.

luisRubiera commented 4 years ago

When your community demands something for 4 years and you (Docker) close your eyes it shows that you're not looking in the same direction as them :/

phedders commented 4 years ago

And now docker gave up and sold out. Because they could not listen... they lost.

Shame - but hey ho.

xtrasimplicity commented 4 years ago

It's a real shame that something like this doesn't exist. I would've loved to have been able to create onFailure hooks, which could take place when the health checks fail.

i.e.

services:
  app:
    image: myapp:latest
    hooks:
      onFailure:
        - # Call a monitoring service (from the host machine) to tell it that the service is offline.

This would be useful for times where the application doesn't bind to a socket/port. Kubernetes is probably the way to go, here, but this is a fairly large infrastructure change and overkill for a very small environment.

Edit: To get around this, I ended up updating the entrypoint of my container to "wrap" the monitoring functionality. i.e.

# /app/bin/run_with_monitor
#!/bin/bash
set -eE

updateMonitoringSystem() {
 # do something here... This is run from the container, though, unfortunately.
 if [[ $? -eq 1 ]]; then
  # Failed!
 else
  # All is good!
 fi
}

trap 'updateMonitoringSystem' EXIT

$@

# Dockerfile
....
CMD ["/app/bin/run_with_monitor", "./my-app"

Still, it'd be nice to do this without having to modify the image.

LukeStonehm commented 4 years ago

:man_shrugging: Came looking for this basic functionality, that the competitor (Kubernetes) has, and instead I found a dumpster fire.

It's a real shame, now I have to maintain separate docker images for testing locally.

Happy new year :roll_eyes:

Strandedpirate commented 4 years ago

jsternadel commented 4 years ago

@LukeStonehm same here. Needed to do ONE command after the container was stood up but instead was treated with hot garbage. I really don't feel like managing my own images and docker files when an official image gets me 90% or more of the way there.

T-vK commented 4 years ago

A significant amount of programs rely on certain services to exist on startup. For example a MySQL or MongoDB database.

Therefore there is no sane way to use docker-compose in these cases.

Instead users are expected to:

Learn how to write Dockerfiles (and programming)
Learn how to build Docker images
Create Dockerfiles inheriting from the original images, adding code to make sure the containers wait for each other
Regularly check for security updates of the base images
Regularly modify the Dockerfiles to apply the updates
Regularly build Docker images from those Dockerfiles

And this sucks because:

You waste massive amounts of time learning stuff you may not even need otherwise
You regularly waste hardware resources on building and storing Docker images yourself or even on uploading/downloading (pulling/pushing) them
You regularly waste time on writing those Dockerfiles, building them, testing them, fixing them etc...
You potentially compromise the security of your images because you don't know what you are doing
You lose the ability to run only officially verified/signed Docker images

If we had a startup check, all of this wouldn't be necessary and we could simply change image: mysql:8.0.18 to image: mysql:8.0.19 whenever we want and be done!

Realistically this is what's currently happening in the real world:

People create their own Dockerfiles making changes so that they work with docker-compose
They build their images once
And don't patch them regularly
Hackers get happy

And you can't say that docker-compose is only supposed "to do one thing" because it already does pretty much everything. Including pulling and building images even more importantly, specifying dependencies using the depends_on property. This is not even about implementing a completely new feature this is just about passing another parameter through to docker.

@binman-docker @crosbymichael @dmcgowan @ebriney @ehazlett @eunomie @guillaumerose @jeanlaurent @justincormack @lorenrh @manishtomar @olegburov @routelastresort @spencerhcheng @StefanScherer @thaJeztah @tonistiigi @ulyssessouza @aiordache @chris-crone @ndeloof Please reconsider this feature or let's at least have a proper discussion about this.

dopry commented 4 years ago

The task service technique works pretty well for me at this juncture, but does have it's idiosyncracies. We've applied the pattern in our compose files for migrations and application initialization extensively. but I do a agree the a better 'depends_on' that waited on a successful healthcheck or successful exit/task completion would make many tasks easier and more reliable.

blizzz commented 4 years ago

This would really be a helpful addition.

jamespfennell commented 4 years ago

I think it's worth emphasizing that Kubernetes has this functionality through lifecycle postStart.

Strandedpirate commented 4 years ago

k8s != docker-compose. Wrong channel

jamespfennell commented 4 years ago

Sorry for not being clear, but my point was: Kubernetes supports this, and because Kubernetes and Docker compose have many of the same use cases/purposes, that would be an argument for having it in compose. Sorry if I was unclear.

alex-ubitec commented 4 years ago

Good news!!

I think docker has heard us, (on this issue and a few others). https://www.docker.com/blog/announcing-the-compose-specification/

Let's try to work on the specification there to fulfill the community needs. We can try to make this an open and friendly community with this restart.

taythebot commented 4 years ago

Good news!!

I think docker has heard us, (on this issue and a few others). https://www.docker.com/blog/announcing-the-compose-specification/

Let's try to work on the specification there to fulfill the community needs. We can try to make this an open and friendly community with this restart.

Has anyone suggested this change yet? Mailing list isn't available yet so I think the next best place is here: https://github.com/compose-spec/compose-spec

I don't see an issue that describes this problem but not sure if that's the right place...

Edit: I opened an issue at https://github.com/compose-spec/compose-spec/issues/84. Please upvote it to show your support for the feature!

reduardo7 commented 4 years ago

You can use the HEALTHCHECK to do something else like following example:

Code

`Dockerfile`

FROM ubuntu

COPY healthcheck.sh /healthcheck.sh
RUN chmod a+x /healthcheck.sh

HEALTHCHECK --interval=5s CMD /healthcheck.sh

CMD bash -c 'set -x; set +e; while true; do cat /test.txt; sleep 3; done'

`healthcheck.sh`

#/usr/bin/env bash

set -e

FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG'

# Health check

echo 'Run command to validate the container status HERE'

# On success
if [ ! -f "${FIRST_READY_STATUS_FLAG}" ]; then
  # On first success...
  touch "${FIRST_READY_STATUS_FLAG}"

  # Run ON_RUN on first health check ok
  if [ ! -z "${DOCKER_ON_RUN}" ]; then
    eval "${DOCKER_ON_RUN}"
  fi
fi

Run the health check.
- If it fails, exits from script with exit code 1.
- If the health check is ok, the script will continue.
If it is the first health check OK and if DOCKER_ON_RUN environment variable exists, execute it.

Example

`docker-compose.yml`

version: "3.7"

services:
  test:
    build:
      context: .
    image: test/on-run
    environment:
      DOCKER_ON_RUN: echo x >> /test.txt

You can use DOCKER_ON_RUN environment variable to pass a custom command to execute after run.

Execution result

docker-compose build
docker-compose up

Output:

Creating network "tmp_default" with the default driver
Creating tmp_test_1 ... done
Attaching to tmp_test_1
test_1  | + set +e
test_1  | + true
test_1  | + cat /test.txt
test_1  | cat: /test.txt: No such file or directory
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | cat: /test.txt: No such file or directory
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | x
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | x
test_1  | + sleep 3
test_1  | + true
test_1  | + cat /test.txt
test_1  | x
test_1  | + sleep 3

You can see the error cat: /test.txt: No such file or directory until the health check is ready.
You can see only one x inside /test.txt after run.

Hope this can help someone.

Edit 1

If you don't need a health check, you can use the rest of the script.

Edit 2

See https://github.com/reduardo7/docker-on-ready for examples

augustgerro commented 4 years ago

@reduardo7 Thanks for your workaround. Just want to add, in case if your needs to run command one, like for users creation or etc, your can mount volume for the touch "${FIRST_READY_STATUS_FLAG}"

web-ted commented 4 years ago

Many of these solutions are valid workarounds to this problem. For e.g. making an entrypoint script could also resolve this: ENTRYPOINT ["./entrypoint.sh"]

which will include a more complex logic before running the actual service or process. This is still not a hook though that would allow us to inject logic in the container lifecycle:

before creating
before starting
after starting
before destroying
even after destroying
etc ...

I know that not all the above are meaningful but I hope that you get the picture because this is the point. This could also be included in docker-compose with a directive like:

lifecycle:
    before_start: "./beforeStartHook.sh"
    after_destroy: "./afterDestroyHook.sh"

or even like that:

hooks:
    before_destroy: "./beforeDestroyHook.sh"
    before_create: "./fixFsRights.sh"

devendraap commented 4 years ago

I am unable to overwrite file which requires root permission using hook script or bootstrap script approach, since we start container as non root user

zerthimon commented 4 years ago

Wow, such a basic functionality and still not implemented.

grandmaestr commented 3 years ago

This is workaround from @reduardo7 worked for me.

You can use the HEALTHCHECK to do something else like following example:

Code

Dockerfile
FROM ubuntu

COPY healthcheck.sh /healthcheck.sh
RUN chmod a+x /healthcheck.sh

HEALTHCHECK --interval=5s CMD /healthcheck.sh

CMD bash -c 'set -x; set +e; while true; do cat /test.txt; sleep 3; done'
I have the following Dockerfile:
FROM debian:buster

RUN apt-get update && apt-get --no-install-recommends -y install \ ca-certificates \ cpanminus \ default-libmysqlclient-dev \ fonts-dejavu \ gcc \ gettext \ git \ libc6-dev \ libexpat1-dev \ libfribidi-dev \ libgd-dev \ libxslt1-dev \ libyaz-dev \ make \ perl \ pkg-config \ && rm -rf /var/lib/apt/lists/*

RUN adduser --disabled-password --gecos '' koha

USER koha

WORKDIR /home/koha

ADD --chown=koha:koha https://raw.githubusercontent.com/Koha-Community/Koha/v20.11.00/cpanfile .

ENV PERL_CPANM_OPT --local-lib-contained /home/koha/.local RUN export PERL_CPANM_OPT="--quiet --metacpan --notest $PERL_CPANM_OPT" \ && cpanm --installdeps . \ && cpanm Readonly Array::Utils JSON::Validator@4.05 IO::Scalar \ && cpanm Starman \ && rm -rf /home/koha/.cpanm

RUN git clone --progress --depth 1 --branch v20.11.00 https://github.com/Koha-Community/Koha.git koha

COPY --chown=koha:koha koha-conf.xml.in etc/ COPY --chown=koha:koha log4perl.conf etc/

ENV KOHA_CONF /home/koha/etc/koha-conf.xml ENV PERL5LIB /home/koha/koha:/home/koha/.local/lib/perl5 ENV PATH /home/koha/.local/bin:$PATH ENV LANG C.UTF-8

EXPOSE 5000 5001

COPY docker-entrypoint.sh /usr/local/bin/

WORKDIR /home/koha/koha ENTRYPOINT ["docker-entrypoint.sh"] CMD ["starman", "--listen", ":5000", "--listen", ":5001"]

I need to run the following command (to reindex ElasticSearch) after the container starts:

`docker exec koha perl koha/misc/search_tools/rebuild_elasticsearch.pl -d`

but couldn't get it working with ENTRYPOINT or CMD. So wrote a bash script (script.sh)

!/bin/bash

perl koha/misc/search_tools/rebuild_elasticsearch.pl



, made it executable and added this block to the Dockerfile

`COPY script.sh /script.sh
HEALTHCHECK --interval=5s CMD /script.sh`

which works. Question is, since HEALTHCHECK will will run the script every 5 s, is there a way to make  HEALTHCHECK run only once and quit?

reduardo7 commented 3 years ago

@grandmaestr :

Question is, since HEALTHCHECK will will run the script every 5 s, is there a way to make HEALTHCHECK run only once and quit?

The HEALTHCHECK runs every 5s, it's a Docker feature. But, with FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG' (at healthcheck.sh) we prepare the HEALTHCHECK to run only once the first time.

You need to define perl koha/misc/search_tools/rebuild_elasticsearch.pl -d at DOCKER_ON_RUN env variable (at Dockerfile: ENV DOCKER_ON_RUN="perl koha/misc/search_tools/rebuild_elasticsearch.pl -d"), or your healthcheck.sh should be something like following:

#/usr/bin/env bash

set -e

FIRST_READY_STATUS_FLAG='/tmp/.FIRST_READY_STATUS_FLAG'

# Health check

echo 'Run command to validate the container status HERE'

# On success
if [ ! -f "${FIRST_READY_STATUS_FLAG}" ]; then
  # On first success...
  touch "${FIRST_READY_STATUS_FLAG}"

  # Run ON_RUN on first health check ok
  perl koha/misc/search_tools/rebuild_elasticsearch.pl -d
fi

grandmaestr commented 3 years ago

Thank you @reduardo7, I'll try this out tomorrow and let you know how it goes. Much appreciated!

reduardo7 commented 3 years ago

I created a repository with examples about how to implement this required feature:

https://github.com/reduardo7/docker-on-ready

Luc45 commented 3 years ago

Why this has to be so complicated?

fescobar commented 3 years ago

@Luc45 this is the next challenge after landing in Mars

alexrecuenco commented 3 years ago

@Luc45 this is the next challenge after landing in Mars

It will certainly take a similar amount of time ;)

superswan commented 3 years ago

This would be a nice feature. I have a case where I need to run a single command after starting PHP service and don't want to add a bash script wrapper or anything like that.

imod commented 3 years ago

@superswan not sure (and also never tried), but is this working for you: https://github.com/docker/compose/issues/1510#issuecomment-505424908

henryclw commented 2 years ago

It's 2022, and yet we don't have an elegant way to do this.

Ordiel commented 2 years ago

@henryclw have you check on command

henryclw commented 2 years ago

@henryclw have you check on command

This is not we want, as https://github.com/docker/compose/issues/1809#issuecomment-237065574 said, we want something that could only be executed once, just like initializing the database or run apt-get update && apt-get upgrade

Strandedpirate commented 2 years ago

vysogot commented 1 year ago

March 2023 and I just wanted to alter a user in a docker-compose after I up some database... but fine, bash script! Nice chat guys! Although, one thing that makes using a script after running docker-compose yuck is that now I have to refer to a container by name. An arbitrary name from the script point of view (unless I go for some global env vars which I won't).

ndeloof commented 1 year ago

Those interested with this topic could help us providing feedback on https://github.com/compose-spec/compose-spec/pull/289

dopry commented 1 year ago

The solution I've landed at for at least the docker compose up scenarios are task containers that setup my databases etc. Here is an example froma django environment

# re-usable yaml anchors
x-task: &task
  # docker stack deploy restart policy for tasks
  deploy:
    restart_policy:
      condition: on-failure
    replicas: 1
  # docker compose stand-alone restart policy for tasks
  restart: on-failure

x-task-ensure-database: &task-ensure-database
  <<: *task
  # image:  this task must extend a service with a postgres image.
  entrypoint: |
    sh -c "
    psql -v ON_ERROR_STOP=0 --host postgres --username \"$${POSTGRES_USER}\" <<EOSQL
      CREATE ROLE $${SERVICE_NAME} LOGIN PASSWORD '$${SERVICE_PASS}';
      CREATE DATABASE $${SERVICE_NAME};
      GRANT ALL ON DATABASE $${SERVICE_NAME} TO $${SERVICE_NAME};
    EOSQL"
  # you must set the following environment variables where you use this anchor.
  # environment:
    # - POSTGRES_USER
    # - PGPASSWORD - superuser password
    # USERNAME and DATABASE to create.
    # - SERVICE_NAME=UserAndDbName
    # - SERVICE_PASS=password

x-service: &service
  # docker stack deploy restart policy for tasks
  deploy:
    restart_policy:
      condition: unless-stopped
    replicas: 1
  # docker compose stand-alone restart policy for tasks
  restart: unless-stopped

x-postgres: &postgres
    image: postgres:14
    environment:
      - POSTGRES_USER=example
      - POSTGRES_PASSWORD=example

volumes:
  postgres-data:
  cms-media:

services:
 postgres:
    <<: *service
    <<: *postgres
    # we're only exposing here, since this is only called from another service.
    expose:
      - "5432"
    # uncomment below if you wish to access this db directly from the host.
    ports:
      - "5432:5432"
    volumes:
      - postgres-data:/var/lib/postgresql

  api-ensure-database:
    <<: *postgres
    <<: *task-ensure-database
    environment:
      - POSTGRES_USER=example
      - PGPASSWORD=example
      - SERVICE_NAME=api
      - SERVICE_PASS=password
    depends_on:
     - postgres

api-ensure-database will keep trying to run until it succeeds. The 'task' services fail and respawn a lot while waiting on their dependencies which isn't ideal, but it does get the job done.

docker / compose

Execute a command after run #1809

Code

`Dockerfile`

`healthcheck.sh`

Example

`docker-compose.yml`

Execution result

Edit 1

Edit 2

Code

`Dockerfile`

!/bin/bash