Applying entire role when just updating config is slow

jbothma commented 4 years ago

Description of problem

Running through the entire role every time I want to update config on an app is very slow - about 6 minutes.

Is there a good pattern for skipping past the role stuff and just running my app stuff?

I know the idea is for config to be ever converging and therefore not run small snippets, but there is precedent for configuring certain things using one playbook, and other things using another - e.g. an admin users playbook, an app playbook, etc.

The the role's default tasks take very long to run, and don't change often, while config should be quick to update so that people don't get tempted to change them manually and diverge from the ansible config.

I tend to use --start-at-task "Dokku app exists" - I haven't dug into the tags ansible-dokku defines to be able to skip them. Setting the tag on the play also sets it for the role's tasks. setting a tag like "app" on each task would be very verbose and annoying and easily forgotten.

How do you do it, or what do you recommend?

How reproducible

Very

Steps to Reproduce

Create a playbook for a simple app
Run the playbook forthe first time, everything installs.
Chang 1 config setting
Run the playbook
Wait 6 minutes for it to complete, or to see something went wrong, fix and wait another 6 minutes.

Actual Results

All tasks including installing docker check if they need to execute, taking a long time

Expected Results

There is some way to just run the app-specific stuff like creating, configuring

ltalirz commented 4 years ago

Hi @jbothma, I certainly agree that running all parts of a long role is often impractical.

Setting the tag on the play also sets it for the role's tasks.

The same is true for import_tasks, see e.g. https://github.com/dokku/ansible-dokku/blob/master/tasks/main.yml

I don't know where in the role all the app-related tasks are, but if they are somewhat located in a couple of files (or can be distributed that way), adding a new tag is not verbose and would seem like the right solution. Happy to review a PR on this.

josegonzalez commented 4 years ago

I use import_tasks like so:

---
- import_tasks: app-cdn.yml

And then something like this:

---
- name: dokku apps:create cdn
  dokku_app:
    app: cdn
  tags:
    - app-cdn

- name: dokku config:set cdn
  dokku_config:
    app: cdn
    config: "{{ env_cdn }}"
  tags:
    - app-cdn

- name: dokku storage:mount cdn /app/storage
  dokku_storage:
    app: cdn
    create_host_dir: true
    mounts:
      - /var/lib/dokku/data/storage/cdn:/app/storage
  tags:
    - app-cdn

- name: dokku certs:add cdn
  dokku_certs:
    app: cdn
    key: /etc/nginx/ssl/cakephp.key
    cert: /etc/nginx/ssl/cakephp.crt
  tags:
    - app-cdn

- name: dokku clone cdn
  dokku_clone:
    app: cdn
    repository: https://github.com/cakephp/cdn.cakephp.org
  tags:
    - app-cdn

So you then I can apply just the changes for a given app by targeting the app-cdn tag.

Additionally, you could have CI go through and ensure each thing in a play has the correct tag based on the filename (so if the task file is named (app-\w+).yml, the regex capture must be a tag of each task in the file) to avoid forgetting tags.

Might be good to add ansible-dokku as a tag to each task in our task files, though I don't know if you can say "skip anything with this tag". In either case, I think adding tags to things you want to target is the best way, and if you know you want to run specific tasks, then it would be good to ensure you know how they are structured/what tags are available before doing so :)

jbothma commented 4 years ago

Thanks. I went with adding tags to my own tasks and it works nicely.

I like the idea of CI checking it somehow. Will think about that.

I agree, for a case like this it's probably good to know precisely what's tagged and will run, rather than using tags inside ansible-dokku that might include something we don't want run at a particular time. Thanks!

dokku / ansible-dokku