Missing gems in `6.0.0` build?

[davidknezic]

Infos

• Docker version: -
• Docker-compose version: -
• Operating system (Docker host): -

Expected behavior

Trying to upgrade to the latest docker build tagged 6.0.0, I expect zammad-init to succeed.

Actual behavior

The command zammad-init fails with error:

zammad@zammad-56b6bdd786-dk4h5:~$ /docker-entrypoint.sh zammad-init
initialising / updating database...
Could not find puma-4.3.12, activerecord-nulldb-adapter-0.8.0, graphql-2.0.17, tzinfo-data-1.2022.7, dalli-3.2.3, terser-1.1.13, autoprefixer-rails-10.4.7.0, doorkeeper-5.6.4, omniauth-rails_csrf_protection-0.1.2, omniauth-github-1.4.0, omniauth-gitlab-2.0.0, omniauth-google-oauth2-0.8.0, omniauth-saml-1.10.3, omniauth-weibo-oauth2-0.5.2, writeexcel-1.0.5, telephone_number-1.4.19, twilio-ruby-5.74.3, net-ldap-0.17.1, zendesk_api-1.38.0.rc1, rack-2.2.6.4, msgpack-1.6.0, nio4r-2.5.8, raindrops-0.20.0, concurrent-ruby-1.2.0, vite_ruby-3.2.14, tilt-2.0.11, version_gem-1.1.1, omniauth-1.9.2, omniauth-oauth2-1.7.3, ruby-saml-1.14.0, addressable-2.8.1, zeitwerk-2.6.7, nokogiri-1.13.10, rack-test-2.0.2, rails-html-sanitizer-1.4.4, erubi-1.11.0, globalid-1.0.1, i18n-1.12.0, minitest-5.17.0, dry-cli-0.7.0, dry-types-1.7.0, mini_portile2-2.8.1, loofah-2.19.1 in locally installed gems
Run `bundle install` to install missing gems.

Steps to reproduce the behavior

Run build tagged 6.0.0 with command zammad-init. Reverting back to zammad-5.4.0-26 worked again.

[stefanstidlffg]

same here

[jvllmr]

+1

[fnetz]

Same problem here. I got it to run by backing up the /opt/zammad (zammad-data) volume and replacing it with an empty one (apparently the right gems are in the docker image).

[stefanstidlffg]

@fnetz your solution didnt work here. empty zammad-data results in: z6-test-zammad-init-1 | /docker-entrypoint.sh: line 55: contrib/packager.io/database.yml.pkgr: No such file or directory z6-test-zammad-init-1 exited with code 1

[stefanstidlffg]

it seems, that if there is already a database -> /opt/zammad doesnt get filled.

if i drop the database also, then /opt/zammad gets filled. and this "default" content can I use with my old database.

i moved /opt/zammad/storage/ from my old install to the newly created dir and now it starts up and migrates the database

[fnetz]

I guess there is some difference in setup/version, for me, it migrated directly with an empty volume.

Anyway, it's a hacky workaround and the maintainers hopefully have a better solution.

[yogo1212]

I can confirm this is still happening with the latest image (4e8422751540)

For instance, Gemfile.lock specifies addressable 2.8.1, the version in the image is /usr/local/bundle/cache/addressable-2.8.4.gem. Other than the gem lock, are there any other files that need to be updated?

[monotek]

Are you trying to use docker exec and running the docker-entrypoint.sh manually? That will likely not work, as all the env vars are not set in your shell.

[yogo1212]

Not at all! I use compose.

For testing, I've been starting zammad-init manually like this:

# alter Gemfile+lock - e.g., this was the Gemfile.lock with the fewest errors:
$ wget https://github.com/zammad/zammad/raw/stable/Gemfile.lock
$ podman run --rm  --env-file /home/zammad/zammad.env --uidmap 1000:0:1 --gidmap 1000:0:1 --uidmap 0:1:1 --gidmap 0:1:1 --user 1000:1000 --network zammad -v "/mnt/zammad/zammad:/opt/zammad" docker.io/zammad/zammad-docker-compose:latest zammad-init

EDIT: remove -it which was for colored output

[yogo1212]

I've just tried using --entrypoint bash and running bundle install - yields the same error.

[monotek]

Not sure what you're doing at all. It's not meant to work this way.

[mgruner]

I was able to reproduce this locally. It happened when I first built the Zammad container from develop, and then rebuilt it from stable locally. Something is wrong with the docker build caching. docker build --no-cache and docker system prune didn't solve this for me -- I had to wipe my local docker storage completely and then it worked.

So right now I would assume this affects local docker deployments where docker build is used to create the image.

[Bence98007]

Anybody found a solution yet? I've tried nuking the opt/zammad, ofc didnt work, tried nuking the opt/zammad/storage and overwrite the Gemfileand Gemfile.lock but no luck, the init container still links to old dependencies. 😢

[spitzlei]

I just succesfully finished the upgrade this morning (had the error above, too). What I did:

1. used the up-to-date docker-compose.yml (switching volumes to zammad-var:/opt/zammad/var is the important part here)
2. update the env file (I think IMAGE_REPO is the important part here, the new image at ghcr.io seems to work, the one from dockerhub, not)

There were 2 Problems during this update: a) I removed :ro from zammad-config-nginx:/etc/nginx/sites-enabled:ro because of an "read only" error during nginx-container startup. b) I added the env vars from here: https://github.com/zammad/zammad-docker-compose/pull/357/commits/ddc85d87b2371101386633bf3375729879bcd202 (double check the names! There is POSTGRESQL_PASS, POSTGRES_PASSWORD, POSTGRESQL_PASSWORD depending on container!)

After this the migration worked and the Version 6 is running.

[yogo1212]

@spitzlei that looks very promising!

For anyone who has moved to file storage, there also needs to be a mount for that: path-to-zammad-storage:/opt/zammad/storage

[yogo1212]

@spitzlei i can confirm that it works :-)

overall, it's a cool change and i'm glad that code and data are now more separated.

[technoicon]

@spitzlei that looks very promising!

For anyone who has moved to file storage, there also needs to be a mount for that: path-to-zammad-storage:/opt/zammad/storage

Where do I need to put this? I assume in the docker-compose.yml file (or docker-compose.override.yml)?

but where in here does it go? Would you be able to provide a live example?

[technoicon]

Hi,

My Init Logs:

I, [2023-07-10T16:55:30.445469#15-6180] INFO -- : Using the Redis back end for Zammad's web socket session store. rake aborted! Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/xxxxxx/xxxxxx/xdx/x/x/xxxxxxxxxxxxxxxxxxxxxxxx /opt/zammad/app/models/store/provider/file.rb:30:in `binread'

[yogo1212]

@technoicon

I'm working with systemd and regular folders so my configuration likely won't help you.

I suggest you try something along the lines:

• create a volume zammad-storage
• start a temporary container podman run -it --rm .. and mount both the new zammad-storage as well as the old zammad-data
• inside the container, copy or move the content of the storage folder from zammad-data directly into the new volume (for me, the new volume only contains a folder called fs)

Naturally, you'll want to have a backup that you can fall back to if it fails.

Finally, add the storage volume through a method of your liking:

    volumes:
      ...
      - zammad-storage:/opt/zammad/storage

That should be it.

I'm also not using the nginx container, so I also had to make sure that nginx serves /opt/zammad/public from the image - I assume that's not something you need to do.

[technoicon]

I ended up moving my data to the database, then upgrading.

Another issue I had was I had setup my elasticsearch container to less than 2g of RAM. Needs at least 3g in the new update.

[kale1d0code]

I'm still struggling to get this to work I've used versions 66, 67, 69, and 71 (71 has it's own issues)

all of them had a problem with my postgres password not being URI safe (so I created a new env var with a version of the password that is uriencoded)

Then the websocket service kept trying to look for the database at zammad-postgresql so I added that as an alias for my dbms service

The services are all still failing however. I get the feeling that these services are supposed to be reading settings in the /opt/zammad/var directory as it has been added as a mount in the new docker-compose.yml in this repo

I got this working on 66 needed to add postgresql environment variables to all services (yet still ignores the POSTGRESQL_HOST variable value)

[snorte]

For a docker newbie. Someone who has a guide to upgrade from 5.4 to 6.0?

[mgruner]

Upgrade should now work better with the latest version. Please see 7f31a9a

[snorte]

Do I need to change anything to get 6 version update?

[MrGeneration]

@snorte please consult our community https://community.zammad.org