"arm/v7" architecture missing in current docker builds?

[simonkuehling]

Please verify that this bug has NOT been raised before.

• [X] I checked and didn't find a similar issue

Describe the bug*

During a docker pull a minute ago I got the following error:

$ docker-compose pull
Pulling inventree-db     ... done
Pulling inventree-server ... pulling from inventree/inventree
Pulling inventree-worker ... pulling from inventree/inventree
Pulling inventree-proxy  ... done

ERROR: for inventree-server  no matching manifest for linux/arm/v7 in the manifest list entries

ERROR: for inventree-worker  no matching manifest for linux/arm/v7 in the manifest list entries
ERROR: no matching manifest for linux/arm/v7 in the manifest list entries

Looking at docker hub I do indeed see no arm/v7 OS/ARCH listed for latest and stable - has this to do with ongoing Dockerfile refactoring?

Steps to Reproduce

$ docker-compose pull

Expected behaviour

find a suitable docker image I guess ;-)

Deployment Method

• [X] Docker
• [ ] Bare metal

Version Information

Hardware is a Raspberry Pi 4 - my last update was 3 months ago:

$ docker image ls inventree/inventree
REPOSITORY            TAG       IMAGE ID       CREATED        SIZE
inventree/inventree   latest    b304aa931b5c   3 months ago   850MB

Please verify if you can reproduce this bug on the demo site.

• [ ] I can reproduce this bug on the demo site.

Relevant log output

[none]

[SchrodingersGat]

@simonkuehling some recent upstream changes for the cryptography python package have made it more difficult to build for raspbian, so we no longer offer that by default. We were not sure how many people (if any) were using the docker builds on raspbian tbh!

We need someone to be able to test if the docker image still works on raspbian. Are you able to check this for us?

1. Pull down latest code
2. cd into code directory
3. Run docker build . --target production

[simonkuehling]

Oh, i see - sure, I will give that a try right away. Here is the output:

$ docker build . --target production
[+] Building 1264.2s (16/16) FINISHED                                                                                                                        docker:default
 => [internal] load build definition from Dockerfile                                                                                                                   0.1s
 => => transferring dockerfile: 4.79kB                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                      0.2s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/python:3.10-alpine3.18                                                                                              2.2s
 => [inventree_base  1/10] FROM docker.io/library/python:3.10-alpine3.18@sha256:a578a0d69e5df6a9d7af73699553dc80717b2871a24cc9536130a40512b41295                      24.5s
 => => resolve docker.io/library/python:3.10-alpine3.18@sha256:a578a0d69e5df6a9d7af73699553dc80717b2871a24cc9536130a40512b41295                                        0.2s
 => => sha256:a578a0d69e5df6a9d7af73699553dc80717b2871a24cc9536130a40512b41295 1.65kB / 1.65kB                                                                         0.0s
 => => sha256:dea0af8d19fa394b2f379ea1a34e05868ee10d0d2e0ac192543577e836e8bbc3 1.37kB / 1.37kB                                                                         0.0s
 => => sha256:b905d9f061ee99a4a2f70ae543d39534d0ebd5a2a3160586d8040e33b46ab236 6.28kB / 6.28kB                                                                         0.0s
 => => sha256:633ba29fd335042456b6e2c073636f6fa30de56f1331c442914739b92a479974 2.90MB / 2.90MB                                                                        11.5s
 => => sha256:1236e9bf79acbf88e15dcaae128ebaf83d42568946907850955a6b758231acc3 621.77kB / 621.77kB                                                                     0.5s
 => => sha256:7fbdde676e3feacce6fa851256b1c686302870b25e3046b4cda73470341f7fb9 11.27MB / 11.27MB                                                                      11.6s
 => => sha256:d20359761b99a9b8ca0aa6e49e6a5ebb2afdaa26bd813c5855a5cd100367ee9e 240B / 240B                                                                             2.2s
 => => extracting sha256:633ba29fd335042456b6e2c073636f6fa30de56f1331c442914739b92a479974                                                                              0.3s
 => => sha256:952f0aa3a3458a9764847c257a30239d349daf74974307382e367148f6512e5b 3.08MB / 3.08MB                                                                        13.3s
 => => extracting sha256:1236e9bf79acbf88e15dcaae128ebaf83d42568946907850955a6b758231acc3                                                                              0.6s
 => => extracting sha256:7fbdde676e3feacce6fa851256b1c686302870b25e3046b4cda73470341f7fb9                                                                              1.5s
 => => extracting sha256:d20359761b99a9b8ca0aa6e49e6a5ebb2afdaa26bd813c5855a5cd100367ee9e                                                                              0.0s
 => => extracting sha256:952f0aa3a3458a9764847c257a30239d349daf74974307382e367148f6512e5b                                                                              0.8s
 => [internal] load build context                                                                                                                                     11.8s
 => => transferring context: 50.80MB                                                                                                                                  10.8s
 => [inventree_base  2/10] RUN apk add --no-cache     git gettext py-cryptography     libjpeg libwebp zlib     py3-pip py3-pillow py3-cffi py3-brotli pango poppler-  51.7s
 => [inventree_base  3/10] RUN mkdir -p /home/inventree                                                                                                                1.0s
 => [inventree_base  4/10] WORKDIR /home/inventree                                                                                                                     0.2s 
 => [inventree_base  5/10] COPY ./docker/requirements.txt base_requirements.txt                                                                                        0.6s 
 => [inventree_base  6/10] COPY ./requirements.txt ./                                                                                                                  0.3s 
 => [inventree_base  7/10] RUN if [ `apk --print-arch` = "armv7" ]; then     printf "[global]\nextra-index-url=https://www.piwheels.org/simple\n" > /etc/pip.conf ;    0.8s 
 => [inventree_base  8/10] RUN apk add --no-cache --virtual .build-deps     gcc g++ musl-dev openssl-dev libffi-dev cargo python3-dev     jpeg-dev openjpeg-dev li  1149.2s 
 => [inventree_base  9/10] COPY tasks.py docker/gunicorn.conf.py docker/init.sh ./                                                                                     1.8s 
 => [inventree_base 10/10] RUN chmod +x init.sh                                                                                                                        1.0s 
 => [production 1/1] COPY InvenTree ./InvenTree                                                                                                                       16.6s 
 => exporting to image                                                                                                                                                13.8s 
 => => exporting layers                                                                                                                                               13.8s 
 => => writing image sha256:a3b9caff980f60f6b3b92dd4ee9464a74389b5a5864839316f7b0c14f1923db6   

Seems to have run without issues...

[matmair]

@simonkuehling you should be able to use that image now - might be beneficial to tag the image.

[SchrodingersGat]

Nice, it looks like our recent docker changes have got it working on raspbian again. As the rasbian user base is low, we are no longer publishing this image by default. Glad that you have got it working locally though!

[simonkuehling]

Thanks for the hints - I have updated my upgrade routine to building myself with

$ docker build --tag inventree/inventree:latest . --target production

integrates neatly into the existing setup just like pulled from docker hub, sweet!

[simonkuehling]

I may have found a hickup in my self-compiled docker image here: I am missing all translations in the UI - do you have a hint on how to debug this?

[wolflu05]

Have you run invoke update?

[simonkuehling]

@wolflu05 yes, that was run (I double checked again to be sure...)

[SchrodingersGat]

@simonkuehling those stats are compiled as part of the invoke update step and should be inside your container. You should sh into your running container, and see if you have the file /home/inventree/InvenTree/InvenTree/locale_stats.json.

If not, run the invoke translate-stats command (inside your container) and see if the file gets generated.

It may be that in your custom docker setup, this file is not being generated, or is being deleted when the container restarts

[simonkuehling]

The locale_stats.json file is not there unfortunately:

/home/inventree/InvenTree/InvenTree # ls
__init__.py          config.py            filters.py           middleware.py        sentry.py            tasks.py             tests.py
__pycache__          context.py           format.py            migrations           serializers.py       template.py          unit_test.py
admin.py             conversion.py        forms.py             mixins.py            settings.py          test_api.py          urls.py
api.py               email.py             helpers.py           models.py            social_auth_urls.py  test_middleware.py   validators.py
api_version.py       exceptions.py        helpers_model.py     permissions.py       static               test_tasks.py        version.py
apps.py              exchange.py          management           ready.py             status.py            test_urls.py         views.py
ci_render_js.py      fields.py            metadata.py          sanitizer.py         status_codes.py      test_views.py        wsgi.py

Trying the invoke translate-stats command inside of the inventree-server container I get an import error:

/home/inventree/InvenTree/InvenTree # invoke translate-stats
Traceback (most recent call last):
  File "/usr/local/bin/invoke", line 8, in <module>
    sys.exit(program.run())
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 387, in run
    self.parse_collection()
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 479, in parse_collection
    self.load_collection()
  File "/usr/local/lib/python3.10/site-packages/invoke/program.py", line 716, in load_collection
    module, parent = loader.load(coll_name)
  File "/usr/local/lib/python3.10/site-packages/invoke/loader.py", line 91, in load
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/inventree/InvenTree/InvenTree/tasks.py", line 18, in <module>
    from django.db.migrations.executor import MigrationExecutor
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/__init__.py", line 1, in <module>
    from .migration import Migration, swappable_dependency  # NOQA
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/migration.py", line 1, in <module>
    from django.db.migrations import operations
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/operations/__init__.py", line 1, in <module>
    from .fields import AddField, AlterField, RemoveField, RenameField
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 2, in <module>
    from django.db.models import NOT_PROVIDED
  File "/usr/local/lib/python3.10/site-packages/django/db/models/__init__.py", line 3, in <module>
    from django.db.models.aggregates import *  # NOQA
  File "/usr/local/lib/python3.10/site-packages/django/db/models/aggregates.py", line 5, in <module>
    from django.db.models.expressions import Case, Func, Star, When
  File "/usr/local/lib/python3.10/site-packages/django/db/models/expressions.py", line 10, in <module>
    from django.db.models import fields
  File "/usr/local/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 11, in <module>
    from django import forms
  File "/usr/local/lib/python3.10/site-packages/django/forms/__init__.py", line 6, in <module>
    from django.forms.boundfield import *  # NOQA
  File "/usr/local/lib/python3.10/site-packages/django/forms/boundfield.py", line 4, in <module>
    from django.forms.utils import flatatt, pretty_name
  File "/usr/local/lib/python3.10/site-packages/django/forms/utils.py", line 7, in <module>
    from django.utils.html import escape, format_html, format_html_join, html_safe
  File "/usr/local/lib/python3.10/site-packages/django/utils/html.py", line 13, in <module>
    from django.utils.http import RFC3986_GENDELIMS, RFC3986_SUBDELIMS
  File "/usr/local/lib/python3.10/site-packages/django/utils/http.py", line 8, in <module>
    from email.utils import formatdate
  File "/home/inventree/InvenTree/InvenTree/email.py", line 6, in <module>
    from django.core import mail as django_mail
  File "/usr/local/lib/python3.10/site-packages/django/core/mail/__init__.py", line 9, in <module>
    from django.core.mail.message import (
  File "/usr/local/lib/python3.10/site-packages/django/core/mail/message.py", line 2, in <module>
    from email import (
ImportError: cannot import name 'charset' from partially initialized module 'email' (most likely due to a circular import) (/home/inventree/InvenTree/InvenTree/email.py)

Since email.py was added recently - is this a pointer?

[SchrodingersGat]

That's odd, I have not seen that circular import popping up anywhere else. Running invoke translate-stats works without issue on my end.

/home/inventree/InvenTree/InvenTree # invoke translate-stats

Try running the command from the top-level directory (where tasks.py is located)

[simonkuehling]

@SchrodingersGat alright, running it from the top-level directory worked - and it fixed the translations in the UI!

Question ist now, why doesn't it generate during invoke update?

[SchrodingersGat]

Ok, can you try deleting that file and running invoke update?

[simonkuehling]

Can you confirm which file I should delete - locale_stats.json?

[SchrodingersGat]

Correct, locale_stats.json

[SchrodingersGat]

And see if it gets regenerated by running invoke update in the correct directory

[simonkuehling]

yes, confirmed. locale_stats.json now got generated during invoke update.

I just realized, that although the translations are shown in the language selection drop-down now - the UI isn't actually translated when selecting a language other than EN... Tried with emptied cache as well..

German is selected, Set Language klicked, but UI stays english:

[simonkuehling]

Well, it's a pretty wild mix - there are some parts that are translated (table titles for example...):

[simonkuehling]

All of the steps above have been issued from within a running container, not sure if I made this clear enough. After restarting the container, translations are gone again - seems not to be held persistent in the container...

[wolflu05]

Restarting or recreating? Maybe we need to run this command in the Dockerfile so that this file is stored in the container image.

[simonkuehling]

When I stop the container, then run invoke update and start the container again, no translations are available - like this:

$ docker-compose down
$ docker-compose run inventree-server invoke update
$ docker-compose up -d

When I sh into the running container, I can run invoke update in there, which generates locale_stats.json and translations are then available in the Settings dop-down.

$ docker exec -t -i inventree-server /bin/sh
/home/inventree # invoke update

But after a docker-compose down -> docker-compose up -d translations are not available again.

[wolflu05]

Yes, then you mean recreate. Basically down removes the containers and up creates them. Additionally run creates its own container no matter if there is already one started, so run inventree-server invoke update only updates in the container and everything is lost afterwards which is not in a volume.

Either you can mount the translation file as a volume to temporary fix this issue, or we should run the translation task on docker build so that the translations are available in the image.

[simonkuehling]

ok, I see.

Since those files are not dependent on user data, I think it would be best to include them in docker build, right. That would also speed up the invoke update run for all dockerhub image users, even if only slightly, right?

[SchrodingersGat]

Definitely looks like we need to add that locale_stats.json file to the compiled docker image.

[simonkuehling]

@SchrodingersGat I think I'm still struggling to understand the difference between the prebuilt dockerhub images and the one I am building myself here - why is the translation compilation working with the dockerhub images but not with the image I am building myself...?

[SchrodingersGat]

@simonkuehling can you refresh my memory here

• what behaviour are you seeing on your system (building locally)
• how are you verifying "good" behaviour on the prebuilt images?

[simonkuehling]

@SchrodingersGat the topic shifted a bit from the initial issue (was: no arm/v7 docker image available -> now: translations are not working in locally built image)

The current problem as far as I have understood from our tests earlier is that the generation of locale_stats.json during invoke update is not persistent with my self-built image. In the debugging above we have proven that it is indeed beeing generated correctly during update, but just vanishes as soon as i recreate the container.

For comparison as "good" behaviour I am referring to the demo installation which is running the latest production dockerhub image I suppose...? All Translations are available for selection and working through the Settings page...

[SchrodingersGat]

Looking through the docker pathways I think that what is happening for the demo server is that the translation stats are being computed through the invoke update command, and the container is persisting until the next update (which happens daily).

If the demo docker image was restarted, the stats would likely be gone.

We could either:

• Expose the stats file to an external volume (e.g. data)
• Run the translation stats update script whenever the docker image runs
• ???

[simonkuehling]

I see, that might explain why it is working over there.

Sounds logical to me to just expose the file to the data volume like the other precompiled assets... right?

[SchrodingersGat]

@simonkuehling I have created a new issue specific to the translation stats file, and closing this one out

[judahrand]

Was the conclusion here that arm/v7 docker images are intended to have no support? Is this still the case?

[SchrodingersGat]

We do not push arm/v7 builds to dockerhub but it should be possible to build a compatible docker image yourself using the dockerfile

[judahrand]

We do not push arm/v7 builds to dockerhub but it should be possible to build a compatible docker image yourself using the dockerfile

Would there be any interest in building and pushing arm/v7? Something that a PR might be accepted for? Or is it a deliberate decision not to?

[matmair]

We pushend images in the past but the CI ran for a very long time causing significant friction in the development flow so it was removed. I do not think we would re-add it if the CI takes any longer then the current CI item for docker.

[wolflu05]

What about installing the 64 bit raspberry pi os version? Most modern raspberry pis >= 3 should support that. That way arm64 docker images should run fine.