Migrate from 5.0 to 5.5

[marshalleq]

Placeholder for how to do this - presently, simply pulling the new version fails. Anyone else done this and know how?

[blackandred]

Could you provide more details?

[marshalleq]

I am making the assumption that this is not just me, so purposefully left the details out. The basic problem is that if I replace my current version 5 taiga docker repository with the 5.5 one it fails. Perhaps I need to do an export and reimport it or something. I am wondering if anyone else has tried to get to 5.5. and how they have done that.

[Sawrz]

I had and still have issues upgrading from 5.0 to 5.5.

The first error occurred during the server container startup, telling me about some slack migration errors. I solved it by removing slack from the PLUGINS environment variable.

Now, the container starts as expected. However, I can't log in properly. Taiga tells me my username or password is wrong, which isn't the case since I use a password manager, and before the update, everything worked fine.

So I checked the logs and found that:

The above exception was the direct cause of the following exception:
.
.
.
django.db.utils.ProgrammingError: column users_user.verified_email does not exist
LINE 1: ...rs_user"."email_token", "users_user"."new_email", "users_use...

It seems like a database migration error. I can roll back without any issue, but I'm still curious about what happens there. And, of course, in the long term, I want to upgrade.

I thought of a not verified email since I had no Taiga email set up. So I went through the process and set everything up, but I got the same issue. Since I'm the only one using Taiga, I don't know which emails aren't verified.

There are other users like Bitbucket, GitHub, etc. in the admin panel but I don't want to delete them since I don't know if I mess things up.

That's my journey, so far, in upgrading from 5.0.15 to 5.5.5 (it seems it doesn't matter if I use an older 5.x version).

[blackandred]

1) Slack error - I think we should have some kind of tests coverage and tests on CI in such cases, it should be on our todo list 2) Database migration error - If the migrations are starting correctly in the container, then it is on Taiga project side, on on the infrastructure side. From a DevOps perspective I would recommend you all to migrate version-by-version 5.0 -> 5.1 -> 5.2 -> 5.3 -> 5.4 -> 5.5, all the time incrementing minor part.

[blackandred]

Hmm... maybe this one https://github.com/riotkit-org/docker-taiga/issues/45 could be somehow related?

[blackandred]

1) https://github.com/riotkit-org/docker-taiga/issues/47 2) Please let me know if migrating step-by-step works

[Dzordzu]

The problem is that 5.1/5.2/5.3/5.4 versions were not provided

[Sawrz]

I would recommend you all to migrate version-by-version 5.0 -> 5.1 -> 5.2 -> 5.3 -> 5.4 -> 5.5, all the time incrementing minor part.

That was my idea too. Since somewhere in between the databases changed their structure. A proper migration should fix this, at least in theory.

The problem is that 5.1/5.2/5.3/5.4 versions were not provided (docker version)

And that's my issue at this point. I'm not able to update in minor version steps.

Hmm... maybe this one #45 could be somehow related?

I'm not sure about this. I delete all containers and volumes first, before using docker-compose to build my stack. So it touches only the database and the media folder located on my hard drive. But maybe I'm getting it somehow wrong.

[Dzordzu]

The issue is with migrations. They are absent (the upgrade causes ERROR: column users_user.verified_email does not exist)

[Dzordzu]

What's interesting theris wiki do not say anything about breaking changes from 5.0 to 5.5. It can simply mean that migrations are not triggered during docker-compose up

[Sawrz]

Its been a while since I worked with Django. I think I had some issues back then with migration as well. But I think they were related to drastic changes in my database due to development.

What's interesting theris wiki do not say anything about breaking changes from 5.0 to 5.5. It can simply mean that migrations are not triggered during docker-compose up

I don't know how the taiga folks version policy but a minor bump usually means an added feature (column) but no breaking changes. And from that point of view, it's no breaking change since a migration can probably fix any issue.

So, it seems @Dzordzu might have a point here. However, if it's an automated process, having those minor version bumps shouldn't be too much work either, right?

[blackandred]

Sorry, I didn't check that - there is a gap in the releases of Taiga. They just didn't produce middle versions.

[blackandred]

If the migrations are the issue - and there are no middle versions, then we need logs from migrating between 5.0 to 5.5 I will try today in few hours to upgrade from 5.0 to 5.5 on the empty instance, I hope I would be able to reproduce it.

[Sawrz]

I checked the Dockerfile:

RUN cp /opt/taiga-conf/taiga/local.py /usr/src/taiga-back/settings/local.py \
    && cp /opt/taiga-conf/taiga/docker.py /usr/src/taiga-back/settings/docker.py \
    && cp /opt/taiga-conf/taiga/celery.py /usr/src/taiga-back/settings/celery.py \
    && j2 /opt/taiga-conf/locale.gen.j2 > /etc/locale.gen \
    \
    # NOTICE: In case that "collectstatic" would not be found, it means in Django that the Taiga was not loaded
    # and only basic Django application is configured because of an error, so Taiga commands are not loaded in that case
    \
    && cd /usr/src/taiga-back/ && python manage.py collectstatic --noinput \
    && mkdir -p /var/log/nginx /var/lib/nginx \
    && touch /var/run/nginx.pid

If I remember correctly you need to apply a migration before python manage.py collectstatic --noinput.

See here. I changed from those guys to your repo since I like your all in one solution more. Maybe that helps.

[blackandred]

Why the database migrations would be depending on static fles? Do you have any clue? That would mean to execute collectstatic on each container restart, right after migrations. Anyway I could try to check it.

On Wed, Oct 7, 2020 at 7:47 PM Sandro Wrzalek notifications@github.com wrote:

I checked the Dockerfile:

RUN cp /opt/taiga-conf/taiga/local.py /usr/src/taiga-back/settings/local.py \ && cp /opt/taiga-conf/taiga/docker.py /usr/src/taiga-back/settings/docker.py \ && cp /opt/taiga-conf/taiga/celery.py /usr/src/taiga-back/settings/celery.py \ && j2 /opt/taiga-conf/locale.gen.j2 > /etc/locale.gen \ \

NOTICE: In case that "collectstatic" would not be found, it means in Django that the Taiga was not loaded

# and only basic Django application is configured because of an error, so Taiga commands are not loaded in that case
\
&& cd /usr/src/taiga-back/ && python manage.py collectstatic --noinput \
&& mkdir -p /var/log/nginx /var/lib/nginx \
&& touch /var/run/nginx.pid

If I remember correctly you need to apply a migration before python manage.py collectstatic --noinput.

See here https://github.com/devinsolutions/docker-taiga/blob/master/docker-compose.advanced.yml. I changed from those guys to your repo since I like your all in one solution more. Maybe that helps.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/riotkit-org/docker-taiga/issues/46#issuecomment-705094282, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFOAJBVFGTUKCMSSJF3I4XTSJSSS5ANCNFSM4Q7DQ4WQ .

[Sawrz]

I add my logs, maybe that helps as well:

,
2020-10-07 19:52:00,027 DEBG 'backend' stderr output:,
[2020-10-07 19:52:00 +0200] [46] [INFO] Booting worker with pid: 46,
,
2020-10-07 19:52:00,045 DEBG 'backend' stderr output:,
Trying import local.py settings...,
,
2020-10-07 19:52:00,095 DEBG 'backend' stderr output:,
Trying import local.py settings...,
,
2020-10-07 19:52:00,184 DEBG 'backend' stderr output:,
Trying import local.py settings...,
,
2020-10-07 19:52:01,185 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs),
2020-10-07 19:52:01,185 INFO success: send_app_notifications entered RUNNING state, process has stayed up for > than 1 seconds (startsecs),
2020-10-07 19:52:01,185 INFO success: backend entered RUNNING state, process has stayed up for > than 1 seconds (startsecs),
2020-10-07 19:52:43,300 DEBG 'nginx' stdout output:,
127.0.0.1 - - [07/Oct/2020:19:52:43 +0200] "GET / HTTP/1.1" 200 92023 "-" "curl/7.64.0" "-",
,
2020-10-07 19:53:21,981 DEBG 'backend' stderr output:,
ERROR:2020-10-07 17:53:21,463: Internal Server Error: /api/v1/auth,
Traceback (most recent call last):,
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute,
    return self.cursor.execute(sql, params),
psycopg2.errors.UndefinedColumn: column users_user.verified_email does not exist,
LINE 1: ...rs_user"."email_token", "users_user"."new_email", "users_use...,
                                                             ^,
,
,
The above exception was the direct cause of the following exception:,
,
Traceback (most recent call last):,
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner,
    response = get_response(request),
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response,
    response = self.process_exception_by_middleware(e, request),
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response,
    response = wrapped_callback(request, *callback_args, **callback_kwargs),
  File "/usr/src/taiga-back/taiga/base/api/viewsets.py", line 106, in view,
    return self.dispatch(request, *args, **kwargs),
  File "/usr/local/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view,
    return view_func(*args, **kwargs),
  File "/usr/src/taiga-back/taiga/base/api/views.py", line 460, in dispatch,
    response = self.handle_exception(exc),
  File "/usr/src/taiga-back/taiga/base/api/views.py", line 449, in dispatch,
    self.initial(request, *args, **kwargs),
  File "/usr/src/taiga-back/taiga/base/api/views.py", line 379, in initial,
    self.perform_authentication(request),
  File "/usr/src/taiga-back/taiga/base/api/views.py", line 327, in perform_authentication,
    request.user,
  File "/usr/src/taiga-back/taiga/base/api/request.py", line 264, in user,
    self._authenticate(),
  File "/usr/src/taiga-back/taiga/base/api/request.py", line 430, in _authenticate,
    user_auth_tuple = authenticator.authenticate(self),
  File "/usr/src/taiga-back/taiga/auth/backends.py", line 62, in authenticate,
    if not user or not user.is_active:,
  File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 256, in inner,
    self._setup(),
  File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 392, in _setup,
    self._wrapped = self._setupfunc(),
  File "/usr/local/lib/python3.7/site-packages/django/contrib/auth/middleware.py", line 24, in <lambda>,
    request.user = SimpleLazyObject(lambda: get_user(request)),
  File "/usr/local/lib/python3.7/site-packages/django/contrib/auth/middleware.py", line 12, in get_user,
    request._cached_user = auth.get_user(request),
  File "/usr/local/lib/python3.7/site-packages/django/contrib/auth/__init__.py", line 189, in get_user,
    user = backend.get_user(user_id),
  File "/usr/local/lib/python3.7/site-packages/django/contrib/auth/backends.py", line 102, in get_user,
    user = UserModel._default_manager.get(pk=user_id),
  File "/usr/local/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method,
    return getattr(self.get_queryset(), name)(*args, **kwargs),
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 402, in get,
    num = len(clone),
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 256, in __len__,
    self._fetch_all(),
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all,
    self._result_cache = list(self._iterable_class(self)),
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__,
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size),
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1140, in execute_sql,
    cursor.execute(sql, params),
  File "/usr/local/lib/python3.7/site-packages/raven/contrib/django/client.py", line 127, in execute,
    return real_execute(self, sql, params),
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute,
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute),
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers,
    return executor(sql, params, many, context),
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute,
    return self.cursor.execute(sql, params),
  File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__,
    raise dj_exc_value.with_traceback(traceback) from exc_value,
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute,
    return self.cursor.execute(sql, params),
django.db.utils.ProgrammingError: column users_user.verified_email does not exist,
LINE 1: ...rs_user"."email_token", "users_user"."new_email", "users_use...,
                                                             ^,
,
,
2020-10-07 19:53:21,981 DEBG 'nginx' stdout output:,
172.19.0.4 - - [07/Oct/2020:19:53:21 +0200] "POST /api/v1/auth HTTP/1.1" 500 37 "https://<REDACTED>/login?next=%252Fdiscover" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:81.0) Gecko/20100101 Firefox/81.0" "<REDACTED>",
,

[Sawrz]

Why the database migrations would be depending on static fles? Do you have any clue? That would mean to execute collectstatic on each container restart, right after migrations. Anyway I could try to check it.

Do you mean in their setup?

However, that could lead to other issues since those guys populate the database on the first run manually. So you might get errors when trying to migrate a non-existent database.

[blackandred]

I think I found the issue, I'm now testing it, probably I will release a fix as soon as I confirm that this is the case.

[blackandred]

I found an issue, it was serious. The migrations were not always executing - so application installation was successful, but upgrade not. Again I think there should be some tests coverage for that, I created an issue already to add some tests executed by CI.

I just tagged a RC version for you for testing: https://github.com/riotkit-org/docker-taiga/releases/tag/1.4.1-RC1 I tested it locally only, and the application is up in recent version v5.5.5, migrated from v5.0.14

CI will produce images with a few minutes: https://quay.io/repository/riotkit/taiga?tab=tags

[blackandred]

When you will say that the image resolves your issue I will tag a new stable version without RC and we can close the issue.

[Sawrz]

The upgrade worked for me. I'm on 5.5.5. now and the logs showed that the migrations applied. =)

Thanks for your quick response!

[Dzordzu]

Wow. It was the quickest fix I've ever seen. And what's astonishing - it works. Thanks!

EDIT: It's not sarcastic. You've done amazing job. Within very few hours after spotting the issue, you've fixed it and deploy. And it's your volountary work, not some kind of the job

[blackandred]

@Dzordzu Thanks, I do so on daily basis for non-profit org. and open source :wink: My wage labour will start in 2 hours, but before it starts I have a moment for voluntary :stuck_out_tongue:

Regarding the issue - I see there are 2 people who tested, I'm tagging a stable version and closing the issue, thanks.

[blackandred]

https://github.com/riotkit-org/docker-taiga/releases/tag/1.4.1

[marshalleq]

Awesome thanks for the amazing help guys - you both exceed my talents for sure! Can't wait to try it!