Loss of inventory in upgrade from 3.6 to 3.20

[andreusoler8]

Summary

I am following the official upgrade documentation (https://www.thethingsindustries.com/docs/getting-started/upgrading/) to upgrade from my current version of TTS 3.6.0 to the latest version available. Following the steps, I manage to update the version but neither the registered users, nor the gateways, nor the applications with their devices are maintained. What should I do to be able to keep all my data without being deleted?

Current Situation

Upgrade done successfully.

Desired Situation

Update performed keeping all previous data.

Code of Conduct

• [X] I agree to follow TTN's Community Code of Conduct.

[NicolasMrad]

Thank you for submitting the issue @andreusoler8 , can you please provide logs for the upgrade process because it is not possible not upgrade to v3.30 directly from v3.6, you have to upgrade to 3.19 first.

[andreusoler8]

Hi @NicolasMrad, these are the logs (with the command 'docker compose logs') created during the upgrade process to version 3.19 with the same problem of loss of inventory:

thethingsstack-redis-1 | 1:M 05 Jul 2022 11:07:03.143 # User requested shutdown... thethingsstack-redis-1 | 1:M 05 Jul 2022 11:07:03.143 Calling fsync() on the AOF file. thethingsstack-redis-1 | 1:M 05 Jul 2022 11:07:03.143 Saving the final RDB snapshot before exiting. thethingsstack-redis-1 | 1:M 05 Jul 2022 11:07:03.173 DB saved on disk thethingsstack-redis-1 | 1:M 05 Jul 2022 11:07:03.173 # Redis is now ready to exit, bye bye... thethingsstack-redis-1 | 1:C 05 Jul 2022 11:08:21.052 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo thethingsstack-redis-1 | 1:C 05 Jul 2022 11:08:21.052 # Redis version=7.0.0, bits=64, commit=00000000, modified=0, pid=1, just started thethingsstack-redis-1 | 1:C 05 Jul 2022 11:08:21.052 # Configuration loaded thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.052 monotonic clock: POSIX clock_gettime thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.053 Running mode=standalone, port=6379. thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.053 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.053 # Server initialized thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 Reading RDB base file on AOF loading... thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 Loading RDB produced by version 7.0.0 thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 RDB age 2845757 seconds thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 RDB memory usage when created 0.82 Mb thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 RDB is base AOF thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 Done loading RDB, keys loaded: 0, keys expired: 0. thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:21.054 DB loaded from base file appendonly.aof.1.base.rdb: 0.000 seconds thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:22.732 DB loaded from incr file appendonly.aof.1.incr.aof: 1.678 seconds thethingsstack-redis-1 | 1:M 05 Jul 2022 11:08:22.732 Ready to accept connections thethingsstack-redis-1 | 1:M 05 Jul 2022 11:09:22.099 10000 changes in 60 seconds. Saving... thethingsstack-redis-1 | 1:M 05 Jul 2022 11:09:22.100 Background saving started by pid 21 thethingsstack-redis-1 | 21:C 05 Jul 2022 11:09:22.143 DB saved on disk thethingsstack-redis-1 | 21:C 05 Jul 2022 11:09:22.144 Fork CoW for RDB: current 4 MB, peak 4 MB, average 3 MB thethingsstack-redis-1 | 1:M 05 Jul 2022 11:09:22.200 * Background saving terminated with success

[adriansmares]

Hey @andreusoler8,

Could you please provide the logs for the actual migration operations ? (is-db migrate, ns-db migrate etc.)

We cannot really tell what caused your inventory to be lost only from the Redis logs - they do not show any data loss themselves. Also be sure to change the image for Redis from redis:latest to redis:6.2.

[danijelk]

@NicolasMrad

Thank you for submitting the issue @andreusoler8 , can you please provide logs for the upgrade process because it is not possible not upgrade to v3.30 directly from v3.6, you have to upgrade to 3.19 first.

Is the 3.19 jump documented anywhere, just upgraded to 3.22.1 but had issues as I had to do 3.19 migrations first and then jump to 3.22, without your comment I would probably have never found out I feel.