search

nilsteampassnet / TeamPass

Collaborative Passwords Manager
https://www.teampass.net
1.66k stars 542 forks source link

Upgrade from TeamPass 3.0.0.22 to 3.1.2.12 - Some Items gone #4073

Open kultur-burgenland opened 7 months ago

kultur-burgenland commented 7 months ago

Hi, I'd like to share my experience, including challenges, with upgrading TeamPass from version 3.0.0.22 to 3.1.2.12. 🙂

Steps to Reproduce

  1. I copied all files from version 3.1.2.12 into my TeamPass folder.
  2. Attempting to start the upgrade in my test environment failed on the first upgrade screen, resulting in a JSON error displayed in the developer console. Fortunately, the solution provided in this GitHub issue comment resolved my issue: https://github.com/nilsteampassnet/TeamPass/issues/3553#issuecomment-1490828271
  3. Subsequently, I was able to complete the upgrade process and access my TeamPass instance. However, I encountered several issues, particularly with my AD/LDAP configuration, which required adjustments. More on that later.
  4. As admin, I manually started all the tasks ("Remove orphaned items", "Recreate cash" and so on...) and set up routines for them.
  5. With all non-critical functionalities working as expected, I proceeded to update my production environment.
  6. A few hours post-update, a colleague reported missing entries within a folder (not a private one, but one several people had access to). An investigation and comparison between the updated and original databases revealed that approximately 70 of my ~3500 items, including their related log entries, were missing. At first I thought this happened during the upgrade, but after a second test I found out, that executing the tasks deletes the items.
  7. Despite examining these entries, no clear pattern emerged regarding their deletion. They varied in description, email addresses, URLs, and passwords / password characters, including special characters like !, ?, and @.

Following this, I manually recreated one of the missing items, which subsequently seemed to interfere with the cron job execution, leading to an error when accessing the "Task" menu as an admin, which I did not encounter before creating the item:

DataTables warning: table id=table-tasks_in_progress - Ajax error.
For more information about this error, please see http:// datatables.net/tn/7

Those were the only warnings I got: PHP Warning: Undefined array key 1 in /home/teampass/public_html/TeamPass/pages/tasks.php on line 232, referer: http://192.168.0.207/teampass/index.php?page=tasks PHP Warning: Undefined array key 1 in /home/teampass/public_html/TeamPass/pages/tasks.php on line 260, referer: http://192.168.0.207/teampass/index.php?page=tasks PHP Warning: Undefined array key 1 in /home/teampass/public_html/TeamPass/pages/tasks.php on line 282, referer: http://192.168.0.207/teampass/index.php?page=tasks PHP Warning: Undefined array key 1 in /home/teampass/public_html/TeamPass/pages/tasks.php on line 326, referer: http://192.168.0.207/teampass/index.php?page=tasks

After that I rolled back to the old version.

Additional issues encountered with the new installation include:

Server configuration

Operating system: Ubuntu Server 22.04 Web server: Apache 2.4.52 Database: MariaDB 10.6.16 PHP version: 8.1.27 Teampass version: 3.0.0.22 -> 3.1.2.12 Teampass configuration file: - Updated from an older Teampass or fresh install: Update

kultur-burgenland commented 7 months ago

Okay, what I found out: The items definitely get deleted when starting the "Clean orphaned items" task at version 3.1.2.12, which does not happen on 3.0.0.22. So I guess something makes the software believe those items are orphaned after the update. When directly comparing the database entries, I don't see any differences except the newly added columns "item_key", "created_at", "updated_at" and "deleted_at". Does anyone have an idea what could cause this issue?

Arash9703 commented 7 months ago

Hello , I do confirm that the folder creating problem exists in 3.1.2 in my new debian (LAMP server ) instance. Searched a lot but didn't came up with any solutions ! @nilsteampassnet any suggestions would be appreciated.

kultur-burgenland commented 7 months ago

Update: The Ajax error in the tasks menu seems to be related to a missing column when updating.

PHP Fatal error:  Uncaught MeekroDBException: Unknown column 'task_id' in 'where clause' in /home/teampass/public_html/TeamPass/vendor/sergeytsalkov/meekrodb/db.class.php:934\nStack trace:\n#0 /home/teampass/public_html/TeamPass/vendor/sergeytsalkov/meekrodb/db.class.php(890): MeekroDB->queryHelper()\n#1 /home/teampass/public_html/TeamPass/vendor/sergeytsalkov/meekrodb/db.class.php(116): MeekroDB->query()\n#2 /home/teampass/public_html/TeamPass/sources/logs.datatables.php(1080): DB::__callStatic()\n#3 /home/teampass/public_html/TeamPass/sources/logs.datatables.php(906): getSubtaskProgress()\n#4 {main}\n  thrown in /home/teampass/public_html/TeamPass/vendor/sergeytsalkov/meekrodb/db.class.php on line 934, referer: http://192.168.0.207/teampass/index.php?page=tasks

After adding the column manually, the error is gone.

However, this does not solve the problem with processing the routine task suddenly removes some items after the update. :/ I even did a fresh install today, adding all the users, items, etc., through the database, but the problem remains.

kultur-burgenland commented 6 months ago

I have found the problem that causes some entries to be deleted after the upgrade: I previously deleted some inactive users, but their user ID in the id_user column of log_items remained unchanged, which seemed as intended by the software. Before the upgrade to version 3.1.1 (tested with 3.0.0.22 and 3.1.0) this was fine and the corresponding log entries only showed an empty name, but in version 3.1.1 the logic of the software/routines suggests that these entries are orphaned as soon as they contain even a single log entry that does not match an existing user ID, resulting in the item and all associated log entries being deleted completely. If the associated log entries are updated to a different user ID, e.g. 1 (admin), they are suddenly no longer deleted when the routine is executed. @nilsteampassnet I don't think this is intentional and would recommend fixing this as soon as possible, as it may lead to data loss without realizing it before it's too late...

In the end, I solved it by updating the corresponding columns as described, but I still only updated to version 3.1.0 because newer versions still have the bug that you can't create folders or items because the folder structure is displayed as undefined.