`AttributeError: module '\''lib'\'' has no attribute '\''OpenSSL_add_all_algorithms'\''` caused by outdated pyOpenSSL

tytan652 commented 1 year ago

Describe the bug

If synapse happen to be installed with pyOpenSSL 22.0.0, update and restoration will always fail because the service can't start.

Cleaning pyOpenSSL from the backup fix the issue because it installs a newer version of pyOpenSSL.

sudo tar --delete --file synapse-pre-upgrade2.tar apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/pyOpenSSL-22.0.0.dist-info/

sudo tar --delete --file synapse-pre-upgrade2.tar apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/OpenSSL/

The filename synapse-pre-upgrade2.tar may differ.

Context

Hardware: computer
YunoHost version: 11.0.10.2
I have access to my server: Through SSH | through the webadmin
Are you in a special context or did you perform some particular tweaking on your YunoHost instance?: no
Using, or trying to install package version/branch: 1.71 | 1.74
If upgrading, current package version: 1.74

Steps to reproduce

Get synapse installed with pyOpenSSL 22.0.0 (so before a newer version was released)
With an up to date system updating or restoring synapse will not work because AttributeError: module '\''lib'\'' has no attribute '\''OpenSSL_add_all_algorithms'\''

Expected behavior

Update or restore without issues.

Logs

Update 1.71 to 1.74: https://paste.yunohost.org/raw/laramokoqi
Restoration after failure: https://paste.yunohost.org/raw/owomafigap
Restoration through webadmin: https://paste.yunohost.org/raw/afumobabun

MayeulC commented 1 year ago

Same issue happening to me. It's especially annoying as backup and restore take so long (1 hour to back up, 3.5 hours and counting for restoration...). If you tell me restore will fail, I'm going to kill the process and apply your workaround, thanks a lot!

Edit: it restored cleanly after a few hours, it seems. I was able to install the upgrade after that.

fflorent commented 1 year ago

Thank you very much @tytan652, your trick saved me hours of sweating.

Thatoo commented 1 year ago

Is it possible to avoid the crash, need to clean pyOpenSSSL, restore and then update?

Is it possible for example to force update of pyOpenSSSL instead before updating synapse?

tio-trom commented 1 year ago

So the fix provided in the thread is only to restore synapse, correct? The update will fail anyway. Correct? It failed for me too but I normally do server snapshots before I update synapse so I could restore no problems. Log https://paste.yunohost.org/raw/ecabacicef

tytan652 commented 1 year ago

So the fix provided in the thread is only to restore synapse, correct?

Yes, because I had a failure when updating.

The update will fail anyway. Correct?

If you don't update pyOpenSSL 22.0.0 beforehand, yes. Unfortunately, I don't know how to interract with synapse python env to update pyOpenSSL.

Thatoo commented 1 year ago

So the fix provided in the thread is only to restore synapse, correct? The update will fail anyway.

But after the update has failed, you do the fix before restoring the last backup you have done (just before updating is the best) and then you can update safely.

tio-trom commented 1 year ago

So the fix provided in the thread is only to restore synapse, correct? The update will fail anyway.

But after the update has failed, you do the fix before restoring the last backup you have done (just before updating is the best) and then you can update safely.

Thing is my Synapse is around 100GB in size and restoring it may fail or take way to long. I'd better wait for a fix. I usually update with no backups for Synapse, only server snapshots, since it is way faster and safer to restore in case of a failure.

MayeulC commented 1 year ago

So the fix provided in the thread is only to restore synapse, correct? The update will fail anyway. Correct? It failed for me too but I normally do server snapshots before I update synapse so I could restore no problems. Log https://paste.yunohost.org/raw/ecabacicef

No, it's modifying the backup before restoring. The upgrade will then succeed.

You can also stop synapse, remove that OpenSSL folder, and perform the normal upgrade, I think. Or instead of removing the folder, trigger the upgrade from pip (ran as the synapse user with the correct python version).

I agree it's annoying to wait for backup and restore, it took about a day for me (one backup, two restores).

One possible fix would be to delete or upgrade dependencies before installing/starting the synapse upgrade. Or not saving the pip site-packages during backup, since they are obtained again later.

Josue-T commented 1 year ago

Can you try the https://github.com/YunoHost-Apps/synapse_ynh/tree/fix_issue_335 branch to see if it fix the issue ?

tytan652 commented 1 year ago

Can you try the https://github.com/YunoHost-Apps/synapse_ynh/tree/fix_issue_335 branch to see if it fix the issue ?

This require to have an instance of synapse with pyOpenSSL 22.0.0, once someone has tried to normally update (and fail) they can no longer try this since the instance is removed because of the failure.

Edit: Which I no longer have since I met the issue.

tio-trom commented 1 year ago

Can you try the https://github.com/YunoHost-Apps/synapse_ynh/tree/fix_issue_335 branch to see if it fix the issue ?

I can try. What is the comment to upgrade and also skip the backups?

sudo yunohost app upgrade synapse -u -b https://github.com/YunoHost-Apps/synapse_ynh/tree/testing --debug

MayeulC commented 1 year ago

Can you try the https://github.com/YunoHost-Apps/synapse_ynh/tree/fix_issue_335 branch to see if it fix the issue ?

I can try. What is the comment to upgrade and also skip the backups?

sudo yunohost app upgrade synapse -u -b https://github.com/YunoHost-Apps/synapse_ynh/tree/testing --debug

I'm pretty sure the url needs to come just after -u, so probably sudo yunohost app upgrade synapse -u https://github.com/YunoHost-Apps/synapse_ynh/tree/fix_issue_335 -b --debug (note that I also changed the URL to point to the correct branch).

KamilBaczkowski commented 1 year ago

I am also affected by this issue. I tried updating synapse a while ago, and I can't restore my backup because of this error.

edit: Will using the workaround from the first post result in any breakage? I have seen on the forums that it may lead to data loss (So I lost my crypto keys and my verification (and all the encrypted messages), but at least the contacts seems to be there. ~https://forum.yunohost.org/t/solved-synapse-update-failed-restore-not-possible/22888/2)

MayeulC commented 1 year ago

@KamilBaczkowski no, if you only perform the commands in the workaround above, it should not result in any data loss, including cryptographic keys.

In fact, the only way to lose crypto keys is to log out from every client (and not have a key backup).

I still suggest you make a copy of the backup before changing it.

tio-trom commented 1 year ago

I'm pretty sure the url needs to come just after -u, so probably sudo yunohost app upgrade synapse -u https://github.com/YunoHost-Apps/synapse_ynh/tree/fix_issue_335 -b --debug (note that I also changed the URL to point to the correct branch).

Thanks I will test it these days and let you know.

Josue-T commented 1 year ago

Fixed by #358

YunoHost-Apps / synapse_ynh