[NC 20+] Legacy encryption - mitigation and documentation thereof

[Kuphi]

With Nextcloud 20, the term "Legacy encryption" was introduced. The documentation does say little about what legacy encryption is: https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_migration.html I found another document, which gives hints about what could be meant: https://eprint.iacr.org/2020/1439.pdf

I run the command occ encryption:scan:legacy-format with output similar to this: sudo -u www-data php /var/www/nextcloud/occ encryption:scan:legacy-format Scanning all files for legacy encryption Scanning all files for username /username/files_trashbin/files/file1.odt.d1592989576 does not have a proper header /username/files_trashbin/versions/folder1/file2.xls.v1585723230 does not have a proper header /username/files_versions/folder2/file3.xls.v1189677150 does not have a proper header

Now I got some files with inproper headers. Does my nextcloud still use legacy encryption for all files or does it use a more secure encryption? What are the differences?

When running occ encryption I also encountered a new command: Command "encryption" is not defined. Did you mean one of these? encryption:change-key-storage-root encryption:decrypt-all encryption:disable encryption:disable-master-key encryption:enable encryption:enable-master-key encryption:encrypt-all encryption:list-modules encryption:migrate-key-storage-format encryption:recover-user encryption:scan:legacy-format encryption:set-default-module encryption:show-key-storage-root encryption:status

There is no documentation about encryption:migrate-key-storage-format but an excerpt from the file /nextcloud/core/Command/Encryption/MigrateKeyStorage.php - https://fossies.org/linux/nextcloud/core/Command/Encryption/MigrateKeyStorage.php 75 ->setName('encryption:migrate-key-storage-format') 76 ->setDescription('Migrate the format of the keystorage to a newer format')

Am I supposed to use this command? I have key-type: user keys ( https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_details.html#key-type-user-key ) Will I still be able to use user keys after running this command, and is it save to run it?

My feature request is: please answer this questions and update the documentation of nextcloud. Thank you very much for maintaining nextcloud! :)

[Kuphi]

For the documentation record: The command encryption:migrate-key-storage-format can not be run successfully, when your nextcloud is in maintenance mode. user@server:/home/user# sudo -u www-data php /var/www/nextcloud/occ encryption:migrate-key-storage-format Nextcloud is in maintenance mode - no apps have been loaded Updating key storage format Start to update the keys: 4 [----->----------------------] In Manager.php line 187: Module with ID: OC_DEFAULT_MODULE does not exist. encryption:migrate-key-storage-format You have to turn off maintenance mode first: sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off then it runs smoothly, but without any notice before start. Does it not bear any risks?:

user@server:/home/user# sudo -u www-data php /var/www/nextcloud/occ encryption:list-modules OC_DEFAULT_MODULE: Default encryption module [default*]

user@server:/home/user# sudo -u www-data php /var/www/nextcloud/occ encryption:migrate-key-storage-format Updating key storage format Start to update the keys: 20 [============================]Key storage format successfully updated

After the migration, occ encryption:scan:legacy-format does still output the same files as "does not have a proper header". Am I supposed to delete those files? Can I simply rm them and then do occ files:scan? (All my files with inproper header are either in the trashbin or in the folder files_versions.

[mark-orion]

Please someone explain what these "...does not have a proper header" error messages mean and how they can be fixed.

[leblitzdick]

Please someone explain what these "...does not have a proper header" error messages mean and how they can be fixed.

I also have the same question, can i set "encryption.legacy_format_support" to false without any problems?

[BenedictBauerHHN]

I have the same output on occ encryption:scan:legacy-format of course to other files then the OP. Could it lead to problems if I set encryption.legacy_format_support to false?

[developerrespig]

I would also be interested in some more information about that issue from an official side. Besides of that I just had a look into the legacy format scanner where this message is originating from (you can find the source code of that checker here: https://github.com/nextcloud/server/blob/1448b7c923d079c9616f87df3ffa52a4656ac6cc/apps/encryption/lib/Command/ScanLegacyFormat.php (I guess)).

With my limited php skills I guess it looks like these messages mean, that the check failed (because the private method returns false and the expected "All scanned files are properly encrypted. You can disable the legacy compatibility mode." doesn't get rendered). Question for me would be now: What to do about them? As far as I have seen them on my end they are only related to file versions or trash bins. But for now I just leave that configurational entry in the config.php and I guess you should do the same until someone says something different...

Hope to have something official here soon fingers crossed

[Kuphi]

Can we get a comment regarding this issue?

After upgrading to NC 20.0.3, it does anew complain about old server side encryption format. I thought I already resolved this?

"Das alte serverseitige Verschlüsselungsformat ist aktiviert. Wir empfehlen, es zu deaktivieren. Für weitere Einzelheiten sehe bitte in die Dokumentation."

[eckeSolutions]

In many instances the problem are old versions of files that are a few months old or even years. Cleaning those versions with "occ versions:cleanup" removed all does “not have a proper header” errors for me. Proceed with care!!! Might delete necessary file versions!!! running "occ encryption:scan:legacy-format" after that tells me it is save do set encryption.legacy_format_support to false

This is not a general solution, but should work for some people with this problem (if “/versions/” is included in the file urls). A solution to set the proper header would be nice though.

[Kuphi]

In many instances the problem are old versions of files that are a few months old or even years. Cleaning those versions with "occ versions:cleanup" removed all does “not have a proper header” errors for me. Proceed with care!!! Might delete necessary file versions!!! running "occ encryption:scan:legacy-format" after that tells me it is save do set encryption.legacy_format_support to false

This is not a general solution, but should work for some people with this problem (if “/versions/” is included in the file urls). A solution to set the proper header would be nice though.

Proceed with care? "occ versions:cleanup" does delete ALL previous versions of all files. This is only a solution if you do not need any previous versions at all.

[eckeSolutions]

Proceed with care? "occ versions:cleanup" does delete ALL previous versions of all files. This is only a solution if you do not need any previous versions at all.

Yes, that’s why I added 3 exclamation marks. In my instance it was no problem. And to be on the save side I have daily backups of everything.

[DPTJKKVH]

It is really sad that nobody of the team even acknowledges that encryption:migrate-key-storage-format is undocumented and elaborates in a few words what it does and if we should do it after disabling legacy encryption.

Also @developerrespig seems to be correct. I removed the bad header files manually, ran the check, got the "you can disbale legacy encryption" message and did just that.

I see neither issues nor changes so far which is good. But it drives me nuts that I don't see the point and don't know if we should migrate the storage format because it seems to be security related.

@schiessle @rullzer @nickvergessen can you please explain if we should run encryption:migrate-key-storage-format or not after disabling legacy encryption, even if you don't feel to explain it any further? Just a YES or NO is sufficient for the time being.

Many thanks in advance!

[thomwiggers]

I don't think encryption:migrate-key-storage-format solves this problem. There doesn't seem to be any migration option if I look in the source code of the Encryption module.

[DPTJKKVH]

Well I can report that disabling legacy encryption breaks my installation, even though the encryption:scan:legacy-format tells me I can disable it.

To be more specific: Encrypted files cannot be downloaded anymore via web GUI and sharing files (e.g. publicly) doesn't work as well.

The GUI tells me that my private key is missing and I should log out and back in. But that doesn't fix it.

Then it tells me to set a new password. When changing paswords it tells me again that my private key is missing and I should log out and back in. Doing that again doesn't fix it.

So neither logging in nor changing passwords works.

Fortunately just enabling legacy encryption fixes the issue and I can access all files again. However this foreshadows doom for when legacy encryption won't be supported anymore.

Either the scanning tool is broken or there is an important step missing after or before disabling legacy encryption.

I find it rather odd that simply disabling legacy encryption in the config is supposed to change encryption schemes. Also the "has been migrated" boolean in config.php is suggesting there might be more to it.

Oh btw: The encryption:migrate-key-storage-format does not run through and produces and error about an expexted boolean value for decrypt.

[thomwiggers]

It's not unlikely that the files need to be migrated from the client side if you're using user key encryption. I only have a single user, and managed to migrate by touch-ing the few affected files. This triggered a re-upload in the client. This does however still leave old versions in files_versions that now show up in the scan tool.

Either the scanning tool is broken or there is an important step missing after or before disabling legacy encryption.

I don't think the tool is broken, there simply isn't any migration path yet.

[DPTJKKVH]

I don't think the tool is broken, there simply isn't any migration path yet.

That might be and it would be nice to have some official input on that.

The main issue I have at this point is that there is a scary orange warning in the admin panel that highly recommends admins to disable legacy encryption asap while at the same time it seems like there isn't a proper way to do that without breaking things.

I know everyone is de facto on leave for Christmas at this point and probably only paying customers get emergency support but one simple "Yea it's broken at the moment. Don't worry, we will fix it in 2021." would be nice or a little hotfix that simply removes the warning so nobody else falls victim to that would go a long way.

I know Nexctloud owes us nothing when we don't pay for support but it would be nice to not let people run into their doom right before Christmas.

[RedKage]

Hmmm I have been using nextcloud for many years, migrated from owncloud 8 or something. So I must have started with nextcloud 10 or 11. I have been using the default encryption module since then, and done web upgrades.

I have seem some posts somewhere about encryption.legacy_format_support, and I set it to false Recently then I upgraded to 20.0.4. Then emptied the trashbin.

Then I ran encryption:scan:legacy-format which took a while, and it told me a had a few files. they were all versions files. I cleaned them up using versions:cleanup

Finally I have ran encryption:migrate-key-storage-format on version 20.0.4. Maintenance mode was activated while doing so Nothing much was showing on stdout, and after about 1 hour it was done. Everything is working properly, @DPTJKKVH I have had no weird things about changing password.

For me all is good, though I'm not sure anything was changed. I did huge backups before doing this, so in theory I could make some diffs to see if some stuff were actually changed... but, we are talking hundreds of gigs here.

[DPTJKKVH]

@RedKage I also run Nextcloud from the day there was a migration path from ownCloud.

What's interesting is that you ran key storage migration in maintenance mode. @Kuphi said it only worked when he disabled maintenance mode.

Also just for more background info: I had some files with bad/missing headers but none of them were versions or in the trash. I just backed them up on my client, removed them and emptied trash. Then I ran the legacy check and it said all is fine.

[Amon-san]

i also had this issue. i "fixed" it by copying the respective files to another directory, deleting the original and then moving the copy back in place. note: using 'move file' in the first place did not yield the same result for me.

[Kuphi]

i also had this issue. i "fixed" it by copying the respective files to another directory, deleting the original and then moving the copy back in place. note: using 'move file' in the first place did not yield the same result for me.

Yes. By copying the files, they will lose all previous file versions and will be newly encrypted.

[Alvarord]

Hello,

I face a similar issue as @DPTJKKVH

Well I can report that disabling legacy encryption breaks my installation, even though the encryption:scan:legacy-format tells me I can disable it.

I have upgraded from 19.0.7.1 to 20.0.5 yesterday

I cleaned the result from:

occ encryption:scan:legacy-format When I deactivate the _'encryption.legacy_formatsupport' => false then I can't access files and I see the #8546 Invalid private key. I tried this without success.

Then I change the value of _'encryption.legacy_formatsupport' => true and I can access back to the data but the message Invalid private key for encryption app. Please update your private key password in your personal settings to recover access to your encrypted files. keeps appearing.

Not sure how can I help to track it.

I migrated from owncloud to nextcloud long ago (not sure in which version).

In the meantime I will do a full backup of my files (now that I can access them) just in case I break anything.

Any ideas about how to proceed?

More information:

root@machine:/var/www/html# sudo -u www-data php occ   encryption:scan:legacy-format
Scanning all files for legacy encryption
Scanning all files for user1
......
All scanned files are properly encrypted. You can disable the legacy compatibility mode.

root@machine:/var/www/html# sudo -u www-data php occ  encryption:status
  - enabled: true
  - defaultModule: OC_DEFAULT_MODULE

root@machine:/var/www/html# sudo -u www-data php occ  config:system:get version
20.0.5.2

Edit: I tried to ran this step: sudo -u www-data ./occ encryption:encrypt-all But I don't have same output as reported by @yahesh

I will try execute it again right after I finish the backup and I will report my result.

[DPTJKKVH]

@Alvarord I ran encrypt-all multiple times and it doesn't fix anything. The issue as we experience it, stays the same.

I can only guess that people like us, who migrated from owncloud to nextcloud years ago, need some command to migrate everything encrypted in legacy mode to the new encryption, possibly including the new key storage format?

But alas two months go by and the developers have not even acknowledged that people have problems.

I fear for the worst when we have to update to v21 and our encryption breaks for good, forcing us to deploy from scratch, setup dozens of users and reupload gigabytes or terabytes worth of data.

[micah]

On 2021-01-19 06:51:47, DPTJKKVH wrote:

I can only guess that people like us, who migrated from owncloud to nextcloud years ago, need some command to migrate everything encrypted in legacy mode to the new encryption, possibly including the new key storage format?

I did not upgrade from owncloud, and I had this problem.

[Alvarord]

Hello @DPTJKKVH ,

Is strange maybe they consider it should be solved with the encrypt-all, or they havent been able to reproduce it.

I saw in the bash history this morning that I executed the command yesterday but it was quite fast and I though it was because it was ciphering just a few old files. Today after doing the backup (luckily for me is only 100GB) I re-executed it and I noticed 2 things

First it finish always with an error:

In Storage.php line 287:

  Could not decrypt key

In Crypto.php line 122:

  Authenticated ciphertext could not be decoded.

@DPTJKKVH do you have the same issue?

And every time I execute it it cipher just a few files less than 1000 files But at every try it is trying different groups of files (normally shared ones).

I will try to gather which file/s produce the issue.

Regards.

[DPTJKKVH]

@Alvarord you can't encrypt your files with encrypt-all because when you disable legacy encryption the server, for some reason, can't decrypt your private key. Try the same with legacy encryption enabled (just switch the "false" flag back to "true" in the config.php) and it will work.

Disabling legacy encryption does seemingly require the private keys to be stored in a different format. However encryption:migrate-key-storage-format interupts with an error. Maybe try this in both legacy and non legacy mode and tell us what your output is.

I think you and me have the same problem.

@micah do you HAVE the problem or HAD the the problem and fixed it?

[micah]

On 2021-01-19 07:27:40, DPTJKKVH wrote:

@micah do you HAVE the problem or HAD the the problem and fixed it?

well, honestly I dont know if I have the problem still. I saw in my administration panel that this was deprecated and I needed to run the encryption:scan:legacy-format command. It produced a number of weird errors and seemed like it failed. I tried moving those files out of the way, like people here said, and re-ran it.

As far as I know, this just 'scanned' for files, and didn't do anything? Will I need to do something more?

If I run this command and I don't get an error, do I no longer have the problem?

When I run it now, it does not fail, and it says:

All scanned files are properly encrypted. You can disable the legacy compatibility mode.

How do I do that?

[Alvarord]

Hello @DPTJKKVH

@Alvarord you can't encrypt your files with encrypt-all because when you disable legacy encryption the server, for some reason, can't decrypt your private key. Try the same with legacy encryption enabled (just switch the "false" flag back to "true" in the config.php) and it will work.

same issue enabling or disabling it. A few files are encrypted and it finish with same error.

Trying right now the second part with legacy active (I think this took yesterday a while). I will try with both values and update you later.

Thanks for the ideas to try.

[DPTJKKVH]

@micah when it runs through without an error you are (in theory) ready to disable legacy encryption as described here:

https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_migration.html

Try that and then log into your normal user account on your nextcloud server and see if it tells you that your private key couldn't be decrypted. If that happens, you have the same issue as we have.

If you don't get any message, try downloading and opening your files via the web gui. If it works: Congratulations, you were lucky. If not, you suffer the same issue as we do.

[DPTJKKVH]

Please report your results when you are done. The more test cases we have, the better. @micah

[micah]

On 2021-01-19 08:09:05, DPTJKKVH wrote:

@micah when it runs through without an error you are (in theory) ready to disable legacy encryption as described here:

https://docs.nextcloud.com/server/20/admin_manual/configuration_files/encryption_migration.html

Try that and then log into your normal user account on your nextcloud server and see if it tells you that your private key couldn't be decrypted. If that happens, you have the same issue as we have.

I set encryption.legacy_format_support to be false and then I logged into nextcloud and it doesn't say anything...

I noticed there is also a setting encryption.key_storage_migrated, this is currently set to false. Do I need to do something with this too?

If you don't get any message, try downloading and opening your files via the web gui. If it works: Congratulations, you were lucky. If not, you suffer the same issue as we do.

I am able to view the files, download and open them.

So I guess I'm lucky!

micah

[DPTJKKVH]

You are lucky then and I am more and more convinced that the errors @Alvarord and I have are due to us both truly coming from a full legacy encryption while most people already used the new encryption but with legacy support enabled.

[yahesh]

As the one who found that vulnerabilities that require those migration steps:

Having problems with the disabled legacy encryption generally means that you probably have files that are encrypted but not integrity-protected. If your attack scenarios do not contain the modification of files then you don't have to fix this right away, but you should tackle this at some later point in time. For now you can set encryption.legacy_format_support to false and everything should work as before. Unless you know what you're doing, it's a bad idea to re-run the encrypt-all command. This will more likely break things instead of fixing them.

As for encryption.key_storage_migrated, there is another vulnerability through which the key material can be replaced by an attacker, allowing them to replace or read files. Do not just set this value, but instead use the ./occ encryption:migrate-key-storage-format command. This will wrap the key material in an additional authenticated-encryption layer.

[DPTJKKVH]

Hello @yahesh ! Really happy to have you step forward and give some context.

As described by @Alvarord and me, we did run occ encryption:scan:legacy-format and got errors about missing headers. Those were fixed by removing the files in question via normal means like moving them out of the sync folder and back into it. After that the command ran without issues and reported that we are ready to disable legacy encryption.

However when doing that in the config.php all files become unavailable through the web gui and a notification tells us that our private key could not be decrypted. There are two variants of this error notice: One tells you to log out and back in, which does not help. The other one tells you to change your password, which does not help either.

If I understand correctly the encryption:migrate-key-storage-format command is addressing a different issue and disabling legacy mode should work without it.

Still if I run this command either in maintenance mode or in production mode the command fails with a stout error. I hope I have it saved somewhere (will post it later) so I don't have to recreate it again and restore but either way this fails as well.

Are you sure that simply disabling legacy mode in the config.php is enough or do we rather have to re-encrypt them to create the signatures for verification or do I miss something here?

Many thanks!

[Alvarord]

Hello, second part:

Disabling legacy encryption does seemingly require the private keys to be stored in a different format. However encryption:migrate-key-storage-format interupts with an error. Maybe try this in both legacy and non legacy mode and tell us what your output is.

I didn't have any issue with the migrate-key-storage-format

root@machine:/var/www/html/# sudo -u www-data php occ encryption:migrate-key-storage-format Updating key storage format Start to update the keys: 4 [============================]Key storage format successfully updated With and without legacy active it works and everytime I try to encrypt all after with or without the legacy it fails after a few hundred files.

Seems that after the new re-run of the encrypt-all today I could login and remove the message about the private key properly (WEIRD maybe I didn't push the yes to accept the the first encryption?)

I was testing the download always with the same file and after the several tries with encrypt-all luckily I could download it without the legacy active and I can see in the traces the following:

encrypt_all_at_13_31: encrypt files for user user1 (1 of 4): /user1/files/path/filetested.pdf Seems the encrypt all make that file work now.

So seems my issue at this time is JUST in the encrypt-all that stops. I will try to make a script to make it run all night to check how many files it encrypt. As @yahesh points maybe is not a good idea (and I have no idea about what Im doing) but I think I have found something and I have backups so maybe it is possible to improve the error on the encrypt-all, when the key fails there is no track about "what" fails, neither it checks the integrity of the information so maybe when you encrypt-all you should advice about integrity issues too.

I have no idea about php but seems could be nice to add more info on this line

https://github.com/nextcloud/server/blob/3cb8d85cf475061bcfbe6d901a9b9a0c29258ab4/lib/private/Encryption/Keys/Storage.php#L287

To check why the key decryption fail and maybe even it could be nice to catch it.

@DPTJKKVH you have problem with all the files? The encrypt all is succesful? Can you retry it?

[yahesh]

@DPTJKKVH As far as I remember there has been another issue in which people had the problem that the official command to scan for legacy encrypted files (./occ encryption:scan:legacy-format) would not find all occurences. I suggested to write a third-party script for that but have not yet come around to actually implementing it.

When the whole web UI breaks for you when disabling the legacy encryption this means that your private key is secured using the legacy encryption as well. Unfortunately, as far as I know, there is currently no built-in command to re-encrypt everything.

You understood correctly that the ./occ encryption:migrate-key-storage-format command fixes a different vulnerability.

[DPTJKKVH]

@yahesh thank you so much for clarifying the part with the legacy encrypted private key. I am certain this must be my issue, probably in combination with not all legacy files being correctly detected.

@yahesh I pinged "the usual suspects" over a month ago but nobody seemed to be in inclined to even acknowledge the issue. If you have any way to get @schiessle @rullzer @nickvergessen or someone else's attention for this, it would be greatly appreciated. I do not use NC commercially but I run it for friends and family to keep them away from Dropbox, Google and so on (and I try to introduce it into the company I work for). It would be an atrocious time for everyone if I have to deploy a fresh Nextcloud instance, recreate all accounts, have everyone reupload their files (after having them back it up unencrypted locally) and everyone having to create dozens of new app tokens for their different devices.

I am thankful if you did build some unofficial tools to fix this but ultimately the team should create an official migration path, even if that path incorporates your code.

[yahesh]

@DPTJKKVH Do you know if you use user-based encryption keys? I would assume so because the master key encryption has been introduced much later. It might be almost sufficient if all users changed their passwords. This would force Nextcloud to rewrite the private key files. (Addition: Unfortunately, there's also an additional key for public sharing which probably cannot be forced to be rewritten that easily. Installations with the master key encryption are out of luck as well.)

Concering the official command not finding all legacy encrypted file, my current guess is that those legacy encrypted private keys are the ones that aren't considered during the scan.

[DPTJKKVH]

@yahesh I do use user-based encryption and not by accident. I don't want to be able to decrypt user files even if I wanted to. :)

I already tested changing passwords both on my personal account, the admin account (I have those seperated) and had another user change their password. In all instances the error that the private key could not be decrypted and the files being either unavailable or downloaded in encrypted state remained.

[DPTJKKVH]

Or should everyone change their password before disabling legacy mode and then I go through the motions of scanning etc.?

[yahesh]

@DPTJKKVH My assumption is that only the private key of the logged-in user will be decrypted. But those logic paths are outside of what I have poked around yet.

[DPTJKKVH]

@yahesh but I did understand you correctly: If changing passwords helps it must be done before disabling legacy encryption by every user, right? Because this is something I would love to try but I want to make sure I did not misunderstand you. ;)

Of course you assume no responsibility either way.

[yahesh]

@DPTJKKVH: Yes, each user would have to change their password once before disabling the legacy encryption. As the private keys are protected with the user passwords, they have to be rewritten when the password is changed. Unfortunately, I think this might not be enough, though, as the pubShare private key cannot be rewritten that easily.

[DPTJKKVH]

@yahesh is the pubShare key only for public shares (as the name suggests) or also used for shares between users of the same server?

[yahesh]

@DPTJKKVH They are only used for public shares. Sharing between users is done using the receiving user's public key.

[DPTJKKVH]

@yahesh many thanks for taking the time to explain and answer my questions. I will tell everyone to prepare for another try.

I will report back with results.

[Alvarord]

Hello,

I want to point one important section in my previous comment:

I was testing the download always with the same file and after the several tries with encrypt-all luckily I could download it without the legacy active and I can see in the traces the following:

encrypt_all_at_13_31: encrypt files for user user1 (1 of 4): /user1/files/path/filetested.pdf Seems the encrypt all make that file work now.

This means several things:

• The encryption:scan:legacy-format command is failing. Because it is not scanning all files properly or at least it is not detecting an issue in the file I mentioned.
• The encrypt-all command SHOULD be launched after the scan is successful and review integrity issues as mentioned in case #8546

This is really dangerous because what I can see is that some files are properly accessible deactivating the legacy but some are not. This means that if the administrator trust the scan result it can appear that all files are correct without being true. Because maybe not all files had been encrypted properly.

As mentioned in #8546 my installation can be used at this time for experimentation if needed.

[DPTJKKVH]

Hello @yahesh . I followed your advice and encountered a sadly familiar error. I have a stack trace and I will explain step by step what I did so it is hopefully useful to you, the team and me. I want to thank you in advance for your help! You don't seem to have a donation option. Do you want me to donate to some cause you like as a small thank you for the time you put into helping me?

Ubuntu 18.04 Nextcloud 20.0.5 Encryption: User keys without master key Nextcloud is NOT in maintenance-mode

Every user on my friends and family Nextcloud has changed their passwords while legacy encryption was still enabled and the key storage format was not migrated. So everyone's keys should be using the new encryption method now.

Then I ran encryption:scan:legacy-format which produced the following output. (I redacted the user names for their privacy.)

sudo -u www-data php /var/www/nextcloud/occ encryption:scan:legacy-format Scanning all files for legacy encryption Scanning all files for A Scanning all files for B Scanning all files for C Scanning all files for D Scanning all files for E Scanning all files for F Scanning all files for G Scanning all files for H All scanned files are properly encrypted. You can disable the legacy compatibility mode.

I decided to migrate the key format first before disabling legacy encryption because the migration command produced an error message last time I tried. Unfortunately it did again. Here is the full output with only one user name redacted to A. The A corresponds to the user A above btw.

sudo -u www-data php /var/www/nextcloud/occ encryption:migrate-key-storage-format Updating key storage format Start to update the keys: 0 [>---------------------------]An unhandled exception has been thrown: TypeError: Argument 1 passed to OC\Security\Crypto::decrypt() must be of the type string, boolean given, called in /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php on line 187 and defined in /var/www/nextcloud/lib/private/Security/Crypto.php:113 Stack trace:

0 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(187): OC\Security\Crypto->decrypt(false)

1 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(167): OC\Core\Command\Encryption\MigrateKeyStorage->traverseFileKeys('/A/files_en...')

2 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(167): OC\Core\Command\Encryption\MigrateKeyStorage->traverseFileKeys('/A/files_en...')

3 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(167): OC\Core\Command\Encryption\MigrateKeyStorage->traverseFileKeys('/A/files_en...')

4 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(167): OC\Core\Command\Encryption\MigrateKeyStorage->traverseFileKeys('/A/files_en...')

5 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(167): OC\Core\Command\Encryption\MigrateKeyStorage->traverseFileKeys('/A/files_en...')

6 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(258): OC\Core\Command\Encryption\MigrateKeyStorage->traverseFileKeys('/A/files_en...')

7 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(234): OC\Core\Command\Encryption\MigrateKeyStorage->updateUserKeys('', 'A')

8 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(101): OC\Core\Command\Encryption\MigrateKeyStorage->updateUsersKeys('', Object(Symfony\Component\Console\Output\ConsoleOutput))

9 /var/www/nextcloud/core/Command/Encryption/MigrateKeyStorage.php(83): OC\Core\Command\Encryption\MigrateKeyStorage->updateKeys('', Object(Symfony\Component\Console\Output\ConsoleOutput))

10 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OC\Core\Command\Encryption\MigrateKeyStorage->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

11 /var/www/nextcloud/3rdparty/symfony/console/Application.php(1000): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

12 /var/www/nextcloud/3rdparty/symfony/console/Application.php(271): Symfony\Component\Console\Application->doRunCommand(Object(OC\Core\Command\Encryption\MigrateKeyStorage), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

13 /var/www/nextcloud/3rdparty/symfony/console/Application.php(147): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

14 /var/www/nextcloud/lib/private/Console/Application.php(215): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

15 /var/www/nextcloud/console.php(100): OC\Console\Application->run()

16 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')

I hope you know what is causing this error because even after looking into Crypto.php and MigrateKeyStorage.php I do not understand what goes wrong.

Many thanks in advance!

[DPTJKKVH]

To rule out any file modifications here are two Gist of both my MigrateKeyStorage.php and Crypto.php. (I did not meddle with those in any way.)

MigrateKeyStorage.php: https://gist.github.com/DPTJKKVH/88b203732550221740871a7eddb69e11

Crypto.php: https://gist.github.com/DPTJKKVH/12504d857e215cf0beafc87100051946

[yahesh]

@DPTJKKVH I had a look at your stacktrace and I'd think that it shouldn't interrupt the execution of the key migration. When looking at MigrateKeyStorage.php#L187 one can see that the migration checks whether a key file has already been migrated by trying to decrypt it. When the decryption succeeds, the migration process just skips the file. When the decryption fails, the corresponding Exception is cought and the migration process encrypts the key file.

[DPTJKKVH]

Hey @yahesh . Thanks for looking into it!

So this is a whole new bug and I should probably open a new bug report, right?

[yahesh]

I'm not yet sure. Looking at the trace some more I stumbled over the Argument 1 passed to OC\Security\Crypto::decrypt() must be of the type string, boolean given message. It seems as if the call to file_get_contents() in MigrateKeyStorage.php#L184 fails and returns a boolean instead of the contents of the file. I assume this is not meant to happen.

[DPTJKKVH]

@yahesh I noticed that as well (see the first version of my post with the stacktrace) but edited it out because I didn't want to jump to conclusions.

Is there anything I can do to help debug this?