Foundry376 / Mailspring

:love_letter: A beautiful, fast and fully open source mail client for Mac, Windows and Linux.
https://getmailspring.com/
GNU General Public License v3.0
15.55k stars 907 forks source link

[SNAP] Update to gnome extension and core22 #2464

Closed soumyaDghosh closed 1 year ago

3v1n0 commented 1 year ago

Yes, please.

soumyaDghosh commented 1 year ago

I am getting no way to test this snap also.

soumyaDghosh commented 1 year ago

No problem, this is initial, and I am always there to help.

bengotow commented 1 year ago

Hi @soumyaDghosh and @3v1n0, I merged this PR and ran a new Snapcraft build.

The logs are here: https://app.travis-ci.com/github/Foundry376/Mailspring/jobs/611277919

The build should be available as the edge release of the Mailspring snap. (Revision 524). If you get a moment, could you try doing a snap refresh mailspring --edge (I think that's the command) and verify that it works for you, and then we'll ship it as part of the next release?

Thank you!

soumyaDghosh commented 1 year ago

@bengotow On my first glance, I can see this as an issue

image

and, another problem is

Error: Could not call remote method 'decryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.
    at /snap/mailspring/524/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71
    at IpcMainImpl.<anonymous> (/snap/mailspring/524/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:390:28)
    at IpcMainImpl.emit (node:domain:475:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)
    at EventEmitter.emit (node:events:390:28)
    at EventEmitter.emit (node:domain:475:12)

    at /snap/mailspring/524/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:468:25
    at IpcMainImpl.<anonymous> (/snap/mailspring/524/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:390:28)
    at IpcMainImpl.emit (node:domain:475:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)
    at EventEmitter.emit (node:events:390:28)
    at EventEmitter.emit (node:domain:475:12) {
  message: "Recovered Error: Could not call remote method 'decryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.\n" +
    '    at /snap/mailspring/524/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71\n' +
    '    at IpcMainImpl.<anonymous> (/snap/mailspring/524/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)\n' +
    '    at IpcMainImpl.emit (node:events:390:28)\n' +
    '    at IpcMainImpl.emit (node:domain:475:12)\n' +
    '    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)\n' +
    '    at EventEmitter.emit (node:events:390:28)\n' +
    '    at EventEmitter.emit (node:domain:475:12)\n'
} { pluginIds: [] }
(node:11568) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
(Use `mailspring --trace-deprecation ...` to show where the warning was created)

I am probably stuck on this screen

image

I checked the snappy-debug result also, and this seems to be a code issue, rather than snap confinement issue.

Phylu commented 1 year ago

The error seems to be related to the efforts of removing keytar in combination with electron/remote (https://github.com/Foundry376/Mailspring/pull/2471). I will see if I can reproduce (and hopefully fix) this.

bengotow commented 1 year ago

Thanks for investigating @Phylu - your new solution worked nicely for me on macOS. It seems like there's some discussion here (https://github.com/electron/electron/issues/32598). I think we're waiting for the ready event already, but one thing you could try is moving require('@electron/remote/main').initialize(); down inside the ready event handler in main.js. (Maybe? sort of a random guess...)

Let me know if you're able to reproduce! I can also try to get a Linux box going but my mac has an ARM chip now which has made it harder to test 🤦‍♂️

Phylu commented 1 year ago

Thanks for investigating @Phylu - your new solution worked nicely for me on macOS.

I tested this on Linux, Mac and Windows and it worked for me. I assume, that the lifecycle and/or access to the os dependencies is different from within a snap then from outside. So probably an issue with the libsecret binding or something similar. I will let you know once I figure out what is the issue.

As I have not created any snap release yet: Is there an easy way to build a snap locally and install it? I might need this for testing.

bengotow commented 1 year ago

Hey @Phylu, hmm I think that if you install the snapcraft tool on your machine, you can run npm run build to create a normal Mailspring .deb file, and then /snap/bin/snapcraft --use-lxd should package it up into a snap? It's been a while since I've done it, but I think the snapcraft command line tool finds the snap/snapcraft.yaml file by itself and uses that + the deb to create a snap.

3v1n0 commented 1 year ago

You should check what are the dmesg apparmor messages when running if that's a permission issue.

Also, if this is using libsecret, it may need to add the plug password-manager-service, which requires manual connection though (cfr)

3v1n0 commented 1 year ago

Ah, however libsecret should just use a local keyfile storage when running under snap, so in theory all should work... Just keeping things locally.

Unless something like https://github.com/3v1n0/matterhorn-snap/commit/6667e9ce06f0c68fceccc6cda857374cf629b80d is done

Phylu commented 1 year ago

I built Mailspring on the latest master, packaged it as a snap and installed it. When I connect it to the gnome-keyring it works in certain cases: sudo snap connect mailspring:password-manager-service

See my log output:

➜  Mailspring git:(master) ✗ sudo snap install mailspring_1.12.0_amd64.snap --dangerous 
mailspring 1.12.0 installiert
➜  Mailspring git:(master) ✗ mailspring 
...
Error: Could not call remote method 'encryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString. Encryption is not available.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString. Encryption is not available.
    at /snap/mailspring/x1/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71
    at IpcMainImpl.<anonymous> (/snap/mailspring/x1/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:390:28)
    at IpcMainImpl.emit (node:domain:475:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)
    at EventEmitter.emit (node:events:390:28)
    at EventEmitter.emit (node:domain:475:12)

    at /snap/mailspring/x1/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:468:25
    at IpcMainImpl.<anonymous> (/snap/mailspring/x1/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:390:28)
    at IpcMainImpl.emit (node:domain:475:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)
    at EventEmitter.emit (node:events:390:28)
    at EventEmitter.emit (node:domain:475:12) {
  message: "Recovered Error: Could not call remote method 'encryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString. Encryption is not available.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString. Encryption is not available.\n" +
    '    at /snap/mailspring/x1/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71\n' +
    '    at IpcMainImpl.<anonymous> (/snap/mailspring/x1/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)\n' +
    '    at IpcMainImpl.emit (node:events:390:28)\n' +
    '    at IpcMainImpl.emit (node:domain:475:12)\n' +
    '    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)\n' +
    '    at EventEmitter.emit (node:events:390:28)\n' +
    '    at EventEmitter.emit (node:domain:475:12)\n'
} { pluginIds: [] }
...
➜  Mailspring git:(master) ✗ sudo snap connect mailspring:password-manager-service      
➜  Mailspring git:(master) ✗ mailspring                                           
Running database migrations
App load time: 176ms

{"error":null}

However, this only works if I run mailspring at least once before running the sudo snap connect mailspring:password-manager-service command. If I run the command before ever running mailspring, or when I install the snap from the edge channel, and run it (no difference whether I connect it to the password-manager-service or not), I get a different error message:

➜  Mailspring git:(master) ✗ mailspring                                           
...
TypeError: Cannot set properties of null (setting 'testaccount@gmail.com-imap')
    at KeyManager.extractAndStoreAccountSecrets (file:///tmp/nylas-build/electron-packager/linux-x64/mailspring-linux-x64/resources/app/src/key-manager.ts:64:43)

So, honor me completely confused now...

I think we're waiting for the ready event already, but one thing you could try is moving require('@electron/remote/main').initialize(); down inside the ready event handler in main.js. (Maybe? sort of a random guess...)

@bengotow Moving the remote initialization does not seem to have any effect.

There is one nice thing however: When I am able to connect an e-mail account and the full UI is loaded, the tray icon is updated and shown correctly.

Phylu commented 1 year ago

Ok, some more debugging led me to the following conclusions:

I will need to debug the failing key migration further.

Phylu commented 1 year ago

I just confirmed that my fix in #2472 is working not only within my snap test environment, but also with the .deb package on my production mailspring. I will do a few more tests on MacOS and Windows, but I hope that this is resolved now. :)

Bildschirmfoto vom 2023-10-11 17-42-33

bengotow commented 1 year ago

Hey folks, I published a new snap (rev 525) to edge. Want to see if that resolves the issue for you as well @3v1n0, @soumyaDghosh? I wish we could automatically attach the password-manager-service or avoid that step, but I think that we do want to store email passwords to the login keychain so it does sorta make sense!

soumyaDghosh commented 1 year ago

image

I have the slot connected, still getting this,

Screenshot from 2023-10-12 23-26-39

The tray icon is also very bad. Also, a suggestion, I am thinking to give a try and run it under wayland.

soumyaDghosh commented 1 year ago

I wish we could automatically attach the password-manager-service or avoid that step, but I think that we do want to store email passwords to the login keychain so it does sorta make sense!

Make a auto connect request in the forum.

Phylu commented 1 year ago

@soumyaDghosh

I have the slot connected, still getting this

So you see some logs in the console? Are you using a Mailspring ID?

The tray icon is also very bad. Also, a suggestion, I am thinking to give a try and run it under wayland.

Can you adjust the snap accordingly?

Make a auto connect request in the forum.

Everything that I have read is, that there is no way to auto connect for security reasons. Do you have a link here?

soumyaDghosh commented 1 year ago

So you see some logs in the console? Are you using a Mailspring ID?

I see, and I made a mailspring id once, but I was never able reuse it, so, I skipped it this time.

Can you adjust the snap accordingly?

I'll but maybe later. Currently in a vaccation for Durga Puja.

Everything that I have read is, that there is no way to auto connect for security reasons. Do you have a link here?

Actually, yes it is. I have also faced the same thing. The problem here the dialog that comes up isn't selectable. I can't copy that command from there. So, I think better to make it selectable atleast.

3v1n0 commented 1 year ago

I wish we could automatically attach the password-manager-service or avoid that step, but I think that we do want to store email passwords to the login keychain so it does sorta make sense!

So... No :) As mentioned auto-connection of such interface won't ever happen, as it's dangerous.

When running confined libsecret should use the file backend: https://github.com/GNOME/libsecret/blob/115474aa6762da/libsecret/secret-file-backend.c#L580-L587

So, in theory it should just create local files into the ~/snap/mailsrping folder and keep the data stored there instead of storing it in the shared user keys storage.

As per this it's not really clear to me why not connecting the interface does not work.

3v1n0 commented 1 year ago

Ah, on that dialog, you can avoid saying to install things where on snap, or non-snap documentation by checking the existance of the SNAP env variable.

Phylu commented 1 year ago

So, in theory it should just create local files into the ~/snap/mailsrping folder and keep the data stored there instead of storing it in the shared user keys storage.

As per this it's not really clear to me why not connecting the interface does not work.

@3v1n0 I have no idea, why it did not work in my tests without connecting the snap. If you have any ideas of things that I can/should try, I am open for suggestions. :)

3v1n0 commented 1 year ago

Mh, that's werid because using the secret tool inside the snap it indeed can save things properly:

marco@tricky:~$ secret-tool store --label="fooo" bar baz
Password: 
marco@tricky:~$ secret-tool lookup bar baz
A password
marco@tricky:~$ ls $SNAP_USER_DATA/.local/share/keyrings/ -lht
total 8.0K
-rw------- 1 marco marco 772 Oct 17 22:27 default.keyring
-rw------- 1 marco marco 772 Oct 17 22:25 default.keyring~
marco@tricky:~$ date
Tue Oct 17 22:27:22 CEST 2023
marco@tricky:~$ 

However, as per the safe-storage docs, have you tried forcing running mailspring with --password-store="gnome-libsecret" or even with --password-store=basic? as given the data is all inside a confinement there's no much need to go through libsecret.

I can't test it myself now as per the new dialog that appears even when I've mailspring:password-manager-service connected (pleas use snapctl is-connected password-manager-service to check it).

Phylu commented 1 year ago

Unfortunately, I was not able to get this running without connecting mailspring to the password-manager-service. I have no idea where the problem is, but I just can't get the safeStorage from electron running in this case.

I am now working on cleaning up the migration to ensure that it can be recovered and retried if it fails due to the snap version being started without the access after the upgrade.

See: https://github.com/Foundry376/Mailspring/pull/2474

bengotow commented 1 year ago

Hey folks, those two new PRs have been merged and I've built Snap rev 526 -- this is our release candidate for 1.12. I'm hopeful that the in-snap default safe-storage will work for other folks, but if they encounter the error, @Phylu has made the error message and retry flow pretty smooth so they're able to enable the password-manager-service connection and try again.

Phylu commented 1 year ago

Awesome. I will try to install the edge snap from snapcraft and let you know if it still works.

soumyaDghosh commented 1 year ago

@bengotow Two things to mention

  1. Users who didn't connect the password-service-manager plug and opened the app, need to clear their data, for the app to work.
  2. The appindicator is still just a black square in my ubuntu 23.10
bengotow commented 1 year ago

Hey @soumyaDghosh - hmm, I think that #1 may be ok since users will be going straight from the old version to this new one? I think it's moving from the previous 1.12 draft release to this 1.12 draft release that requires clearing data? I will run the upgrade flow on another machine tonight and see.

The appindicator issue is interesting, it seems like that must be either a snap container issue or something related to #2433... did you notice if that was only in the snap version? Will see if I can repro.

Phylu commented 1 year ago

The tray icon issue only happened for me while Mailspring was in a "dangling" state, before any mail accounts were set up. As soon as I configured any mail account, the correct icon appears.

I don't have the issue when using the .deb version or running the development build. I assume that somehow the snap reserves the icon space with a black placeholder before the correct icon is shown.

As this is unrelated to the keytar issue and only affects the snap (where I have very limited knowledge), I leave this task for someone else. ;-)

But I think, that we might also just live with it for now, as it disappears after the initial setup.

soumyaDghosh commented 1 year ago

Also, why is the autostart option not there in the snap package. Can't install the deb though in my 23.10 due to dependency issues.

Phylu commented 1 year ago

I just tried the snap upgrade path, and the migration that worked during all my tests is now broken again. I don't get it... :( Line https://github.com/Foundry376/Mailspring/blob/master/app/src/key-manager.ts#L33 does now return null so the passwords are not migrated properly:

keytar.getPassword(SERVICE_NAME, KEY_NAME).then(raw => {

And this is the same code that worked in the version before in the _getKeyHash() method. I am really puzzled about this behaviour now.

@soumyaDghosh The .deb will be installable again with the 1.12 release. See: https://github.com/Foundry376/Mailspring/pull/2473

Phylu commented 1 year ago

The tray icon issue only happened for me while Mailspring was in a "dangling" state, before any mail accounts were set up. As soon as I configured any mail account, the correct icon appears.

I checked this again as well and it seems that the icon in the 1.12 snap is not displaying the transparent background of the icons correctly. I don't think that it is an issue with the icons as the empty icon is black as well before the actual icon is set up. @soumyaDghosh Do you have any idea how to solve this as this seems to be a different issue with the update to the core22 based snap.

soumyaDghosh commented 1 year ago

keytar.getPassword(SERVICE_NAME, KEY_NAME).then(raw => {

Getting the same issue, even with normal installation


{"error":null}
[8575:1024/120440.805471:ERROR:browser_main_loop.cc(267)] GLib-GObject: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
Error: Could not call remote method 'decryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.
    at /snap/mailspring/526/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71
    at IpcMainImpl.<anonymous> (/snap/mailspring/526/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:390:28)
    at IpcMainImpl.emit (node:domain:475:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)
    at EventEmitter.emit (node:events:390:28)
    at EventEmitter.emit (node:domain:475:12)

    at /snap/mailspring/526/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:468:25
    at IpcMainImpl.<anonymous> (/snap/mailspring/526/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:390:28)
    at IpcMainImpl.emit (node:domain:475:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)
    at EventEmitter.emit (node:events:390:28)
    at EventEmitter.emit (node:domain:475:12) {
  message: "Recovered Error: Could not call remote method 'decryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.\n" +
    '    at /snap/mailspring/526/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71\n' +
    '    at IpcMainImpl.<anonymous> (/snap/mailspring/526/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)\n' +
    '    at IpcMainImpl.emit (node:events:390:28)\n' +
    '    at IpcMainImpl.emit (node:domain:475:12)\n' +
    '    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:161:10935)\n' +
    '    at EventEmitter.emit (node:events:390:28)\n' +
    '    at EventEmitter.emit (node:domain:475:12)\n'
} { pluginIds: [] }
(node:8575) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
soumyaDghosh commented 1 year ago

@soumyaDghosh Do you have any idea how to solve this as this seems to be a different issue with the update to the core22 based snap.

With update to 23.10 for some reason my appindicators are any ways properly not working, and appindicator related issues are addressed with many electron snaps. Some of those are addressed like this

https://github.com/Foundry376/Mailspring/blob/master/snap/snapcraft.yaml#L55

EDIT: getting no option for launch with system start and close with app closes without disabling the appindicator.

Phylu commented 1 year ago

@soumyaDghosh In your initial tests, were you able to still connect to your secrets stored via Keytar? When I am building the snap with the original implementation of the key-manager.ts, I still can't access the secrets stored by the operating system.

Obviously, if keytar can't connect the migration won't work and the new secrets can't be written. @bengotow What do you think about switching to the following upgrade path?

  1. Change from Keytar to safeStorage (basically reverting this MR).
  2. Build a new snap when the keys are already migrated (we would need to test this again then)
  3. Upgrade the electron version

This means a bit more effort and more versions, but I think it will be faster than looking for another solution now. Unfortunately, I cannot try this out as I am unable to build an old snap that works with the custom build better-sqlite from my host machine.

The alternative would be that the upgrade works on all environments nicely, but if you are on snap, you will need to do the following to prevent errors:

  1. If you use a Mailspring ID log out (and log in again)
  2. Reconnect all mail accounts

Step 2 will hardly fail if a Mailspring ID exists, but its secret was not migrated.

Phylu commented 1 year ago

@systemizer

  1. The appindicator is still just a black square in my ubuntu 23.10

I just saw, that another snap (multipass) had the same problem regarding the appindicator icon having a black background. Maybe this is a bigger snap issue and not really related to mailspring?

Bildschirmfoto vom 2023-10-25 23-29-28

Phylu commented 1 year ago

@bengotow What is your stake in getting the upgrade done?

I think we can either try to do a 3 step upgrade path now (Remove Keytar, Upgrade Snap, Upgrade Electron):

  1. Change from Keytar to safeStorage (basically reverting this MR).
  2. Build a new snap when the keys are already migrated (we would need to test this again then)
  3. Upgrade the electron version

Or we just ship it as it is now with the effect that some people may need to log out and log in again in their Mailspring ID and then reconnect all e-mail accounts afterwards.

bengotow commented 1 year ago

Hey gang -- I tested this a bit more over the weekend on Ubuntu 18, upgrading from the current snap to the edge snap in this thread. The keychain migration actually worked for me, and on the first try! 🙌 As far as I know I didn't have any ports connected beyond the defaults. It's possible that using an earlier edge release and THEN the final edge release created some weird scenarios above, or that Ubuntu 23 doesn't work and Ubuntu 18 does.

I think the worst case is that some users have to sudo snap connect mailspring:password-manager-service, or relink their accounts, but it seems it /should/ work(and did work for me), so it may be a low % of people.

The app indicator was also broken for me on Ubuntu 18 + snap, but in a different way -- it was totally missing, and I couldn't get it to appear.

Our plan with this release is to have everyone upgrade to the modern safeStorage, and then ship a follow-up release moving the entire app to the latest version of Electron. My guess is that the appindicator will either be magically fixed (🤞) or broken in different ways, so I vote we allow it to render poorly in this release for snap users, and QA + fix in a week or two when we're shipping the Electron 22 release. We can have our auto-update system move non-snap users through the releases in two steps (ensuring they hit this one and migrate their keys), so we shouldn't need to wait more than ~2 weeks, and primarily for the snap users to upgrade.

Thanks @Phylu for all the hard work on this -- I think our lives (With this and with lots of other things) will be much better when we're tracking Electron again, so I want to push through and ship this step :-)

bengotow commented 11 months ago

Hey folks,

If you've updated to the latest release, it'd be interesting to upgrade to the new Electron 22 version and see if that resolves the app indicator issues. You can find those builds below:

Ben

Windows:

https://mailspring-builds.s3.amazonaws.com/client/89078ce5/win-x64/MailspringSetup.exe

Mac:

Intel: https://mailspring-builds.s3.amazonaws.com/client/390a301d/osx/Mailspring.zip
Apple Silicon: https://mailspring-builds.s3.amazonaws.com/client/390a301d/osx/Mailspring-AppleSilicon.zip

Linux:

snap refresh mailspring --edge (rev 528)
https://mailspring-builds.s3.amazonaws.com/client/89078ce5/linux/mailspring-1.13.0-0.1.x86_64.rpm
https://mailspring-builds.s3.amazonaws.com/client/89078ce5/linux/mailspring-1.13.0-amd64.deb
soumyaDghosh commented 11 months ago

@bengotow I updated to the latest edge. and it went very well. I just want to know that why don't we get the open on startup feature in snaps

image

foundry376-bot commented 11 months ago

This pull request has been mentioned on Mailspring Community. There might be relevant details there:

https://community.getmailspring.com/t/password-management-error-after-the-latest-upgrade/7298/15

3v1n0 commented 11 months ago

Both with stable and edge version I'm basically unable to use mailspring, despite having the safe storage connected (that is per se a bad thing, as mentioned already above):

{"error":null}
Error: Could not call remote method 'decryptString'. Check that the method signature is correct. Underlying error: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.Underlying stack: Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.
    at /snap/mailspring/528/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71
    at IpcMainImpl.<anonymous> (/snap/mailspring/528/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:513:28)
    at IpcMainImpl.emit (node:domain:489:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:2:82058)
    at EventEmitter.emit (node:events:513:28)
    at EventEmitter.emit (node:domain:489:12)

    at /snap/mailspring/528/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:468:25
    at IpcMainImpl.<anonymous> (/snap/mailspring/528/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)
    at IpcMainImpl.emit (node:events:513:28)
    at IpcMainImpl.emit (node:domain:489:12)
    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:2:82058)
    at EventEmitter.emit (node:events:513:28)
    at EventEmitter.emit (node:domain:489:12) {
  cause: {
    stack: 'Error: Error while decrypting the ciphertext provided to safeStorage.decryptString.\n' +
      '    at /snap/mailspring/528/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:465:71\n' +
      '    at IpcMainImpl.<anonymous> (/snap/mailspring/528/usr/share/mailspring/resources/app.asar/node_modules/@electron/remote/dist/src/main/server.js:323:27)\n' +
      '    at IpcMainImpl.emit (node:events:513:28)\n' +
      '    at IpcMainImpl.emit (node:domain:489:12)\n' +
      '    at EventEmitter.<anonymous> (node:electron/js2c/browser_init:2:82058)\n' +
      '    at EventEmitter.emit (node:events:513:28)\n' +
      '    at EventEmitter.emit (node:domain:489:12)',
    message: 'Error while decrypting the ciphertext provided to safeStorage.decryptString.'
  }
} { pluginIds: [] }
(node:206556) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
(Use `mailspring --trace-deprecation ...` to show where the warning was created)
Raven: 429 - undefined

Sometimes it starts with the error dialog, sometimes it can't sync and once I try to reconnect it fails while writing the tokens with a similar error.

After some retries it randomly worked, but once I restart it, it fails again.

soumyaDghosh commented 11 months ago

@3v1n0 yup.. same here.

bengotow commented 11 months ago

Hey folks, interesting - the edge release isn't able to consistently save your passwords, even after an uninstall + reinstall of the snap? (I think it was mentioned earlier, but if you downloaded one of the early test builds in this thread, and the migration failed or had issues, it might be worth clearing all the app's data)

It definitely sounds like https://github.com/electron/electron/issues/32598 is relevant. The comments there suggest that this can happen if you're running the app from different user profiles. (eg: sudo mailspring one time and then non-sudo mailspring, or something odd like that...) I wonder if the snap setup causes that to happen in some odd way.

3v1n0 commented 11 months ago

So... Indeed more than the snap confinement itself the presence of a SNAP env is something that leads different behavior.

In fact if you run inside the snap as snap run --shell mailspring -c 'mkdir -p /tmp/HOME && env SECRET_BACKEND=file HOME=/tmp/HOME XDG_CONFIG_HOME=/tmp/HOME/config env -u SNAP $SNAP/usr/bin/mailspring --no-sandbox --password-store=gnome-libsecret' so bascailly launching mailspring without the SNAP variable set, it works also on multiple runs.

Note that this work also without secret service connection as I was saying already. As this should go through xdg portal, if any.

In fact, the current snap isn't using the host secret service at all from what I see, as nothing gets saved there, while it gets properly confined.

Another note, --password-store=gnome-libsecret (or the selected storage) should always be used inside the snap, otherwise electron will have different behavior if the snap is used in KDE or other DEs, while here we should consistently save the data in a snap-local database.

Phylu commented 11 months ago

In difference to the previous behaviour, you won't see any stored credentials in the keyring. The keyring is only used to encrypt the mail passwords before they are stored in the config.json file in the Mailspring settings directory.

This only means a change compared to the earlier versions. Where we needed the gnome keyring for keytar to have a backend storage, on KDE now kwallet can be used to get an encryption key instead. No more need for gnome dependencies on KDE.

foundry376-bot commented 11 months ago

This pull request has been mentioned on Mailspring Community. There might be relevant details there:

https://community.getmailspring.com/t/password-management-error-after-the-latest-upgrade/7298/27

bengotow commented 11 months ago

Hey folks,

Just wanted to follow-up here as well as in Discourse. We've released Mailspring 1.13.0 today, which addresses issues with the snapcraft build of 1.12.0. I was able to reproduce the issues reported above, where the snap was able to write your passwords but not read them after a restart, causing it to succeed on the first run and then get into an invalid + broken state after quitting + restarting.

I fixed it by copying the snapcraft.yaml steps from the chromium snap, which I observed was reading / writing from the keychain successfully! They were building libsecret into the snap, which did the trick for us as well.

If you snap refresh mailspring, you'll get the new revision which addresses these issues. If your Mailspring install is broken, you'll launch into a bit of a mess and I'm really sorry for the hassle. You'll need to:

Thankfully, with this upgrade Mailspring is running on Electron 22 and the new safeStorage APIs, which gives us a clear path forward and makes it much easier to incorporate upstream security patches and updates. I appreciate everyone's patience and I'm sorry this has been a rocky update!

soumyaDghosh commented 11 months ago

@bengotow Mailspring snap now uses gnome-42-2204 content snap. And that content snap already builds libsecret from source. So, I'll look into this. Why it's not able to look for it.

3v1n0 commented 11 months ago

@bengotow Mailspring snap now uses gnome-42-2204 content snap. And that content snap already builds libsecret from source. So, I'll look into this. Why it's not able to look for it.

Exactly... In fact I wanted to suggest to include libsecret in the snap too, but then I noticed that depending on gnome that's not needed at all since it should use the same.

Also the one provided in the gnome snap should behave the same, unless I'm missing something... mmhm