search

GrapheneOS / os-issue-tracker

Issue tracker for GrapheneOS Android Open Source Project hardening work. Standalone projects like Auditor, AttestationServer and hardened_malloc have their own dedicated trackers.
https://grapheneos.org/
357 stars 21 forks source link

regression: The update to 20240104000 breaks multi-profile SMS #3022

Closed KJ7LNW closed 9 months ago

KJ7LNW commented 9 months ago

Hello,

I setup GrapheneOS on my wife's phone and she uses a secondary profile for everything. Everything has been working great until a day or two ago when her secondary profile no longer received SMS messages. Only the admin profile shows SMS messages. This release includes a rebase upon AOSP; if this really is a regression, then maybe that is the cause? But we need to verify if it is or is not a regression:

Here is the configuration, please see my question at the bottom so we can test further:

I would like to verify that it is indeed a regression in 20240104000, but I'm not sure how to roll back. LineageOS used to have a tool for selecting your ROM version.

Question: Is there a way to roll back to 2023120800 without blowing away the current configuration?

Thanks for your help!

thestinger commented 9 months ago

We have no other reports of this and many other people heavily use secondary users. You need to make sure to check the configuration including multiple users configuration.

I would like to verify that it is indeed a regression in 20240104000, but I'm not sure how to roll back. LineageOS used to have a tool for selecting your ROM version.

That would be both broken and not secure...

Question: Is there a way to roll back to 2023120800 without blowing away the current configuration?

No, and having the January security update is important.

KJ7LNW commented 9 months ago

We have no other reports of this and many other people heavily use secondary users. You need to make sure to check the configuration including multiple users configuration.

I have, for an hour today permuting different settings. I even tried wiping the Messaging app's data. All I can see for now is that it worked a couple of days ago, and it seems to have broken after the update. This is speculative, so somehow I need to test with an older version to verify.

No, and having the January security update is important.

Perhaps, but I can do without January updates for the purpose of testing.

Can I flash an older .zip with fastboot without breaking the installation?

If so, how?

thestinger commented 9 months ago

You can install an older version of the OS if you want to do that. It will be a fresh install and the data will be gone. The issue you're having is probably a configuration issue. A fresh install with an older version of the OS not having the issue wouldn't demonstrate that it was a regression unless a fresh install of the current version was also tested to confirm the issue occurs there and wasn't caused by configuration. Many people who heavily use secondary users on GrapheneOS. It's not unusual. It wouldn't be the only case of the problem if it simply regressed in the update, and likely would have been reported while in Alpha and Beta.

The releases on the servers are 2023123100 and 2024010400. There isn't room to keep a bunch of older releases, but they're kept longer locally. There are currently 4 releases kept locally, which are 2023121200, 2023123000, 2023123100 and 2024010400. Older releases are not officially archived.

KJ7LNW commented 9 months ago

There isn't room to keep a bunch of older releases, but they're kept longer locally.

Locally where, locally on my device?

Older releases are not officially archived.

Ok. I still have 20231208000 on my system in case it becomes useful.

You can install an older version of the OS if you want to do that. It will be a fresh install and the data will be gone.

How do developers do bisections? I find it hard to believe that there is no way to install an older version without wiping the data.

For example, can I modify the deployment script in the .zip and comment out the wipe data part so it just overwrites the files? There must be a way...

thestinger commented 9 months ago

Locally where, locally on my device?

Your device doesn't download entire releases in the first place, only differences from previous to latest as part of a delta update, and then ends up in the same place as a fresh install of the new release.

How do developers do bisections?

Developers are usually using userdebug builds signed with their own keys, not the production releases (user builds) signed with official release keys. They're also not generally doing development on a locked device unless it's specifically for testing things related to verified boot / attestation or they're testing official releases which are not things that most of the team are involved in doing.

I find it hard to believe that there is no way to install an older version without wiping the data.

Your device is running a production release and is locked. You can't downgrade the OS. There are 3 layers of security checks you would need to bypass: signature verification with downgrade protection for the update package, signature verification with downgrade protection for update_engine and signature verification with downgrade protection for verified boot which is tied into attestation, disk encryption, keystore, etc.

In general, it's also broken to downgrade software that way. New versions move to new data / configuration formats. New versions must correctly handle all previous versions of data / configuration which could have been used on any still supported device. Old versions do not and can not handle all the data / configuration of new versions.

This also applies to apps, not only the OS. You can't downgrade apps without reinstalling them either. If the app supports a cross-version backup/restore mechanism, you can use that. The OS should only restore the data to an equal or greater version of the app via an OS backup mechanism.

KJ7LNW commented 9 months ago

Ok, so its possible but complicated, I'm fine with that. But lets start with the easy part:

The issue you're having is probably a configuration issue

Lets assume that this is so. I would be so grateful if you can help me through the configuration issue!

Are there any configurations besides the per-user setting "Turn on phone calls & SMS"? If so, what other configuration settings affect this?

Keep in mind that the user of this phone doesn't login to the owner account and has no clue about how to change settings. SMS has worked since installation on Dec 8. The problem has only manifest in the last day or two.

Anything you can do to help would be greatly appreciated.

KJ7LNW commented 9 months ago

Well, I don't know what happened because I tried this same procedure yesterday. Today I tried again and it started working. Very strange. Here is the procedure that "fixed" secondary user SMS in case it helps someone else.

  1. End Session on the secondary user
  2. Turn off "Turn on phone calls & SMS" for that user
  3. Login to secondary user
    • Send the user an SMS from another phone
    • No message is displayed, which is expected with SMS off for the user.
  4. End session
  5. Turn on "Turn on phone calls & SMS"
  6. Login to user
    • Send the user a test message
    • It worked!
KJ7LNW commented 9 months ago

Oh wait, its not fixed yet. This just got even more weird:

It worked shortly after login, but now it is back to the same behavior: a "New Message" notification pops up, but as soon as I tap on it the window disappears and the message is never shown in the stock messaging app.

KJ7LNW commented 9 months ago

Here is what I've found:

  1. Messages arrive when logged in as the secondary user (the "New Message" notification is displayed), and when you tap the notification it loads the stock messaging app, but that text message does not exist in the message for that contact.
  2. If I switch to the Owner user and back to the secondary user, the messages that didn't appear before from (1) are now visible.
  3. Doing an "end session" doesn't appear to change anything, just switching between users is sufficient.

Questions:

thestinger commented 9 months ago

Please try using the reset app preferences reset option. This will reset all permissions to their default state which will be annoying but might solve the problem.

KJ7LNW commented 9 months ago

I ran the preferences reset on both owner and the secondary user, but the behavior remains. I tried ending session and toggling the "Turn on phone calls & SMS" option for that user to see if it would help after the pref reset, but no change.

KJ7LNW commented 9 months ago

Below is part of the logcat trace from the secondary account that is linked above.

Could it be a permission issue in DatabaseUtils? It has an exception stack trace pretty close to the message receipt, only 68ms apart. Would it be possible to add READ_SMS access to DatabaseUtils and see if it fixes the problem? If so, how?

01-07 13:30:26.588  2321  2321 I SHANNON_IMS: 9575 [SMS#] ImsSms onGsmSmsReceived : slotId=0, msgTpId=1, verstat=0, tpduLen=32, token=1708493723, isMwiMsg=false (ImsSms$NetworkListener%onGsmSmsReceived:304)
01-07 13:30:26.611  2321  2321 I SHANNON_IMS: 9576 [SMS#] ImsSms gsm OriginatingAddress=+1<REDACTED-MY-NUMBER> (GsmSmsUtils%getOriginatingAddress:92)
01-07 13:30:26.640  2321  3357 I SHANNON_IMS: 9578 [IMPL] #:BLUE:# Commit ImsSmsImpl acknowledgeSms slot 0  [com.shannon.imsservice.impl.ImsSmsImpl@cf044ae] (ImsSmsImpl%acknowledgeSms:87)
01-07 13:30:26.640  2321  3357 I SHANNON_IMS: 9579 [SMS#] ImsSms acknowledgeGsmSms :slotId=0, token=1708493723, messageRef=0, result=1, handleResult=1, msgTpId=1 (ImsSms%acknowledgeGsmSms:969)
01-07 13:30:26.641  2321  3357 I SHANNON_IMS: 9580 [ADAP] Sending gsm sms ack msgTpid [1] result [1] with errorCode [0] [SLID:0 ] (SmsAdaptor%sendGsmSmsAck:96)
01-07 13:30:26.641  2321  3357 I SHANNON_IMS: 9581 [HELP] #:PURP:# SOL TX{RILC_REQ_AIMS_SEND_SMS_ACK} : result : 1  msg-tpid : 1  error-code : 0  [SLID:0  Active:true] (ImsRilHelper%sendRequest:313)
01-07 13:30:26.641  2321  3357 V ImsRIL  : [ImsRIL_171]> STATUS_MAX 01 00 00 00 01 00 00 00 00 
01-07 13:30:26.656  2378  2884 E DatabaseUtils: Writing exception to parcel
01-07 13:30:26.656  2378  2884 E DatabaseUtils: java.lang.SecurityException: Permission Denial: reading com.android.providers.telephony.SmsProvider uri content://sms/sent from pid=11762, uid=1010144 requires android.permission.READ_SMS, or grantUriPermission()
01-07 13:30:26.656  2378  2884 E DatabaseUtils:     at android.content.ContentProvider.enforceReadPermissionInner(ContentProvider.java:1011)
01-07 13:30:26.656  2378  2884 E DatabaseUtils:     at android.content.ContentProvider$Transport.enforceReadPermission(ContentProvider.java:813)
01-07 13:30:26.656  2378  2884 E DatabaseUtils:     at android.content.ContentProvider$Transport.query(ContentProvider.java:253)
01-07 13:30:26.656  2378  2884 E DatabaseUtils:     at android.content.ContentProviderNative.onTransact(ContentProviderNative.java:107)
01-07 13:30:26.656  2378  2884 E DatabaseUtils:     at android.os.Binder.execTransactInternal(Binder.java:1387)
01-07 13:30:26.656  2378  2884 E DatabaseUtils:     at android.os.Binder.execTransact(Binder.java:1328)
empratyush commented 9 months ago

Hey @KJ7LNW DatabaseUtils is not missing any permissions.

DatabaseUtils is part of the framework which enforces permission. Can you please try a few troubleshooting steps?

a) Make sure SMS permission is granted for the Messaging app. (Settings -> Apps -> See All Apps -> (select) Messaging -> Permission -> SMS) b) revoke and regrant SMS permission for Messaging app. b) Try any other Messaging app if it doesn't work then it might be a system issue rather than a Messaging app one. c) provide the output of adb shell pm dump com.android.messaging if another Text Messaging app works.

KJ7LNW commented 9 months ago

a) Make sure SMS permission is granted for the Messaging app. (Settings -> Apps -> See All Apps -> (select) Messaging -> Permission -> SMS) b) revoke and regrant SMS permission for Messaging app.

Before opening this Github issue I tried (a) and (b) above, no change. I'll try again and report back just to double-check.

b) Try any other Messaging app if it doesn't work then it might be a system issue rather than a Messaging app one. c) provide the output of adb shell pm dump com.android.messaging if another Text Messaging app works.

I tried QKSMS but it didn't fix the problem, so I went back to the stock app. I've read that some SMS apps don't work with unpriveleged GrapheneOS profiles. If you know of a (non-google) SMS app that you know does work on GrapheneOS under a secondary non-owner profile, then let me know which one, and I'll give it a try.

KJ7LNW commented 9 months ago

@muhomorr,

@KJ7LNW Try to clear storage of Messaging app in the secondary user.

already tried that too.

KJ7LNW commented 9 months ago

@empratyush,

a) Make sure SMS permission is granted for the Messaging app. (Settings -> Apps -> See All Apps -> (select) Messaging -> Permission -> SMS)

Here is what I tried today:

  1. Swipe the messaging app away from open apps so it "closes"
  2. Turn off the messaging app's SMS permissions
  3. End session
  4. Log back into the profile
  5. Turn on the messaging app's SMS permissions
  6. Send test1 from another device: no notification was recieved
  7. Switch profile to Owner: the SMS does exist in the owner account
  8. Switch back to secondary account: no text message and no notification
  9. Send test2 from another device: no notification was recieved
  10. Swipe the messaging app closed
  11. End session
  12. Login to Owner: test2 arrived
  13. Switch back to secondary account: test2 did not arrive
  14. Send test3 from another device: no notification was recieved
  15. Swipe the messaging app closed
  16. End session
  17. Login as Owner
  18. test3 arrived
  19. Toggled the "Turn on phone calls & SMS" setting for the secondary user off and then on
  20. Switch back to secondary account
  21. Sent test4, no notification, not recieved.
  22. Switch to owner account (without end session)
  23. Switch to secondary account: test4 was recieved on the secondary account
  24. Send test5: not recieved in secondary account, no notification
  25. Switched to owner: test5 arrived
  26. switched back to secondary user: test5 still does not show
  27. end session
  28. switched back to secondary user: test5 still does not show
  29. Switched to owner: marked test5 as read
  30. switched back to secondary user: test5 still does not show

It no longer reliably shows the text message after switching to Owner and back to the secondary user. Maybe it never did, or maybe toggling the messaging SMS permission broke something else.

Anyway, what to try next?

KJ7LNW commented 9 months ago

FYI: I just tried enabling SMS on a different profile that had never used it before. When I send while that profile is active, a notification is displayed, but when you tap on it, the notification disappears quickly and the message is never shown in the messaging app.

Switching back and forth to another user (even Owner) doesn't seem to refresh the messages page anymore, even on this 3rd profile.

I also tried adding a brand-new 4th profile to see if a fresh start would fix things, but it has the same issue, no change.

The only place that SMS messages arrive reliably is the Owner account (which I renamed to "Admin" long ago, but I doubt that the name matters).

KJ7LNW commented 9 months ago

If anyone here can talk me through installing a .zip of an older version without wiping the rest of the OS then I'd like to give that a try. I realize this is unsupported and may just break things more, and I accept that risk. I am very familiar with fastboot and adb, so please be as technical as you would like.

I've already downloaded a couple pre-2024010400 factory zips, so I'm ready to test.

thestinger commented 9 months ago

If anyone here can talk me through installing a .zip of an older version without wiping the rest of the OS then I'd like to give that a try. I realize this is unsupported and may just break things more, and I accept that risk. I am very familiar with fastboot and adb, so please be as technical as you would like.

It's not possible, It will be rejected as invalid due to having a lower version by recovery, update_engine and verified boot. You would have to unlock which will wipe all data.

KJ7LNW commented 9 months ago

You would have to unlock which will wipe all data.

Ah, that's the kicker then. I new I shouldn't have locked it yet!

Would you be willing to release a test build on the alpha release channel that has a newer version, signed by your key, but built at the git commit for 20231208?

This would help us rule out config vs release issues. If its really a config issue, then I'll re-install from 2024010400, but if this is a regression then it would be nice to fix in case it affects others. Such an alpha release would prove that one way or the other.

thestinger commented 9 months ago

Locking also wipes all data. Locking is part of finishing the install.

thestinger commented 9 months ago

Would you be willing to release a test build on the alpha release channel that has a newer version, signed by your key, but built at the git commit for 20231208?

Even if it was okay to do that, it wouldn't boot for you because the security patch level is set lower and can't be downgraded due to verified boot downgrade protection for the OS being based on security patch level.

KJ7LNW commented 9 months ago

This is open source, you have full control here, right? You could tweak the security patch level as part of the build in a test branch so it is greater than what I have now.

thestinger commented 9 months ago

@KJ7LNW It wouldn't be okay to do that though. I think you're missing that it would weaken update and verified boot security for everyone else.

thestinger commented 9 months ago

Someone else might have ideas on how to resolve it but I'm out of obvious ideas.

Try resetting network settings for cellular in Owner reset settings.

KJ7LNW commented 9 months ago

I think you're missing that it would weaken update and verified boot security for everyone else.

No, I get that, thats why I suggested the alpha channel where production users wouldn't generally subscribe.

thestinger commented 9 months ago

The issue is that simply by it existing, it reduces update and verified boot security.

thestinger commented 9 months ago

Even if that wasn't the case, it couldn't go out through Alpha, but rather would need to be sideloaded or use an internal release channel.

KJ7LNW commented 9 months ago

would need to be sideloaded or use an internal release channel.

That would be fine too.

thestinger commented 9 months ago

I'm saying that I can't do it because it would weaken verified boot and update security for others through it existing and being published.

thestinger commented 9 months ago

Please try this:

Try resetting network settings for cellular in Owner reset settings.

There are a few more things to try like clearing data for OS app-based SMS components too after exporting the current message history to save it.

KJ7LNW commented 9 months ago

Try resetting network settings for cellular in Owner reset settings.

Tried, rebooted, no change.

There are a few more things to try like clearing data for OS app-based SMS components too after exporting the current message history to save it.

Sure, how do you export message history?

matchboxbananasynergy commented 9 months ago

Try resetting network settings for cellular in Owner reset settings.

Tried, rebooted, no change.

There are a few more things to try like clearing data for OS app-based SMS components too after exporting the current message history to save it.

Sure, how do you export message history?

Should be able to use something like https://github.com/tmo1/sms-ie to export SMS.

KJ7LNW commented 9 months ago

@matchboxbananasynergy, thanks I'll try it when I get back.

@thestinger: after I export SMS, what should I clear/delete in the OS SMS app settings?

Assuming I understand what you mean, which system apps should have their data cleared? Otherwise please correct me with more detail.

-Thanks!

Nephiel commented 9 months ago

I suffered from this issue today, which resulted in missing 2FA codes from shopping apps and online banking.

Same symptoms: the SMS notification shows up on my secondary (main) user profile, and tapping it opens the stock GrapheneOS SMS app (as it should), but the message is not there.

Switching momentarily to the owner profile made some of the missed messages show up in the app, but eventually stopped working and the messages were lost.

Tried after a full power cycle of the device, and also clearing both cache & data for the SMS app in both profiles (sacrificing history in hopes of getting it working). Didn't work - the issue persists.

KJ7LNW commented 9 months ago

Hi @thestinger, you said

There are a few more things to try like clearing data for OS app-based SMS components too after exporting the current message history to save it.

What should I try to clear, specificalily?

KJ7LNW commented 9 months ago

@thestinger - I'll be reloading the phone soon, which means we will lose any opportunity to troubleshoot in a state that reliably reproduces this specific problem.

If you have anything else for me to try, then please let me know ASAP.

thestinger commented 9 months ago

Search system apps for Phone and Messaging Storage, clear that too, along with clearing Messaging app data again.

KJ7LNW commented 9 months ago

For settings->apps->system->"Phone and Messaging Storage", the "Clear storage" button is greyed out. I tried doing "Clear cache" which is not greyed out, but no change.

How can I enable "Clear storage" for "Phone and Messaging Storage"?

Note that I tried both the Owner use and the secondary user, and the "Clear storage" button is greyed out for both.

thestinger commented 9 months ago

I think you just need to try a factory reset.

KJ7LNW commented 9 months ago

ok. Is "Clear storage" intended to be disabled for "Phone and Messaging Storage"?

thestinger commented 9 months ago

Yes.

KJ7LNW commented 9 months ago

Sadly, it works after a manufacturer reset, so I have no way to help troubleshoot this further.

Since we know at least one other person is having a problem, maybe others will be able to provide additional information to pin down the issue.

Nephiel commented 9 months ago

Apparently I can no longer reproduce it either. Yesterday I received SMS on my user (non-owner) profile, and tapping the notification opened them in the Messaging app without issue.

thestinger commented 9 months ago

It doesn't sound like we're going to be able to work on it. It's not clear why it's happening and we need someone able to reproduce it in order to solve it.

valentinbreiz commented 9 months ago

Hello, I think I have nearly the same problem. I'm using a secondary user for casual use but I never get SMS sounds. Just the text poping in the topbar. The other weird thing is the SMS doesn't appear directly in the messages app, I have to wait a few seconds...sometime minutes This is kind of annoying ahah

thestinger commented 9 months ago

@valentinbreiz That's a different issue. Does it happen with the default Messaging app or a different app?

valentinbreiz commented 9 months ago

i'm using SMS Messenger from the simple apps collection

Screenshot_20240118-023123

valentinbreiz commented 9 months ago

Do you want me to create an issue or it's a known problem? Edit: i think with the default app it was the same. And it works fine on the main user

thestinger commented 9 months ago

Try the default Messaging app in the secondary user. There are known issues with non-default messaging apps in secondary profiles specifically. Don't really need a new issue made for that, it's a known Android limitation. You can probably find info about it in various places including our issue tracker and forum but there may not be a full solution.