G Suite notice about SMS Backup+ becoming restricted

[ghost]

[Note added by @kurahaupo, 2020-07-08]

1. This thread does have useful information but it is somewhat buried because the thread is so long. Please read through it before adding more comments. If you're just looking for a quick fix, there's an explanation at the top of #972.
2. Although this report dates from June 2019, in July 2020 G-suite users are being notified (again) that new installations of SMS Backup+ won't get Gmail sensitive scope access; it's the same problem, just different timing.
3. This policy change should not affect access to Calendars. If you have a problem with calendars, raise a separate ticket. (Note: do not enable Connect your Gmail account (unsupported), as that's only for Gmail, not for Calendars.)
4. The changes to sensitive scope access are explained at https://developers.google.com/terms/api-services-user-data-policy#additional-requirements-for-specific-api-scopes A major use-case is being able to read SMS & MMS alongside email & hangouts messages, and that ought to be sufficient justification under term 3 of the rules "Applications that enhance the email experience for productivity purposes", but it's hard to argue that case when the name says "Backup".

[Original report follows]

I received the attached email from Google that your app will become restricted on 7/8. Thought you'd like to know. SMS Backup.pdf

[MarkMessinger]

A similar email is being sent to ordinary users of Gmail. This one refers to a restriction commencing July 15, 2019. SMS+ Backup.pdf

[B150n]

I had the same experience as @MarkMessinger. Very worried that we'll lose THE best SMS backup app around.

[WebAddict]

Same - July 15 will stop working if "unable to meet the deadline to comply with our updated data policy requirements"

[AEFeinstein]

I have also received this email

[bwwestbrook]

Same email.

Hi,

Although you don’t need to do anything, we wanted to let you know that the following apps may no longer be able to access some data in your Google Account, including your Gmail content. If these apps are unable to meet the deadline to comply with our updated data policy requirements, they'll lose access to your Account starting July 15th, 2019.

SMS Backup+ We are making this change as part of ongoing efforts to make sure your data is protected and private.

You can always view, manage and remove apps you’ve given access to your account by visiting your Google Account.

Thanks, The Google Accounts team

[malversan]

It seems that the app must pass a verification process to continue accessing GMail accounts the way it does.

The Google API scopes to create and read GMail messages (used by SMS Backup+ to backup and restore SMS messages) are now restricted scopes, and apps that use them need to be audited by Google to verify that they comply with the OAuth security policies and to verify that they do not make an ilegitimate use of that APIs.

And for the app to be verified, the developer must apply for that verification. There are limit dates for that too. The too-late deadline is July 15th, when the app will loose access to that APIs if nothing is done.

About restricted API scopes: https://developers.google.com/terms/api-services-user-data-policy#additional-requirements-for-specific-api-scopes

About the verification process: https://support.google.com/cloud/answer/9110914#restricted-scopes

We need the developer urgently.

[malversan]

Please, do not clutter this thread by repeating the same information a hundred times. We all have received the same message today, Jun 25th, so there is no need to post it more than once nor repeating ”me too” once and again.

Let's try to get a solution, not to make a whinning cry of this.

[jpellman]

It's worth pointing out that, while it will be a bit obnoxious, you don't need to use the default OAuth-based mechanism to back up your texts. As stated in the docs here you can configure the app to use plain old vanilla IMAP.

[jason-mehmel]

It's worth pointing out that, while it will be a bit obnoxious, you don't need to use the default OAuth-based mechanism to back up your texts. As stated in the docs here you can configure the app to use plain old vanilla IMAP.

We can keep backing up to gmail, just through IMAP? Would it still track the same SMS label that we've applied to the texts?

[malversan]

I think using GMail IMAP is subject to the same restrictions that require the aforementioned verification process.

Asides, using another IMAP server is a no go for me. This app is not only a backup utility, it also allows to search for specific SMSs using GMail and search for calls you issued using Google Calendar. It's not just a backup, it's much more when combined with a Google account.

The fact is that I think the app already complies with the OAuth security requirements claimed by the Google verification process. So we only need the developer to apply the app for that verification process. I have just emailed him asking for his help. I hope he has a bit of time and interest in keeping his awesome creature alive.

[KyleSanderson]

@jberkel send help

[pingu8007]

It's worth pointing out that, while it will be a bit obnoxious, you don't need to use the default OAuth-based mechanism to back up your texts. As stated in the docs here you can configure the app to use plain old vanilla IMAP.

Yes, we can fallback to vanilla IMAP. And no, Gmail won't let it happen. According to experience from K-9 mail, Gmail will do its best to prevent you access IMAP even with "less secure apps ON" or "2-factor-authentication OFF". (See https://github.com/k9mail/k-9/issues/655)

[jberkel]

Hello everyone. I'm sorry about this situation, SMS Backup+ will no longer have access to Gmail, mainly because it's not an email reading app.

I applied for an exception but it was declined, as expected. Vanilla IMAP might work, but for how long I wonder. And it's very tricky to set up for a casual user. Unfortunately the Android platform is getting more and more closed.

I'm not sure what to do at this point, either remove the app from the store or release a new version which removes the automatic account setup, since that is broken / will be broken soon.

[Hobart]

Off topic: I took the opportunity to hit Donate in the app, thanks for years of great software @jberkel & the other committers.

[sshaikh]

Is it possible to use some kind of personal developer key (assuming such a thing exists)?

[ghost]

I was able to whitelist the app in my Google Apps domain. Do you think that will eventually stop working as well?

[YMJH]

Oh no! This is an unwelcomed shock to me. Thank you @jberkel for your wonderful app. Like so many others, I am not happy about Google's decision. SMS Backup+ has proved invaluable to me over many years. Please, please provide an alternative prior to 15 July 2019 if possible. Ho hum. Google (along with Apple and Microsoft) are controlling too much (IMHO). Please let us know if you are able to provide another app which Google deem acceptable. Thank you.

[bwwestbrook]

@jberkel Great work on the app. It has served us all so well for so long. Keep up the great work!!

[GentlyD]

Thanks @jberkel for the help decluttering my communication. Donation done.

[ZecChelovek]

Forgive my ignorance, but what will happen to my backup of messages already on my google account?

[tomswartz07]

What will happen to my backup of messages already on my google account?

Probably nothing. They're uploaded and currently indistinguishable (from Google's perspective) from regular email messages, save for the unique label if you used it.

[ZecChelovek]

^That is greatly relieving. Thank you.

[survivor303]

So does that mean, that the app cannot save sms back to gmail anymore? Or just restore them from there?

Im perfectly fine, if the app can just copy them to gmail, that is why i use this application, not for restore.

[Jasonfhaught]

You will have to go to advance setting SMS Backup+ then go into customize imap and set the imap.gmail.com:993 and use you email as user ID and your password. Make sure the authinication is set to clear text and security is tls.

After that go to a PC and login to gmail, in the right hand corner click the cog then setting then goto the tab "forward and pop/imap". Under the imap access change it to enable.

Now go to you gmail account settings on the left side click security. Scroll down to less secure settings and turn it on.

Now you should be able to continue to use the app. It is just not as secure.

Google will probably not close this method down anytime soon to many developers and admins you this method for logs

[Sancus]

I'm not sure what to do at this point, either remove the app from the store or release a new version which removes the automatic account setup, since that is broken / will be broken soon.

Personally, I'm happy to keep using this app via IMAP(with gmail or any other IMAP service), so I hope you don't remove it :)

[domhnallog]

Silly question perhaps but could you fork something open like Thunderbird and add in the and backup feature as a feature of that?

I also have made a donation. Thanks very much. It was a great service while it lasted.

[TenTeypek]

Oh, that's so sad. It was the best SMS backup app. Will there be any option to backup to somewhere else, if Gmail doesn't work?

[oryz]

but could you fork something open like Thunderbird and add in the and backup feature as a feature of that?

Was thinking the same thing! Happy to contribute to the effort as well.

Many thanks @jberkel and anyone else that has helped for years of really great service.

[MysticMagican]

For wich reason your application was denied, @jberkel? Was that finally, or are there any thing d for could change or provide to them to keep the app working?

[rdslw]

I described situation to https://news.ycombinator.com/item?id=20282361 Please upvote it and/or comment.

There is a high chance we might get it higher to google echelons this way and get the exemption.

[koying]

@jberkel Sorry to hear what's happening. Let not Google's policies discourage you. Your app is needed, and whether sync is done to gmail or other services is kind of secondary.

May I suggest that the first course of action would be the possibility to do a "full export" to other imap/services. Or is it already somehow implemented ?

Thanks

[TazUk]

Hello everyone. I'm sorry about this situation, SMS Backup+ will no longer have access to Gmail, mainly because it's not an email reading app.

I applied for an exception but it was declined, as expected. Vanilla IMAP might work, but for how long I wonder. And it's very tricky to set up for a casual user. Unfortunately the Android platform is getting more and more closed.

I'm not sure what to do at this point, either remove the app from the store or release a new version which removes the automatic account setup, since that is broken / will be broken soon.

@jberkel thanks for updating us all.

Just to clarify, would the change also impact the syncing of the call log to Google Calendar, or does the verification requirement not affect this functionality? If the latter, I'd appreciate the app remaining in the Play Store even in abbreviated form as both my wife and I rely on the Google Calendar call logs.

[jogilder]

This is the second app that I use which has lost the gmail connectivity. I assumed the previous app developer (Gnotes) had chosen not to go through the hoops but now I wonder. A real shame. I have donated too in the hope that there can be an alternative solution.

[malversan]

Hello everyone. I'm sorry about this situation, SMS Backup+ will no longer have access to Gmail, mainly because it's not an email reading app.

I applied for an exception but it was declined, as expected. Vanilla IMAP might work, but for how long I wonder. And it's very tricky to set up for a casual user. Unfortunately the Android platform is getting more and more closed.

I'm not sure what to do at this point, either remove the app from the store or release a new version which removes the automatic account setup, since that is broken / will be broken soon.

First of all, thanks a lot for your efforts.

Please, do not remove any features by now. Some of us use your app not only for backup purposes, but also for search and logging purposes. Searching SMS messages in GMail and looking for calls in the Google Calendar are great additional features that your app provides. And as long as I understand, Google should not ban your access to GMail using pure IMAP protocol, neither the access to the Google Calendar API, only the GMail API should be restricted.

A question for you, @jberkel: Could the Google Calendar entries be managed using pure IMAP protocol? Are they, currently? If so, by setting "Less secure apps" access in our Google accounts, we could get rid of the entire Google API and the automatic account setup.

By the way, could you please provide a Google email address where I can politely address a good reason not to ban SMS Backup+ access to the GMail API? Better provide it in private, to avoid cluttering them with complaints from ranting people.

[jamgregory]

@malversan - unfortunately, the IMAP protocol is purely for syncing emails, so it wouldn't be possible to sync calendar entries using that :(

(For more info: https://support.office.com/en-us/article/sync-basics-what-you-can-and-cannot-sync-5537d587-4930-4ac2-b044-3568509b1294)

[malversan]

@malversan - unfortunately, the IMAP protocol is purely for syncing emails, so it wouldn't be possible to sync calendar entries using that :( (For more info: https://support.office.com/en-us/article/sync-basics-what-you-can-and-cannot-sync-5537d587-4930-4ac2-b044-3568509b1294)

@jamgregory, as long as I can see that link only refers to Outlook client syncronization, so it does not answer the question I asked.

I have discovered that Google Calendar also supports being managed using the CalDAV protocol, but unfortunately I am afraid it does not allow to get rid of the OAuth credentials setup. https://developers.google.com/calendar/caldav/v2/guide

Some apps already use it: http://www.ubuntubuzz.com/2017/07/how-to-setup-thunderbird-for-google-calendar-caldav-read-write-access.html

But I cannot foresee the impact that using CalDAV protocol would have in the app usability and workload.

[meraj-ahmed]

Just throwing out an idea here. If the app were to include some very basic functionality that allowed you to browse the previously backed up SMS/call records in your Gmail account, would that perhaps qualify the app for the exception as an "email reading app"?

[malversan]

Just throwing out an idea here. If the app were to include some very basic functionality that allowed you to browse the previously backed up SMS/call records in your Gmail account, would that perhaps qualify the app for the exception as an "email reading app"?

That is a pretty good idea. Maybe making statistics with the inbound/outbound call times and SMS messages sent/received.

Altough I suppose that the developer would better accept solutions that do not imply a heavy workload. Anybody here could implement something like that? All in all, the app is open source so the sources are public.

[malversan]

I have just posted an issue in the Google issue tracker, under the GMail API category, exposing why restricting too much the access to the Google APIs for security reasons can paradoxically lead to the Google accounts being less and less secure. I also sent the same message to "oauth-feedback@google.com".

https://issuetracker.google.com/issues/136079176

Of course I use the SMS Backup+ app as an example to defend the argument, so maybe it could lead to someone reconsidering the ban of this app from the GMail API. I do not expect great results from that "political" via, but it is worth trying.

[arnaldop]

Anyone who cares about this product, make your voice heard HERE: https://issuetracker.google.com/issues/136079176

Thanks, @malversan, for getting the ball rolling. If we can do any more to salvage this amazing app, let us know!!!

[JKS258]

Hi,

Like everyone else, I'm enormously grateful for the many years of hard work you've put into such an excellent App, though I'm infinitely aggravated at Google since this is the second of my top two utilities they've killed, or announced they're killing on the Android operating system in the last 30 days! Both of the Apps I use many times HOURLY and both have either increased my productivity multi-fold, or saved me countless times, as is the case of your App, either legally, or from a he-said, she-said debate, or from just having a chronological history of my life since 90% of my communication is via text messaging. In the very least your App has enabled me to reconstruct events via the timeline inherent in text messages, which up until now were safely protected in Gmail, which I also backup frequently. I've been self-employed as a computer consultant for 30 years and the ability to go back YEARS and pull up often valuable information has been priceless and has prevented several liability issues because I had in time-stamped digital format, saved by SMSBackup+, a record of what someone said. So for Google to kill your App by fixing something that "ain't" broken and CONSTANTLY dinking with products that work just fine as is, is REALLY about to drive me over the edge, or more specifically, away from technology altogether! (this is by no means the first time they've killed a great product, with Picasa being one that to this DAY I've never been able to find an equal to). That's a big statement coming from someone who has made a ton of money over 30 years in this industry. But the industry continues to be so fragmented, and in such a perpetual state of change, often just for the sake of change, that I'm rethinking whether I want to continue investing a rapidly increasing portion of my time RE-DOING something that was working just fine as it was. It's funny that 25 years ago I figured that the time would come when my services would no longer be needed because computers would be so easy to operate and EVERYONE would know how to use them. In fact, it's gotten WORSE! The average user still can't format a letter in MS Word, or use even 2% of the potential a computer has, yet they keep buying new ones because they're forced to by the industry.

Like the others, I've used your App since you introduced it, and as far as I know, there IS no equal. So where does that leave us users (not pointing at you of course)? We're all going to end up investing vast amounts of time trying to create a poor-at-best workaround for what is currently a flawless system. I could ramble on forever about how aggravated I am at the first program Google just killed, which I hardly go 10 minutes without using, and now they've hit my second of two priceless Apps, SMSBackup+, and NEITHER have a solution as far as I know. How many years has it been since they killed Picasa and again, there's STILL no viable alternative. Throw in the Google+ fiasco, which I knew from day one was a loser, NEVER used, though the attempt was made to force me to use it, and it makes me wonder who's driving the boat at Google?

OK, complaining over. But I do have one hopefully simple question. Like I read someone else say, ALL I need from SMSBackup+ is to BACKUP to Gmail, so I can do searches and of course HAVE a backup. I will NEVER need to restore from Gmail, so will your App still be able to do a one-way backup to Gmail, or is that part of what's being killed?

Thanks again for all you've done for how many ever years since you wrote your MOST excellent App. It has again been priceless to me and I HOPE that the backup function will NOT go away. If it does, I'm still grateful to you, but even more aggravated with Google. With all of my spare time after finding workarounds for the loss of my most key apps day after day, I plan to write them a letter expressing my dissatisfaction with their fixing things that aren't broken. There are risks with EVERYTHING online that are NEVER going away, so to try and fix what they consider a risk, I have to wonder how many times has that hole been breached that makes it worth them investing their time and money to fix it, plus crippling thousands, if not millions of the people that actually USE the software they're killing?

On a personal level though, sincerely, thank you and continued success, Jay...

On Tue, Jun 25, 2019 at 6:25 PM Jan Berkel notifications@github.com wrote:

Hello everyone. I'm sorry about this situation, SMS Backup+ will no longer have access to Gmail, mainly because it's not an email reading app.

I applied for an exception but it was declined, as expected. Vanilla IMAP might work, but for how long I wonder. And it's very tricky to set up for a casual user. Unfortunately the Android platform is getting more and more closed.

I'm not sure what to do at this point, either remove the app from the store or release a new version which removes the automatic account setup, since that is broken / will be broken soon.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jberkel/sms-backup-plus/issues/959?email_source=notifications&email_token=AJMQPABOOZL4RNNAJTTY4NTP4KLMVA5CNFSM4HYI7LT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYRYUCI#issuecomment-505645577, or mute the thread https://github.com/notifications/unsubscribe-auth/AJMQPADK7GHBL544Y2NTT4TP4KLMVANCNFSM4HYI7LTQ .

[malversan]

Anyone who cares about this product, make your voice heard HERE: https://issuetracker.google.com/issues/136079176

What the heck...? That was NOT the intended strategy, @arnaldop. What I subtly tried to do was to present the indiscriminated API restrictions as a potential threat for Google by forcing users to abandon cautions and good security practices. And by putting the SMS Backup+ case as an example maybe they would rethink about shutting down its access to the GMail API.

But it seems the subtlety is not an extended skill around here. Now you have converted that issue report in a joke, with tenths of messages saying they love this app, thus completely revealing the intention. Even worse, now it clearly appears as a coordinated plot, something that was not. That issue report is not an issue report anymore, it has become the complaint of a bunch of spammers about the problems of a third-party app that Google does not care about. So the tracker administrators will probably delete the entire post without reading past the third irrelevant message of the tenths you have written. You all ruined the attempt to convince Google of something THEY care about.

Understand this: Nobody at Google cares a damn about you liking or needing this app. That was definitely NOT the idea. You can only convince someone with issues that concern HIM, NOT YOU.

By the way, @JKS258, I can only congratulate you for completely ruining any possibility of Google gaining sympathy for this app. Do you know the meaning of politics, diplomacy, or even intelligence? What is all that shit you wrote there about Picasa and spaceships? In which universe do you believe that it is a good strategy to attack frontally the one you want to obtain something from? That is not a shitty forum for you to rant, it is the Google official products issue tracker, and the idea was to gain Google sympathy and confidence on the security of SMS Backup+. Now your rants and not-so-subtile threats against Google have assured that nobody will take seriously any argument in that report. Worse than that, you exposed all that unrelated and inappropiate comments talking as if you were the developer of the app, so you have also assured 100% that Google will never listen to him either. Congratulations, heartly.

Definitely I must learn to avoid trusting in people´s intelligence.

[jcscomsol]

First many thanks for an app used for many many years.

Is it possible for the app to upload to "another" imap server to record sms and call log?. Could caldav be used with say NextCloud for calendar content?

I hope there is a rethink over this decision.

[jpellman]

Honestly any political strategy that is transparently stated on a public forum (which Googlers will probably find pretty quickly, given that [a] they work on, you know, search and [b] you included a link directly to this project) probably isn't going to work anyways. I've always been under the impression that part of any effective political strategy is to keep your cards close to your chest. At this point, you might as well just keep telling them directly that they're being [SILLY COWS] for refusing to allow this app.

[weaversam8]

@malversan ignoring the potential validity of your argument, the hostility is unnecessary.

[malversan]

@jpellman, finding this forum does not invalidate anything, I expected it to happen and my argument would still be completely valid. What invalidates the entire thing it is to convert that issue report in a spamming circus about a problem that Google just does not care at all.

You have to understand that any attempt goes through talking about things they care, not about what you care.

(And of course insulting them never helps to attract their favorable attention, that should be evident for everyone)

@weaversam8, the hostility comes from the fact that I vainly spent a certain amount of time to carefully write that report to accomplish the goal, thinking that I was surrounded by intelligent adults who knew how to do things right. Believe me I do not care about you understanding it or not. If you don't know what an issue tracker is (and all who posted there clearly don't), you simply should not have posted anything there. Period.

In fact there is still people posting love cards in the issue tracker right now. I bet my head that most of them have not even read the report I wrote and simply don´t care what it is about. They just think that´s Twitter. To be honest, I can only think they are stupid and can´t distinguish where and when to do each thing.

[tomswartz07]

To all on this thread, please be aware that every message is sent to all 35 active participants as well as all 172 watchers. I believe we've all made it clear that the issue exists and many have said their piece. @jberkel may choose to lock the thread if the hostile communications continue.

For the sake of keeping items on task, please refrain from adding anything additional unless it's precisely related to resolving the issue; either code suggestions or documentation (from Google or other affected applications) indicating a workaround.

[tvh2k]

Long time SMS Backup+ user. Admittedly I haven't reviewed this app's source but I don't suspect writing call logs to the Calendar will be affected. I believe this app uses the Android calendar provider to write directly to the user's selected calendar on their Android device, rather than calling out to the Google API (https://www.googleapis.com/auth/calendar). Even if it did call the Google API, per Google's OATH API Verification FAQ[1] the restricted scopes are only GMail related (https://www.googleapis.com/auth/gmail*).

Suggest the following way forward, in parallel:

1. Author documents meeting Limited Use criteria (see Google FAQ) and applies for a "restricted scope app verification". Author requests waiver of the security assessment ($15K+) or convinces a third-party assessor to complete the assessment without charge. EDIT: This may be an issue. One app type not permitted to restricted scopes are those that "store or backup data other than email messages in Gmail."

2. We start working on a branch that guides users through setting up GMail backups with IMAP. This will require manually generating an "App Password" [2] (avoids 2FA issues). Looks like Google treats labels as folders over IMAP [3] so that should still be possible.

Thoughts?

[1] https://support.google.com/cloud/answer/9110914#restricted-scopes [2] https://support.google.com/accounts/answer/185833?hl=en [3] https://developers.google.com/gmail/imap/imap-extensions

[malversan]

I think your second point is the only direction at this moment. The solution can be to separate the Calendar API access (which is not going to be banned) from the GMail API access, and then use the IMAP protocol to access GMail, as you say. That would require some development work to change a bit how the app works internally, but hopefully not a lot.

The first way you expose is missing the point that the security verification (I´m afraid) has not been denied for technical security concerns, but because it is not clear for Google what has to do with the user emails an app that is not an email app. By reading their verification process requirements, I think they have not bothered in analyzing the app security (the app already complied with XOAuth2 security, and I think we all know that it doesn´t resend any data to any obscure external storage). They simply have looked what the app is for, and then denied the use of the API restricted scope for that purpose. That light and insufficient evaluation is Google´s standard and automated way of doing things. An exceptional treatment should be required to get the app unbanned from the GMail API (and that was I was trying to get).

P.S.: I could be wrong, but I think that $15K+ security assesment is only required by Google for server infrastructures that make use of the Google APIs. An Android app is a simple client application running in an environment already secured by Google (Android), so that shouldn't apply in this case. Otherwise any developer making apps that use any sensitive Google API should be able to afford that expensive audit, and that´s clearly irrealistic.

[jpellman]

Maybe pinging the Googlers who actually have enough interest to star this project will help the ticket @malversan opened gain traction. That seems a reasonable strategy to me.

Hey @kcc, @sio4, @kevincox, @smike, @kynan :