Adding microg Gsm support / switching from websockets ?

[vuyibiran]

Hello @JavaJens and @all

First of all, thanks for this websocket-version. I am running 3.3.1 at the moment and you did great work! Really appreciate it.

To the technical situation:

We all know that it might take a long time to have websocket support integrated into Signal. There is a lot of discussion with the Signal developers but they refuse the integration for several reasons. It's unclear at the moment if it will ever be integrated. A pity but fair enough - it's their decision.

I read a talk on newscombiner the other day where Moxie Marlinspike stated that he would be pleased to see a 'API-compatible open source implementation of play services' 1 to make it possible to use signal with GMS without the need to install GAPPS.

Well, there is now good progress in this direction with the project microG GmsCore. This aims to be a 'framework to allow applications designed for Google Play Services to run on systems, where Play Services is not available' 2.

So here's the idea: Websocket support is great and really helpful - no doubt about that. But what about modifying Signal that it uses microg Gsm as an alternative? Maybe as a compiling option. There would be a higher possibility to have this integrated into the main app. Wouldn't this comply with your goal to create a solution for people that do not have proprietary play store installed? And as far as I can see you are well familiar with adjusting and changing the Signal code - so this might not be a huge thing for you guys.

What do you think about that? I'd really like to read your thoughts about this idea.

Thanks a lot and kind regards.

[JavaJens]

I haven't heard of that project before, so take my answer with a grain of salt. From my understanding though, API compatible and the way the XDA thread is written I don't see anything that needs to be done from an application developer standpoint.

If there is something to do, I have to admit that I currently don't have the time to commit any valuable effort to this project. So I think a separate fork would be the best approach to this.

[patcon]

Interesting. This seems to be the compatibility library that would allow moxie's suggestion to work: https://github.com/microg/android_external_GmsLib

[relyt29]

This guy claims to be running Signal with GmsCore: (From patcon's link) http://o9i.de/2015/10/23/howto-gmscore.html If that's indeed the case, this fork may no longer have a point. I will be so thrilled especially if it does phone calls too. Definitely going to check this out asap

[JavaJens]

@patcon If I read it right, the link you posted would replace the library needed to build the app right? Whereas, using the GmsCore would not require any changes on the App.

I see the following options:

1. GmsCore would allow People without PlayServices to tweak their device as needed
2. GmsLib would allow People without PlayServices to use Signal anyway

Either way, pretty sweet. @f41c0r Please report on how it works!

[patcon]

@JavaJens actually, my understanding from this readme line lead me to believe that GmcLib was a shim that was required in to be compiled into any app in order for it to properly use GsmCore:

It will try to use the Play Services when installed on the target device. If this is not possible, a basic fallback implementation might be used.

But working on some other stuff though, so haven't even skimmed the source :)

[JavaJens]

@patcon I think GmsLib and GmsCore are two different things. Similar to the Play Services, there is a library that needs to be present for building and one that is on the device.

[ddorian1]

I have just tried it. It's working without the need to rebuild Sigal. I have used the instructions at top of this thread: http://forum.xda-developers.com/android/apps-games/app-microg-gmscore-floss-play-services-t3217616

The only problem is, that you either need to patch and rebuild your rom or have xposed installed (and therfore have your device rooted) to use it.

[pejakm]

I'm using microG GmsCore for about a month already, and it works great with Signal (the original version from Playstore). EuphoriaOS for p880 is already patched, so I had no troubles with that. I compiled GmsCore with the heartbeat interval of 5 minutes, instead of 1, so it's a bit easier on battery...

[vuyibiran]

Good to see that there's a discussion going on now ;)

Running microG GmsCore for a few days. It's working (also calls) with signal build from original sources withouthout further modification.

But right, the rom has to be patched or xposed has to be used. So my initial idea (see first post) was to implement a compile option to use the microg GsmCore without the need of patching the rom. If this would be possible at all. But i guess once there is a compiling option available there would be a high possibility to have this integrated into the signal source code.

[relyt29]

I've been looking at this. I personally do not like the idea of using xposed and whatnot, it's not simple enough for my liking. Plus would be nice to support non-root users as well. I have Signal compiling against android_external_GmsLib ,but it crashes at registration time because of a null pointer dereference, so I'll have to look into that. Something gmslib returns is null which then gets dereferenced in signal.

But no I haven't forgotten about this and the potential for it is very exciting.

[JavaJens]

@f41c0r But you also have the GmsCore installed correct?

From my understanding running only GmsLib would not require root on the device. The question which remains is: what is their "basic fallback implementation" ?

[relyt29]

gmsCore I sideloaded, it never asked for root permissions.

[relyt29]

I'm not sure how in love with this I am. MicroG may be FOSS, but its still basically making a bunch of requests to Google all the time. Maybe just keeping a websockets only approach is better, since it limits your requests to only the TS servers.

[h-2]

From my humble point of view microG is a workaround, but this fork would be the fix. Ultimately a workaround is better than nothing, but if we manage to get something where google get no metadata that would be best.

[JavaJens]

@h-2 I think so as well. Especially considering the effort required to get it running on a users device. So for users that just don't have Google Play because they have e.g. an Amazon device, I don't think this is feasible. Like I stated above, if this "basic fallback" they are talking about in GmsLib would work, this could be a solution for that as well.

[relyt29]

According to my understanding, Gmslib is just a compileable library that provides the andriod.gms package for imports and whatnot. It's still going to make calls to google servers.

[psivesely]

Looked into microG GmsCore. Seems like it sends at least your build ID and device name to Google.

Glad you're keeping up the fork! You're the best @JavaJens!

[relyt29]

About a month ago, I looked into this a fair amount. As has been mentioned here by several people, there are privacy implications (if you don't trust google) of using GMSCore. I worked on gmslib integration with TextSecure as well, you can find the work for that on my repo under the "gmslib" branch: https://github.com/f41c0r/TextSecure/tree/gmslib

In order to get this to work I had to compile and install the following apps onto my phone: https://github.com/microg/android_packages_apps_FakeStore.git https://github.com/microg/android_packages_apps_GmsCore.git

And then compile and installed this into the local maven repository using gradle install: https://github.com/microg/android_external_GmsLib.git

I got it to build successfully using gmslib, but since gmslib doesn't implement everything, it sometimes returns NULL for certain function calls, and that caused segfaults at runtime in the textsecure app. Thats as far as I got and then my interest in this started to go down and I stopped working on it. Here is the relevant info from the logcat:

W/TextSecurePreferences( 5905): Setting push registered: false
W/PushServiceSocket( 5905): Push service URL: https://textsecure-service-staging.whispersystems.org
W/PushServiceSocket( 5905): Opening URL: https://textsecure-service-staging.whispersystems.org/v1/accounts/sms/code/+1XXXXXXXXXX
D/GmsGcmMcsSvc( 3487): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsOutput( 3487): Outgoing message: HeartbeatPing{last_stream_id_received=5}
D/GmsGcmMcsInput( 3487): Incoming message: HeartbeatAck{last_stream_id_received=4}
D/GmsGcmMcsSvc( 3487): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsOutput( 3487): Outgoing message: HeartbeatPing{last_stream_id_received=6}
D/GmsGcmMcsInput( 3487): Incoming message: HeartbeatAck{last_stream_id_received=5}
W/RegistrationService( 5905): org.thoughtcrime.securesms.service.AccountVerificationTimeoutException
W/RegistrationService( 5905):   at org.thoughtcrime.securesms.service.RegistrationService.waitForChallenge(RegistrationService.java:278)
W/RegistrationService( 5905):   at org.thoughtcrime.securesms.service.RegistrationService.handleSmsRegistrationIntent(RegistrationService.java:205)
W/RegistrationService( 5905):   at org.thoughtcrime.securesms.service.RegistrationService.access$000(RegistrationService.java:63)
W/RegistrationService( 5905):   at org.thoughtcrime.securesms.service.RegistrationService$1.run(RegistrationService.java:95)
W/RegistrationService( 5905):   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
W/RegistrationService( 5905):   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
W/RegistrationService( 5905):   at java.lang.Thread.run(Thread.java:856)
D/audio_hw_primary( 1895): select_output_device: AUDIO_DEVICE_OUT_SPEAKER
W/PushServiceSocket( 5905): Push service URL: https://textsecure-service-staging.whispersystems.org
W/PushServiceSocket( 5905): Opening URL: https://textsecure-service-staging.whispersystems.org/v1/accounts/voice/code/+1XXXXXXXXXXX
W/InputMethodManagerService( 2242): Window already focused, ignoring focus gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@40f52c78 attribute=null, token = android.os.BinderProxy@4154eb30
D/GmsGcmMcsSvc( 3487): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsOutput( 3487): Outgoing message: HeartbeatPing{last_stream_id_received=7}
D/GmsGcmMcsInput( 3487): Incoming message: HeartbeatAck{last_stream_id_received=6}
D/audio_hw_primary( 1895): select_output_device: AUDIO_DEVICE_OUT_SPEAKER
I/dalvikvm( 5905): Jit: resizing JitTable from 4096 to 8192
W/PushServiceSocket( 5905): Push service URL: https://textsecure-service-staging.whispersystems.org
W/PushServiceSocket( 5905): Opening URL: https://textsecure-service-staging.whispersystems.org/v1/accounts/code/809630
D/dalvikvm( 5905): GC_CONCURRENT freed 1314K, 14% free 10159K/11724K, paused 3ms+7ms, total 55ms
W/TextSecurePreferences( 5905): Setting push registered: false
W/InputMethodManagerService( 2242): Window already focused, ignoring focus gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@4162d740 attribute=null, token = android.os.BinderProxy@4154eb30
W/CanonicalAddressDatabase( 5905): Hitting DB on query [ADDRESS]
W/PushServiceSocket( 5905): Push service URL: https://textsecure-service-staging.whispersystems.org
W/PushServiceSocket( 5905): Opening URL: https://textsecure-service-staging.whispersystems.org/v2/keys/
W/dalvikvm( 5905): threadid=22: thread exiting with uncaught exception (group=0x40bd3930)
E/AndroidRuntime( 5905): FATAL EXCEPTION: pool-4-thread-1
E/AndroidRuntime( 5905): java.lang.NullPointerException
E/AndroidRuntime( 5905):    at org.thoughtcrime.securesms.service.RegistrationService.handleCommonRegistration(RegistrationService.java:247)
E/AndroidRuntime( 5905):    at org.thoughtcrime.securesms.service.RegistrationService.handleVoiceRegistrationIntent(RegistrationService.java:166)
E/AndroidRuntime( 5905):    at org.thoughtcrime.securesms.service.RegistrationService.access$200(RegistrationService.java:63)
E/AndroidRuntime( 5905):    at org.thoughtcrime.securesms.service.RegistrationService$1.run(RegistrationService.java:97)
E/AndroidRuntime( 5905):    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
E/AndroidRuntime( 5905):    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
E/AndroidRuntime( 5905):    at java.lang.Thread.run(Thread.java:856)
W/ActivityManager( 2242):   Force finishing activity org.thoughtcrime.securesms/.RegistrationProgressActivity
W/WindowManager( 2242): Failure taking screenshot for (328x583) to layer 21040
D/dalvikvm( 2242): GC_CONCURRENT freed 3713K, 44% free 12544K/22260K, paused 9ms+9ms, total 76ms
D/dalvikvm( 2242): WAIT_FOR_CONCURRENT_GC blocked 56ms
E/BufferQueue( 1892): [org.thoughtcrime.securesms/org.thoughtcrime.securesms.RegistrationProgressActivity] drainQueueLocked: BufferQueue has been abandoned!
D/LightsService( 2242): Excessive delay setting light: 66ms
D/LightsService( 2242): Excessive delay setting light: 66ms
D/audio_hw_primary( 1895): select_output_device: AUDIO_DEVICE_OUT_SPEAKER
I/Process ( 5905): Sending signal. PID: 5905 SIG: 9
I/ActivityManager( 2242): Process org.thoughtcrime.securesms (pid 5905) has died.
W/ActivityManager( 2242): Scheduling restart of crashed service org.thoughtcrime.securesms/.service.KeyCachingService in 5000ms
W/ActivityManager( 2242): Scheduling restart of crashed service org.thoughtcrime.securesms/.service.RegistrationService in 15000ms
D/GmsGcmMcsSvc( 3487): Heartbeat initiated, reason: Intent { act=org.microg.gms.gcm.mcs.HEARTBEAT flg=0x4 cmp=com.google.android.gms/org.microg.gms.gcm.McsService (has extras) }
D/GmsGcmMcsOutput( 3487): Outgoing message: HeartbeatPing{last_stream_id_received=8}

Seeing as I think this is enough research into the topic and I think the general consensus is that we don't want to go in this direction, I'm going to close this issue.

[psivesely]

https://github.com/microg/android_packages_apps_GmsCore/blob/master/play-services-core/src/main/java/org/microg/gms/gcm/RegisterRequest.java#L58 could be hardcoded to a fake value for everyone.

[mimi89999]

https://github.com/LibreSignal/LibreSignal/issues/37

[Diapolo]

Just wanted to comment, Signal IS working great with https://microg.org!

[psivesely]

I couldn't get push notifications to work on microg, despite following all their recommendations. I suppose I'm guilty of being busy with other things and not wanting to debug it and try to help upstream figure out what might be going wrong with my ROM/ Carrier/ etc. I feel like I probably need to go back to it because Signal voice has become a work necessity and it doesn't look like LibreSignal is ever going to support that.

[mimi89999]

You may need signature spoofing.

[psivesely]

I spoofed. I'm telling you it wasn't me not following any instructions or troubleshooting tips in the wiki. I'll probably switch back to microG sometime in the future and maybe the issue will be resolved or I'll file a proper bug report. Anyway, this is derailing the thread, even though it's already closed.