Websocket development stalled?

[xmikos]

Hello, please what is current status of Websocket support in TextSecure? Is your fork dead (there hasn't been any activity for a long time)? Are you planning to continue and merge it into official TextSecure?

Thanks for your answers.

[JavaJens]

Hey, there wasn't much enthusiasm on the receiving side and interest has seem to stalled. So, yeah I guess you can call this fork dead. However if there are people willing to contribute and I see a reason to continue work, I might pick it up again. Also contributions are always welcome and if you have a PR or issue please post and I'll take a look at it

[Isgar]

I would like more developement here... But honestly, I still have a version with SMS support and it just works. So I have little motivation looking into contributing atm.

[xmikos]

For SMS encryption there is actively developed fork named SMSSecure (https://smssecure.org). But what I am missing is full internet-based TextSecure version which is independent on Google Cloud Messaging. But I am afraid that my Android/Java knowledge isn't good enough to continue in development of this fork :-(

[xmikos]

@JavaJens What is current status of this fork? Does it still work? What is missing or need to be fixed? I can at least try...

[relyt29]

@JavaJens you should put up some kind of bitcoin donation type thing so users can throw money at you to motivate you to stay current with upstream ;)

[JavaJens]

@f41c0r I don't want any money, I just want to see that people care about this feature and maybe even that it gets added to master someday.

But it seems as if there is some interest, so I will pick it up again. However I'm on vacation right now.

@xmikos That's the next problem. I never had to use my branch ;) Feel free to test and report back. When I'm back I will load my old phone with this branch for testing.

[xmikos]

@JavaJens When will you return from vacation? :-) I am creating independent public F-Droid repository for TextSecure and RedPhone right now (I am awaiting Moxie's wrath when I publish it ;-)) and would love to include WebSocket-based fork in it (I hope Moxie will merge it one day, but from discussions with him that I have read, I am not completely sure about it).

[xmikos]

I have already published my TextSecure & RedPhone F-Droid repository: https://fdroid.eutopia.cz

[b-meson]

@f41c0r and I are working on bringing the code up to the latest TS master and willing to help. We want to help as well. Personally I don't like using f-droid and running unsigned, untested code, particularly against production servers. Let us know how we can help @JavaJens

[jomo]

I'm very looking forward to this. TextSecure seems to be the only messaging app that gets encryption right, I'd like to use it but I don't have Google services installed. I probably can't contribute any code here, though. :disappointed:

[relyt29]

Hey @JavaJens, What is the websockets-reborn branch? Was that an attempt in April to update the websockets fork? Also are you aware that Koush deprecated the android websockets library in favor of this library?

[dev-mb-zz]

Hey Jens, first of all thanks for your work. I just wanted to confirm that there is indeed a huge interest in a GCM/google free version of textsecure. Is there any overview what works well, unreliable and not at all yet? From July on I will try do build, test and write bugreports in case I find some. Best regards an thanks again!

[xmikos]

I also want to make some test builds, but I don't know which branch should we preferably use? Master or websocket-reborn? It seems that websocket-reborn is much newer, but is it complete?

[black-puppydog]

Just chiming in to confirm there is indeed interest for this feature. I haven't been able to use TextSecure for anything actually secure since January or so when I went Google-free on CM. I am completely dependent on the websocktes transport as virtually all of my peers in Germany or Sweden, with me being in France.

So far I have been running a self-built version of TextSecure (built on my now-deleted Ubuntu install) to work around Google Play. I am now running Debian and reluctant to install the SDK there but I can make an extra partition for it. So I would be both motivated and able to test the resulting packages, but unable to contribute since I don't know anything about Android development.

@JavaJens Have a nice vacation! :)

[patcon]

+1 @JavaJens! :)

@xmikos I'd much rather not fragment the space, but if there are enough supportive people and merging doesn't look likely, this might be a good exercise in attempting deterministic builds: https://f-droid.org/wiki/page/Deterministic,_Reproducible_Builds

You could have a trusted member of the fdroid community confirm the tagged builds

[JavaJens]

The websocket-reborn branch was an attempt to get up quickly against the upstream master. In my first version I included an external websocket library and dealt with a lot of ugly wakelocks etc.

In the reborn branch all of this work was no longer needed as the PlayStore version of TS included Websockets already for fetching the messages. The way this worked was that GCM would trigger a WS connection to download messages and this connection was held open as long as the app was in the foreground. IIRC this behavior was changed afterwards again.

I will try to get this branch updated with upstream master asap. Regarding feature completeness, I didn't know of anything that didn't work back then :)

[witchent]

First of all, thanks for that awesome work already done. I'm one of those who think that installing gapps for having a secure text messaging app isn't really a good deal. I also had to many problems with kontalk etc, so I couldn't ask my friends to move. And while telegram was a good compromise, I still feel the need to move to a complete free(as in freedom) messaging program. While I don't really understand all of Moxies motives for being so negative about everything that "frees" us from google (fdroid, websocket etc) it is of course his decision to not include websocket and to ask for the removal of textsecure from fdroid. Nevertheless reading 124 and 1000 was aweful, cause now I'm afraid that even if you manage to get everything working (again), moxie just might decide to block websocket/fdroid off. I'm not experienced at all in java code (they told me c++ is the way to go), but if you decide to revive the project I would love to test it and help as good as I can to debug eventual bugs. I can't build it myself in the moment as I'm overseas, but as soon as I get access to my old computer I will be ready. Sorry for my English, and I don't wanna insult anyone, so please don't feel offended @all.

[JavaJens]

I have updated the reborn branch using "push -f". I apologize for that, but feel free to check it out. I've also added a WS flavor which can be configured to test the Websockets only approach on a device that has PlayServices installed.

[relyt29]

Ugh, we're shit out of luck, if they're using GCM for websockets then we're back to requiring play services which puts the Free Software community in between a rock and a hard place. Either you value your privacy and you use textsecure, or you value your freedom and you abstain from textsecure, essentially leaving you with no good implementation of strong crypto avaliable. (What alternative are you going to use? OTR? not asynchronous. PGP? LOL good luck getting your friends onboard with that!)

I think for a lot of us what drove the desire to see success for the websockets branch was that it would present an alternative to GCM, the original branch you wrote in January by pulling in android-websockets library can be built without pulling in the Play Services dependency at runtime, and so can be used on Replicant without Play installed.

Now if upstream has a websockets implementation working they'll never have any incentive to merge a non-Play Services implementation.

Alternatively, the Free Software community could try to bootstrap a library like you did originally, and then somehow do conditional statements to use the Free Software library to communicate over the network instead of Google Cloud Messaging (ton of work, and then you have to mantain the fork, merging in upstream changes on a regular basis), or write a drop-in replacement for GCM that exposes the exact same API, but doesn't require any closed source code.

:(

[xmikos]

@f41c0r I don't understand your long rant. TextSecure for Android is using their own websocket implementation/library: https://github.com/WhisperSystems/libtextsecure-java/tree/master/java/src/main/java/org/whispersystems/textsecure/internal/websocket (which in turn uses okhttp library: https://github.com/square/okhttp). TextSecure servers doesn't need GCM, there is even browser-based TextSecure client in development: https://github.com/WhisperSystems/TextSecure-Browser, which of course doesn't use GCM, only websockets alone.

Problem is that TextSecure client for Android still needs GCM for push messages and this is what JavaJens solves with his fork.

[witchent]

I think the problem being is that the developer of TextSecure apparently does not value freedom as he does security. He asked the F-Droid maintainer to remove his app, he did not (from what I could see) help the development of the websocket implementation but he did rather hinder it by declining all effort already done. Therefore for me it actually seems not completely unlikely that even if this Fork would do everything upsteam does (i.e. work together with the main server) but without GCM, maybe gets included into F-Droid, that moxie/whispersystems might just think it's safer (read: easier) to just remove/alter the websocket implementation, rendering this fork useless. So I do agree with the underlying bitterness of @f41c0r I think Franklin's famous sentence is partly true in software, too.

[JavaJens]

@f41c0r I'm not quite sure what you mean. Iirc the old branch also required play services for building, sth. where some people said could be solved by using build flavors. But I'm not sure if anyone actually did that.

So I think the real solution would be to completely polish this branch (sth the old one never was), completely test it and then submit a pull request upstream. Only then can we argue about the success here.

I hope that by removing all the kinks, this work can be used upstream and on a quality level move far beyond my first attempt. I'd appreciate any help in testing and reviewing the code before initial submission.

[relyt29]

So my earlier post was before I got a chance to go look at the code, I was under the misunderstanding that GCM implemented websockets as an API in addition to push messaging, and that that's what upstream had chosen to go with in their websocket implementation. Also apologies if it came off as a rant - not trying to rant or be obnoxious or annoying. Sorry.

My primary concern is also not have to require Play Services at runtime - which both reborn and master do. I misunderstood what was going on in my previous post, sorry. Obviously it would be ideal to not have to require Play Services at compile time, but, baby steps. There is a replicant SDK which can supposedly be used to build APKs without requiring any closed source software in the build process but it won't work for textsecure obviously with the various play services dependencies.

I have the websockets reborn branch built and running on replicant without google play services right now. It seems to be working fine thus far.

@JavaJens a) is there any way we can help beyond just using the fork? b) Do you intend to be keeping reborn updated to upstream? Do you want help merging in those commits on a semi regular basis if you do?

[relyt29]

I wrote up a build guide here

[JavaJens]

@f41c0r a) 1) I think the best way to help is to test that everything works. 2) Reviewing the code to make it production ready b) If there is ongoing response that everything still works and people are interested in it, I sure will. The merge conflicts are usually very small, so this should be fairly simple. But a code review never hurts.

For the next step I will look into writing some tests to further strengthen the quality, any help here is appreciated as I haven't used either Mockito nor Espresso.

[ddast]

Thank you so much for the update @JavaJens. I installed it yesterday on a test device without gapps and sent some test messages to this device every now and then. All messages have been received without delay. However, I use only WiFi on this test device.

Since this worked flawlessly I will now also install it on my main device to test it more thoroughly.

[black-puppydog]

thanks @f41c0r for the writeup, much appreciated! I stumbled a bit because the SDK installed by default was 22, but libtextsecure expects 21. (I used Ubuntu make to install Android studio, so this might not generalize to people who install it by hand) I am now running this on my production phone, so I'll see how it goes. Import of old data and sending worked perfectly, so did the SMS confirmation step, So that's encouraging. :)

Thanks you @JavaJens for your work, this is really great!

[patcon]

fwiw it worked beautifully on CM11 device without gapps (running through orwall/orbot as well, using a mobile wifi hotspot), and I haven't noticed any delayed messages so far. This is really really amazing @JavaJens, and I'm incredibly appreciative (also toward whoever at whispersystems worked on the serverside code!)

[black-puppydog]

I have been busy catching up with some people, it works beautifully, though I stayed at home, so on wifi :P

[patcon]

@JavaJens Can you create a PR from the branch so we can make inline review comments? (I'll do it in commits for now)

[jensMF]

Thanks for the great work you do and for the updates.

I have one question now: Against which server is it running? And can I change it?

[JavaJens]

It will build against both staging and production. So you can choose which flavor to run.

[jensMF]

Ok thanks for the fast answer. At the moment, I can't build at all. I use AndroidStudio and if I select "Build Signed APK..." I have to choose "Build Type" but there is none...

[JavaJens]

Try building using ./gradlew build and if that fails open another issue with the error log.

[jensMF]

Ok, it did work with AndroidStudio and flavors now... I had to "Sync Project with Gradle Files" and everything was ok. Thanks for your Help.

[olebowle]

Wow, thanks @JavaJens. It works beautifully! Much appreciated. And of course @f41c0r for the build guide.

[JavaJens]

Thanks to @f41c0r the websocket branch gets all the latest updates frequently. But how do you propose to get a pull request going?

I'd suggest to create a dedicated branch on which we will rebase against upstream master to have the cleanest comparison, but I am a novice regarding long running changes. On this branch I'd like to request all kind of feedback regarding code style and quality. Basically performing an "internal" code review before submitting this branch for a pull request. But any input is highly appreciated. Please see issue #21 for discussion.

[igoralmeida]

I was under the impression the feature/websocket-reborn and https://github.com/JavaJens/TextSecure/wiki/Building-the-Websockets-reborn-fork did not require Google Play Services to build the app. Is that not the case? Following that wiki page I couldn't get past "Run android studio and open textsecure as a project", and the log tells me something similar to what './gradlew tasks' gives me:

      > Could not resolve all dependencies for configuration ':compile'.
      > Could not find com.google.android.gms:play-services-base:6.5.87.
        Required by:
            :TextSecure-JavaJens:unspecified
      > Could not find com.android.support:gridlayout-v7:22.2.0.
        Required by:
            :TextSecure-JavaJens:unspecified

The last one I'm pretty certain I can fix by updating my android SDK environment, but what about Play Services? Am I missing anything or do I have to install it to build websocket-reborn? If all we're missing is some gradle- and java-fiddling to get rid of the dependency, I'd be happy to help.

[relyt29]

It doesn't require play services at runtime, it'll run on a phone without gApps, but it is required at compile time. I put in the list of things you need to install from the sdk manager in the instructions. You should be ok, if you just get the dependencies in the sdk manager

[igoralmeida]

Removed a few references to play services throughout the code and managed to build with assembleProdWebsockets. Now I need to test with someone else, will report back then.

This of course will never get merged, but maybe we could have a parallel websocket-reborn-noplayservices branch for building.

[igoralmeida]

Tested and worked fine between a gapps-free Nexus4 phone and a stock Xperia Z2 (installed TextSecure from google play), both on the same wifi network. Neither received the confirmation SMS (I had to use the 'call me' method for activation), but this may be unrelated.

[witchent]

Compiled and tested on a gapps-free Nexus 4 with an AOSP-Rom. Worked like a flaw, got the confirmation SMS and was able to communicate with a friend who's using the official version. Will watch the battery usage, but that's really awesome , thanks a lot.

[jf-h]

I have been testing the websocket-reborn branch for nearly two weeks now, and all I can say is that it runs surprisingly smooth on my Nexus 5 CM12.1 w/o GApps. Battery drain is fine, though, comparing it to siacs/Conversations, TS w/ WS falls a little bit behind. Still, no biggie for people (like me) who are used to constant connections and frequent polling. Remembering the infamous issues https://github.com/WhisperSystems/TextSecure/issues/1000 and https://github.com/WhisperSystems/TextSecure/issues/127 raised at upstream, I didn’t hope to see a working WS version all too soon. Seems like things changed for the better after 2.6.0. My thanks to everyone who is working on a GCM free version of TS.

[Letty]

thank you for your great work on replacing gcm with websockets. now i can use textsecure on my blackberry (os 10.3.1). the registration via sms is not possible (you recive the sms but ts can't access it), just used the call option and it works fine. as with other android apps, i can't access my contactlist. but thats ok. you can type the number and messages where send. battery usage seems normal so far.

if one of you guys is at cccamp, i will buy you a beer! ;)

[jollcob]

Thanks also from the Jolla/Sailfish OS users! Even accessing contacts is working here.

A lot of people there are interested in TextSecure (https://together.jolla.com/question/11534/sailfish-redphone-and-textsecure-apps/?sort=latest#sort-top). Since there is no real support for GCM we benefit a lot from this development and may get a native app at some point. Please keep up the good work, I hope there will some help from the Sailfish community!

[llelectronics]

A patched WebSockets version (see patch here: https://github.com/llelectronics/textsecure-jolla-edition-patch ) called "Jolla Edition" landed in the official Jolla Store. The patch only skips SMS verification on Jolla and BlackBerry devices as it isn't working there.

Thanks for your great work :)

[dejIO]

@JavaJens Thank you so much it's working great! I'm using your branch with the production server on CyanogenMod 11 without GApps. I've been waiting ages to use TextSecure as it's asynchronous (as opposed to XMPP + OTR with Conversations which I usually recommend).

Messages arrive instantaneously on Wifi, I have yet to test it on mobile data. Contacts show up fine and registration was flawless.

[mad-de]

Hey @JavaJens as you have pointed out, that the issue with the battery drain might be fixable on the server-side, what do you think about a standalone TextSecure-Websocket fork with it's own server? I know that there is a huge demand for a non-google-apps versions, yet the devs seem to be unwilling to tackle that issue. However the CM crew has set up their own servers, which work togethe with the TS servers. Would that be an idea for your fork?

[relyt29]

Federation with the official server would require cooperation from OWS/moxie so if we were to do that all websocket users would only be able to talk to people on our separate server.

it is, however a decent idea to do that for testing to see if it can measurably affect battery drain.

[mad-de]

Depends. Maybe moxie & co would be quite happy to split the server costs in a long run + they could focus on stability and features for the main product, while the WS branch can offer non Google experience etc.