F-Droid Version?

[Hoeze]

Hi, I accidentally stumbled over this app. As it is an Apache-licensed open source app, it would be cool to see it in the F-droid store.

Currently it is hard to find this app / this repository. Probably the "atalk.org" domain should be registered too...

[cmeng-git]

I am more than happy to release aTalk into F-Droid, so more people can benefit from the effort that I have put in. Appreciate if you can provide me the link on how to release an apk into F-Droid.

The registration and maintenance of a domain name requires a yearly financial commitment. I certainly would like to register atalk.org domain; however I am only an individual and does not get any financial support, hence have to do thing within my capacity. Currently I am using the free no-ip service with a domain name atalk.sytes.net that links to my home server. It has an online help for aTalk application. You may visit the site to find out more.

[Hoeze]

How to get your app into F-droid: https://f-droid.org/en/docs/FAQ_-_App_Developers/#how-do-i-get-my-app-included

@ financing: You could try to put a donation link into the readme. A .org domain needs ~ 10,50€ per year (e.g. namecheap) which should be cheaper than play store developer access, so achieving this amount of donation should be no problem if your app gains some popularity.

BTW: I'm excited how fast your home server is :)

[cmeng-git]

My home server setup is only meant for the atalk development and testing. http://www.giadapc.com/products/minipc/cube%20series/D2308.html It is an i7 system with 16GBytes RAM, 500GB harddisk with a 4K 32" display. System OS is Ubuntu 16.04, ejabberd xmpp, LAMP servers and running android studio all in one.

[IzzySoft]

@cmeng-git to get you started with this, I've just filed the corresponding RFP (Request For Packaging). You might wish to watch that one in case the maintainers need some input/help from you. Good luck!

[IzzySoft]

@cmeng-git there seem to be some issues with the JNI stuff. Could you please check via the link in my previous comment? Thanks!

[cmeng-git]

I have read your previous comment, However I am not sure what exactly is the problem. aTalk uses some pre-compiled static libraries for optimized performance. For the aTalk development, I use Android Studio 3 on Ubuntu 16.04. The whole build process uses either the included pre-built static libraries or are built by the NDK as defined in build.gradle file. Any pre-built static library are done outside AS using android NDK toolchains. aTalk uses some external pre-built jar libraries; some to be included in the final aTalk.apk and others are used as tools to update some of the jar libraries default paths. aTalk also uses plugin sources with my own modifications in order to work on aTalk. If you refer aTalk/build.gradel, may be you understand how all these are being used in final aTalk.apk.

However I found that all the above are being flagged as error in F-Droid sanitary check.
Does F-Droid not allow the inclusion of any static libraries, jars etc?

[IzzySoft]

Does F-Droid not allow the inclusion of any static libraries, jars etc?

That's it, exactly. All must be build from source – at least initially; on subsequent runs, the libraries build on a previous run can be re-used (to speed up the build).

[cmeng-git]

I will try to explain the various static libraries and jars used in aTalk build process.

1. aTalk/libs You may delete all the files in this directory. The sub-directory contains the results during AS build process.

2. aTalk/jni//android Please refer to ./static_library_built for instruction on building these static libraries from source. Currently I am trying to move this information into the cmeng-git as individual project for easy tracking. see ffmepg-android project.

3a. aTalk/lib/installer-exclude The sub-directory contains only jar tools use by aTalk/build.gradle during the build process. They are not included in the final aTalk.apk

3b. aTalk/lib/bundles The sub-directory mostly contains all the jar libraries from jitsi. These libraries are mostly downloaded from maven repository. However aTalk cannot use these jars directory, as they may contains modules that are in conflict with android platform, and hard coded paths which do not render easy inclusion in final aTalk.apk. You may refer to aTalk/build.gradle subsection for more information.

• task jarRepackage(type: Copy, description: 'Repackage jar to avoid conflict') You may build these jars from source by downloading them from maven or jitsi github repository.

4. gradle/wrapper/gradle.properties The gradle/wrapper sub-directory is created by AS during first installation and built. However as the gradle build tools are upgraded by AS update over time, it requires changes to the line
   • distributionUrl=https\:// services.gradle.org/distributions/gradle-4.1-all.zip I am not sure how to help on this.

[IzzySoft]

That explanation would have been needed over there in the F-Droid issue. I'm not a dev nor an F-Droid maintainer, so I cannot do anything with those details. I've left a note for the maintainers that those details can be found here.

[susobaco]

I don't understand much either, but from what @cmeng-git explains, I think it would be easier to create a repository and add it to the f-droid client. https://www.f-droid.org/en/tutorials/create-repo/

[IzzySoft]

@susobaco depending on the .apk size, for that it would suffice to attach the .apk to the corresponding release here in releases/, so my updater could pick it for my repo :wink:

[cmeng-git]

Currently aTalk release is signed using google managed key. This key is used and reflected in the Geo-Location service API. If you signed aTalk with another key, the Geo-Location feature will not be working; unless F-droid apk creator wants to request a new google Geo API key and manage this requirements as well. Note also that device pre-installed apk from android playstore, cannot update his aTalk app from F-droid. There is a conflict with the signed certificate between the two versions.

I am able to download the signed apk from android Developer Console. would it be alright o use the same apk and release in F-droid repository. The apk size is around 28 MB currently.

[susobaco]

I'm sorry if I don't understand. But I think it would be good to "free" yourself from google services and try to use more ethical services like openstreetmaps. Kontalk (And I think Conversations is working on it, too) if you have the option to send a geolocation via google maps and openstreetmaps. Would it be very difficult to implement something like this?

https://github.com/siacs/Conversations/pull/2867

https://github.com/kontalk/androidclient/blob/5b1a291b09a0ac6dbbe2bdacc1214c25b1e7398c/app/src/main/java/org/kontalk/position/SendPositionOsmFragment.java

[IzzySoft]

@cmeng-git F-Droid man repository wouldn't accept "binary input" (neither would it "request Google keys"). And while I accept binaries for mine, there's a size limit of 20M per app (incl. multiple versions), as it runs on private space (which is not unlimited).

However, I fully agree with @susobaco concerning the "location service": why not using OSM? That would solve the issue with F-Droid, and free your app from a proprietary dependency.

[cmeng-git]

Thanks you for both of your inputs. I really would like to get atTalk to release into F-droid. However at this moment I really do not have the capacity to support another variation of aTalk.

[IzzySoft]

@cmeng-git We just figured F-Droid can NOT take your app in as it currently stands (see the above mentioned RFP from this post on. Neither can I ship it via my repo as you don't provide an .apk here – and it seems even if you did, it would exceed my limit of 20MB per app.

Things could change if you were able to provide an "F-Droid compatible Gradle flavor" without GMS. Also I'm curious what for an XMPP messenger needs MS Exchange?

[cmeng-git]

If you are referring to the README.md in ews-android-api plugin. actually only certain functions are used in the formatting and displaying the SSL certificate under "Settings... | SSL Certificate", the MS exchange API is not used at all.

GMS is an unique feature for aTalk, removing GMS will only save around <800K of apk size. Major apk size contributors are the native .so libraries (30%), which I am working on. However come next year, it is required for android apk to also support 64-bit lib, This will again increase the size.

I believe F-Droid 20MB size limit will be a challenge for aTalk to fulfuil, although I am constantly looking at way to reduce the apk size.

However the major issue is, personally I really do not have the resource capacity to support another variant for aTalk. aTalk still needs a lot of effort to maintain, even just to keep pace with android OS upgrade new requirements.

[IzzySoft]

Thanks for your fast response!

Just to clarify: my asking towards MS Exchange and GMS was not for size reasons: those are proprietary components not accepted by F-Droid. The size limit is not F-Droids but just with my repo, which runs on private space I need to budget (no organization behind that). My repo often serves as "stepping stone" for apps not yet fully ready for the official repo (e.g. if you'd be working on an F-Droid compatible flavor, and estimate that will take a while – then until it's ready, the app could be served with me).

I understand that (at least currently) you have not the capacity to maintain "another variant" (I'm no dev – but wouldn't setting up a gradle file for a variant without GMS/Exchange be a one-time task, and afterwards only require very little "permanent maintenance"?) But should you manage coming close (or even below) the 20M size, be very welcome to approach me again to serve aTalk via my repo.

[tuxayo]

Would a patch be accepted that makes the use of GMS or MS Exchange optional? Only of course if it's maintainable without too much trouble.

The idea is to get an official encouragement in the case if someone would like to start working on this.

[cmeng-git]

My only concern is the version released for F-Droid and Google Play Store would have different feature sets. What would be the user reaction to this when he realizes this. If user decides to move, there is no direct upgrade path from F-Droid to Google version without uninstalling the former.

By the way, is there any specific reason that an android user will use F-Droid instead of Google Play store installer?

[Rudloff]

My only concern is the version released for F-Droid and Google Play Store would have different feature sets.

We can explain that in the F-Droid description. This would not be the only the app to do that so I think F-Droid users are used to that (as long as it is clearly explained).

If user decides to move, there is no direct upgrade path from F-Droid to Google version without uninstalling the former.

The solution to this is reproducible builds but it is a lot of work to setup.

By the way, is there any specific reason that an android user will use F-Droid instead of Google Play store installer?

Google Play is not FOSS and is not available on open-source Android distributions (Lineage OS, Fairphone Open, etc.). It is also not available in several Android forks (Amazon Fire OS and most Chinese forks like MIUI).

[streaps]

I always look in F-Droid first, just because there are so many crappy applications in the Google Play store and it's harder to find the open source apps.

If aTalk were in F-Droid, I would have discovered it earlier.

What features do need GMS?

[cmeng-git]

Please refer to the following link for more info on this feature

StreetView & map http://atalk.sytes.net/faq.html#feature_02

[IzzySoft]

@cmeng-git Thanks for the links – but they don't explain what you need GMS for. In case it's about GCM/FCM, you might be interested to read How Tutanota replaced Google’s FCM with their own notification system – but going by the permissions requested by your app, it must be something different.

[cmeng-git]

aTalk does not explicitly include GCM service; it may be because aTalk uses GMS service that requires the gcm service for it to operate properly.

[IzzySoft]

Would be good to find the culprit – as else F-Droid cannot build your app. If it uses any GMS service, that's a show-stopper here (for F-Droid that is).

[zz-ha-dum]

constantly looking at way to reduce the apk size.

break apart functuonality like MAXS does?

https://github.com/ProjectMAXS/maxs

use F-Droid instead of Google Play store installer?

I have deleted google bloatware for privacy, performance, and battery lasting.

Google play refuses to be intellectually honest labeling apps.

• free
• adware
• commercial

But it would be harder to garner informed consent for adware installation. A culture of lies and theft is deemed easier.

[IzzySoft]

Google play refuses to be intellectually honest labeling apps.

Even worse, it refuses to honestly list an app's permissions and whether it contains any trackers. Plus it tracks its users. Which is why on my new device I just installed, I this time even resisted the temptation to activate GCM with microG – so for the first time, my device runs completely free of Google proprietary stuff. Reasoning is the same as Morden … urn, @zz-ha-dum gave.

F-Droid is much more honest about that (and clear of that crap) – which is why we want to convince you, @cmeng-git :innocent: I rather stick with F-Droid, which now is even baked into my ROM ("LineageOS for microG" comes with F-Droid pre-installed, which even makes auto-updates possible).

[cmeng-git]

I have modified aTalk source (yet to be released) and removed aTalk dependency on plugins:ews-android-api and its library source. I have checked and found that aTalk does not use GCM/FCM at all.

I see there is no simple way to replace gms feature as aTalk uses many of its features as shown in the source file imports: import com.google.android.gms.maps.CameraUpdateFactory; import com.google.android.gms.maps.GoogleMap; import com.google.android.gms.maps.GoogleMap.OnMarkerDragListener; import com.google.android.gms.maps.OnMapReadyCallback; import com.google.android.gms.maps.OnStreetViewPanoramaReadyCallback; import com.google.android.gms.maps.StreetViewPanorama; import com.google.android.gms.maps.StreetViewPanorama.OnStreetViewPanoramaChangeListener; import com.google.android.gms.maps.SupportMapFragment; import com.google.android.gms.maps.SupportStreetViewPanoramaFragment; import com.google.android.gms.maps.model.BitmapDescriptorFactory; import com.google.android.gms.maps.model.CameraPosition; import com.google.android.gms.maps.model.LatLng; import com.google.android.gms.maps.model.Marker; import com.google.android.gms.maps.model.MarkerOptions; import com.google.android.gms.maps.model.StreetViewPanoramaCamera; import com.google.android.gms.maps.model.StreetViewPanoramaLocation;

aTalk GeoLocation implementation does not expose any of its gms interfaces to outside world. Sharing of location is not possible unless it is triggered by the users themselves.

Actually google has its own applications installed on android devices by default that will monitor user visited places. I do not see any reason why gms features need to be omitted from aTalk, as user privacy is not being compromised, unless user wishes to do so.

If Fdroid still have concern, I can include a GeoLocation option (default to off) to allow user to enable/disable so he/she would not accidentally trigger the gms location sharing.

To understand aTalk GeoLocation implementation, please visit. StreetViews & Map

There is also a demo video on the main page

[IzzySoft]

@cmeng-git it still imports a lot of proprietary stuff (GMS), which is a show-stopper for F-Droid – where apps must be built completely from FOSS components' sources. Apart from that: How can you tell that "user privacy is not being compromised" if you cannot analyze the code? For example, how do you know that com.google.android.gms.maps.model.StreetViewPanoramaLocation or com.google.android.gms.maps.GoogleMap do not transmit the user's location to Google servers?

"Actually google has its own applications installed on android devices by default" – not on mine. And there are many others using custom ROMs without GApps.

TL;DR: as long as any proprietary components are inside, the app cannot be listed at F-Droid.

[cmeng-git]

Thanks for the clarifications for me to understand better. Unless aTalk removes gms and all its features, look like there is no way for it to be listed in F-Droid.

[IzzySoft]

Not in the official repo, correct. But see above: that's no show-stopper for my repo (if it's only that). I reserve up to 20M per app; if your APK doesn't exceed that size, all you'd have to do is to attach it to the corresponding release (and let me know the first time you've done so; after I've set it up on my end, my auto-updater would fetch them from future tags without "human intervention").

But also again: if it's just for GCM, see the Tutanota example. As for maps, there's always OSM.

[cmeng-git]

In aTalk version 1.6.0 release (20181026), I have modified the project source structure to support productFlavors built. The product built for F-Droid has excluded all closed source and proprietary libraries in the final apk generated.

For the F-Droid release build, GMS services has been removed hence its Geo-Location feature support. Any community contribution to support the feature using OSM is most welcome.

[IzzySoft]

Thanks! I've updated the RFP and asked @Rudloff to give it a try.

[licaon-kter]

Are these build too: https://github.com/cmeng-git/atalk-android/tree/master/aTalk/lib ? They need to be build from source for F-Droid or loaded from maven... not JARs :(

[licaon-kter]

@cmeng-git ^

[cmeng-git]

Explanation for aTalk static lib jar:

1. lib/bundles: These libraries belong to Jitsi, Most of them are not released to maven repository. The jar's are downloaded from jitsi internal repository. Also some of the jar defined paths and even source need to be modified to work in aTalk. Refer to build.gradle module: task jarRepackage(type: Copy, description: 'Repackage jar to avoid conflict'). If F-Droid needs to build from source, then the respective version source have to downloaded from jitsi repository and perform build manually.

Note: some of the jar module has been stripped of some modules and replace with source in aTalk src path. Ice has problem in aTalk which Jisti team is not willing to modify, I need to patch to make it work. see build gradle:

        // ice4j-2.0.0 work only with AWS disabled - otherwise hang in AWS EC2 conn.getContent()
        // System.setProperty(MappingCandidateHarvesters.DISABLE_AWS_HARVESTER_PNAME, "true")
        // Objects class only available in API-19 and above
        jarjar2(jarfile: 'libs/ice4j-2.0.0.jar') {
            zipfileset(src: 'lib/bundles/ice4j-2.0.0.jar')
            rule pattern: 'java.util.Objects',
                    result: 'org.jivesoftware.smack.util.Objects'
        }

2. lib/installer-excluded These tools are used for building the aTalk apk. They are not included in the final apk file.

3. lib/native Please refer to README_lib_built.md in ./jni/static_library_build for more information; if F-Droid wants to build from source. The native jni project have released to github

4. lib/smack All the jar's are smack library 4.4.0-alpha2 libraries which are yet to be released to maven repository. aTalk source was modified to use smack v4.3.x earlier. When aTalk moves to use smack v4.4.x to include some of the improved features i.e. DNSSEC and DANE verification that aTalk needs, the modified aTalk is only compatible with smack 4.4.0-alpha2 releases. The releases however are only available in https://igniterealtime.org/repo/org/igniterealtime/smack/smack-core/4.4.0-alpha2-SNAPSHOT/ If required, F-Droid can download the smack source for 4.4.0-alpha and built to be included in aTalk.

Note: aTalk smack-experimental-4.4.0-alpha2 also include source for jingle support which are not in alpha2 release. They are only in the smack pull request. aTalk is trying to migrate to use smack jingle replacing current jingle source in aTalk for streamline and take in new feature implementation in smack. However the jar from the smack library should also work as jingle features are only access in my local separate project files.

If you want more information on Jitsi and smack release status with respect to aTalk. You can refer to their project discussion threads raised by me.

[cmeng-git]

I have cleanup the lib static jar, removing lib/installer-excluded as they are no longer required by aTalk. Also use jar from maven repository if they have released and available.

[licaon-kter]

And /lib/bundles/ ?

[cmeng-git]

No change. Status same as explained in my previous comment.

[Hoeze]

@cmeng-git As long as the sources are available, it shouldn't be a problem to build it automatically, shoud it?

Advantages:

• You don't have to strip and work around pre-compiled binaries
• It is possible to automatically patch the sources where you need to fix them
• You don't have to bother because of un-released maven libraries

[licaon-kter]

@cmeng-git

No change. Status same as explained in my previous comment.

Right, for some reason I did not see you mentioning them, my bad.

Anyway, this requires a lot more work than I'm able to do, hopefully someone can figure a way to build them all from source.

[cmeng-git]

Actually most of Jitsi jars are part of the jitsi-android that I had inheritated many years back. I did made some changes to use the latest jars from them on some of the libraries. As the jars are working, i did not really spent time to rebuild them from source. One reason is most of jitsi jars are built using Ant which I am not familiar with. Jitsi has taken over by commercial, the source are not so well maintain. If you really need to build from source, i really have to spent time to find the matching version source release.

[IzzySoft]

Unfortunately, that will be necessary. F-Droid insists on either building an app completely from sources – or not at all. Part of our inclusion citeria.

[cmeng-git]

Is F-Droid ok if I just find the source and send to them for their own compilation for jars that are used in aTalk apk built. aTalk does not want to include these source in the official aTalk release repository.

[IzzySoft]

You could put them into (a) different repo(s) and make use of Git's "submodule" feature. But details on this are better explained by a developer like @licaon-kter.

[licaon-kter]

I'm no developer :)

But that's the general idea, if possible.

[IzzySoft]

I'm no developer :)

Ugh… no off… ouch. Well, let's say "someone with build experience" then? :fearful:

[cmeng-git]

As I have mentioned, most of jitsi jar libraries used in aTalk, look like they are one off SNAPSHOT releases use in jitsi-android or jitsi. So these releases may not have good source control in their repository; and also lack branch release reference for Grade to fetch the source during built.

Last night, I manage to get all the library sources after google through the online repositories. In the process, I found out some of the earlier history source release are no further exist in their official repository, I have to use one that I have kept locally. Jitsi has been acquired by Atlassian; since then, jitsi source maintenance does not get much attention. Fetching online may posses some risk, as they may not always be there.

For some of the source if they are no too big, I may just include them in aTalk project files. For those big library sources, I will include them as reference source e.g. fmj-1.0-20170519.163031-24-sources.jar in the atalk source reference directory yet to be created. F-Droid can rebuild the jar and replace all those in aTalk lib before build, or just confirm the jar in the aTalk/lib matches the source and use the jars as they are.

[cmeng-git]

I have checked in the latest source changes per the above.