F-Droid...?

[arkhan]

This application meets the requirements for repositories added to F-Droid ...?

[LuccoJ]

Why don't we call it "tracking" rather than "analytics", since I was on https://github.com/deathbaba/Alohalytics and it describes that's what it does - track potentially any action done inside the app, against unique identifiers such as "basic devices information including IDFA and Google Play Advertising IDs"? Option 3 seems much better to me, and it would allow F-Droid to publish it without an anti-feature (which, IIRC, would even hide the app from F-Droid's default configuration).

[mvdan]

@LuccoJ as per https://f-droid.org/wiki/page/Antifeatures, any analytics on by default are a tracking anti-feature.

We don't filter out apps with anti-features. We patch play-services-analytics because it is non-free, not because it is tracking. So no reason why we should remove them.

[LuccoJ]

@mvdan I thought to be able to see apps with anti-features, you had to enable that in the options. Maybe that was in an older version of the client, I haven't checked in a while.

[mvdan]

Yes, this was a very old version of the client. Even then, they were shown by default.

[relan]

Options I see sorted by preference:

Good summary, see my comments below.

Use community servers for maps (no any dependency from us, no risks for F-Droid users). Cons: someone needs to generate and update maps on a regular basis. Or at least sync it with our direct link.

I totally agree that this is the best option. After rethinking this situation I've concluded that this is the only way to go because if a fork uses official servers, the upstream may impose arbitrary limitations on it. This is unacceptable for a FOSS project.

Do not remove open sourced Alohalytics statistics ( https://github.com/deathbaba/Alohalytics ) so we can count F-Droid users as ours and scale servers if needed.

It's worth mentioning that MAPS.ME uses 4 (!) user tracking libraries: Alohalytics, FlurryAnalytics, myTracker and Google Analytics. While Alohalytics is open source, the other 3 are proprietary.

Add “fdroid-“ prefix or “-fdroid” suffix to UniqueClientID so we can block unwanted traffic by user agent in an emergency case.

That's interesting. MAPS.ME generates a unique ID for each installation and passes it to the server. So you can track (and block) requests from each particular device even if analytics is disabled or removed.

To be honest, I find your explanation that you need to track users for

• server load balancing
• DDoS attacks prevention

totally unsatisfactory.

Leave METASERVER_URL empty (I would prefer to clear commit history too), put our direct server into DEFAULT_URLS_JSON and replace OMIM_OS_NAME in Storage::GetFileDownloadUrl to "direct”. At least we lose only one node if DDOS happens.

This has nothing to do with DDoS. A DDoS attack is a complex (and costly!) thing that uses custom software.

Anyway, thanks for explicitly stating your position. F-Droid will stop distributing MAPS.ME soon (the request has already been merged, need to wait for the next repo index update). For those who may be disappointed by this I'd like to remind that the fork was reported to be illegal and thus puts its users at risk. I kindly ask everyone to keep calm.

[biodranik]

@deathbaba I'm not really sure why you'd want to distinguish F-Droid users if the goal is simply to know how many resources to allocate. But anyway, you could distinguish quite simply: just have a (secret) ID that your proprietary built sends to the servers, and the F-Droid build would have to come up with a different ID, so you could tell which is which. I'm not talking about unique IDs for user, but just an ID for the build.

@LuccoJ That’s exactly what I mentioned in option 3.

You know... I can do the same with any apk by using apk downloader or backuping apk from playstore.

@agilob Yes, I know. And we can count these users in this case (analytics is present in official apks) - and it is great! But it is not true for current F-Droid apk. MAPS.ME licensing you are referring to is about the code, not about trademarks (you can’t use MAPS.ME name or our app icon for your own business/another application without our permission) and other finite resources (our servers).

@mvdan option 3 is not needed if 2 is used. And if you add a separate F-Droid flavor I described, it would be even better (you can push it into upstream branch).

That's interesting. MAPS.ME generates a unique ID for each installation and passes it to the server. So you can track (and block) requests from each particular device even if analytics is disabled or removed.

@relan In theory, we can block users one-by-one, but it does not help in the case of apk leak or other unlikely ddos-related event. If you mark all F-Droid build users by a suffix/prefix it is at least feasible to protect our servers. For example, we can redirect users with this suffix to a separate server for maps downloading. And I don’t like this option too, it’s mostly about some kind of compromise.

This has nothing to do with DDoS. A DDoS attack is a complex (and costly!) thing that uses custom software. I use this terminology to describe the case of uncontrolled/unexpected high load which breaks our service for millions of our users.

To clarify again: I like F-Droid community and ready to support it, with only one concern: any potential harm/risk to our existing users should be reduced to minimum or better avoid at all.

[relan]

@deathbaba, I again have to agree with you that all options except 1 (completely independent fork with own servers) will be a compromise that won't completely satisfy either side.

To all: are there people, not affiliated with MAPS.ME or Mail.ru, who would like to do server-side job for an F-Droid-friendly fork? Please reply here.

[rugk]

As a user of MAPS.me I also would very much appreciate a release on F-Droid. So :+1: for this issue in general.

From reading this thread I see there are mostly tracking issues. But I think @deathbaba has made multiple kind suggestions how they would allow clients to use their own servers. Generally thanks to them for doing such a nice thing for the FLOSS community and for staying calm in this discussion.

I also think that 4 tracking libraries are too much and not neccessary (do you really need to track each user with 4 libraries to prevent DDOS?!), so all properitary ones should be removed. As a question BTW: If you disable tracking in the settings all of them are disabled, aren't they? So this is what I would suggest: The one FLOSS library may be included and disabled by default, so it is no antifeature. Additionally you may add this prefix/suffix to the user ID so that MAPS.me can identify the users. If they want it I have no arguments against it from a user perspective as it does not make the user significantly more trackable (the prefix is the same for all users of the build). But what I have not understood yet is one thing: Is this UniqueClientID an ID which is hard-coded into the source, so that you can identify a build (which would help to prevent the Chinse-store-uploads you mentioned as an example) or is it a ID for each device/user of the app as the name suggests? In case the latter is true this is obviously tracking (without using any tracking library) and this antifeature should be removed in the F-Droid build. Alternatively you may just replace it with a hard-coded string (fdroid-<authorname> e.g.) so that it becomes only the "build-tracking" feature mentioned earlier.

---

Another way is obviously the self-hosted server. So why not use rawgit or GitHub pages to host the files? They are available in public and if you can even download them via HTTPS this additional secure won't hurt.*

* Note that you should make sure the TLS connection is properly verified (checking for root certs) in this case.

---

Another general note about all the ones talking about the License: At first the license for open source code does not allow you do use the servers of the authors - that's something completly different. Secondly please don't stick to this "But this is the license". Yes the code is licensed in such a way and you have the right to use it in any way allowed by the license, but please also be human - it is very desirable to have a good relation to the authors and for example there is no argument against making the few modifications for a "F-Droid Flavor" @deathbaba mentioned.

I hope I could give you my point on this and I'd also like to ask whether there is any progress on this issue as the last comment is already two months old.

[rugk]

Any news on this?

[Ezwen]

I would also be very interested in having news on that topic :).

After reading the whole discussion, I think that option 3 is still a very good one for having a first version of MAPS.ME in F-droid. I really wouldn't mind manually downloading and installing the maps, as long as it is explained. It is far from perfect, but would make possible a first set of releases.

And once this is done, we can still hope to have a community server running someday.

[willhelm-mueller]

same for me, currently i use osmand and there i also place the maps manually into the right folder.

[jeremy447]

It would be very great to be able to install Maps.me from F-droid.

[rugk]

As I elaborated earlier the files could just be hosted by GitHub pages or we could even just use the official servers - provided the necessary User Agent/ID changes are made in the fork.

[paride]

The project would gain a lot of popularity and respect among the the free software community if the developers agreed in having maps.me distributed via f-droid.

I don't understand @deathbaba when he writes:

I suppose that apk from f-droid can be uploaded into any Chinese store and we easily can get our servers DDOSed, without even seeing these users and without any easy way to block them not harming ‘legal’ users. That’s why I offered at least a patch which can help us to control such situations.

Would this be a "voluntary" DDoS, or just an excess of users? If it's voluntary I don't think that having the app published somewhere makes any difference, they would just try to make the servers unavailable. But I really don't see the reason to. If it's just a "too many users" problem, why should this come from f-droid and not from the official App store? You already distribute the apk anyway (https://maps.me/apk/), so I can't see the difference. The userbase of f-droid is not so big anyway. But maybe I'm missing the point here.

[willhelm-mueller]

hmhm if someone would make this app 'F-Droid compatible' without using their servers for map retrieval everything should be fine? An additional, offline OSM Map -> maps.me converter is already described here: https://github.com/mapsme/omim/blob/master/docs/MAPS.md So we could use this tool to generate our own map files

Since somebody already cleaned this app from 3rd party trackers / statistic sites it should be doable.

[paride]

The .mwm files http://direct.mapswithme.com/direct/latest/ sum up to about 33GB, so it's not so much. Still GitHub asks for repositories not to exceed 1GB, so using GitHub for hosting is not an option.

[Ezwen]

Why not package a version for which one has to download and install map files manually, for a start?

Le 5 juillet 2016 16:16:20 CEST, legovini notifications@github.com a écrit :

The .mwm files http://direct.mapswithme.com/direct/latest/ sum up to about 33GB, so it's not so much. Still GitHub asks for repositories not to exceed 1GB, so using GitHub for hosting is not an option.

---

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/mapsme/omim/issues/85#issuecomment-230490747

[paride]

I think that a better solution would be to have a user configurable server address in the app. In this way I can host the map files on my home server and then let friends and family use it.

Ideally this should be added upstream, with the official all coming preconfigured to use the official server. In this way people can experiment with custom maps or self-hosting even with the official app. And of course the official app and the f-droid one would diverge less.

Then, if one day a stable hosting for the map files is found, the f-droid app could come preconfigured to use it.

[willhelm-mueller]

and add

3. Add “fdroid-“ prefix or “-fdroid” suffix to UniqueClientID so we can block unwanted traffic by user agent in an emergency case.

Whereas the UniqueClientID should be erased or be the same for all F-Droids ;)

[Thrilleratplay]

@legovini It isn't the size of the repo that is so much the problem as it is the size of the individual files. Even with Github's large file storage, where you are allowed 1GB of free storage, the bandwidth limit is 1GB/Mo.

I have not found a free for open source hosting solution that can be seamlessly integrated into an app that did not have severely limiting storage and bandwidth restrictions. Perhaps a few Google drives accounts or Mega, but those would be difficult to securely administer for a group. Before @relan deleted his fork, there was the idea of using a bit torrent client or other P2P protocol but no one had the time or development skills to assist with that feature. If it were added, it would likely need to be leech only by default and have an option to only sync over wifi. I can only image someone's phone bill if those went unrestricted. I think this is an optimal solution it eliminates the need to use Maps.Me's server or any server for that matter. I would happily seed this from my home computer.

[paride]

@Thrilleratplay the "torrent" solution is too complex to implement and manage on may points of view, it will not be done.

What I suggest is the server address to be user configurable in the app, thus enabling small scale self-hosting.

[juanitobananas]

@relan, do you happen to still have a git repository somewhere with the official-F-Droid-repository-compliant maps.me app? I am thinking of creating a fork that uses a different server (I want to test it at home at first), and it would be great if I could start with your "freed" version of the app. Thanks!

[relan]

@juanitobananas, nope, I've removed the local copy too because I didn't need it. Anyway, those patches would be heavily outdated now, so you'll hardly miss them.

[juanitobananas]

@relan OK, thank you very much anyway!

[paride]

Great news @juanitobananas, keep us posted. Just for you to know: @mvdan, one of the core f-droid developers, recently mentioned the possibility to host the map files in the form of OBB data on f-droid itself. He said they could handle the traffic. But this would significantly alter how maps.me downloads and updates maps...

[bootofood]

@juanitobananas any news on this? It is really important to get this on f-droid!

[juanitobananas]

Oh! I forgot about this... :blush:

It is really important to get this on f-droid!

I'd love to see it in F-Droid too, but I took a look at it but it was a little more complicated than I expected so decided not to do it for the moment.

Sorry!

[rugk]

Oh, that's bad. So what's about the offer of @mvdan? Is it still valid?

[tuxayo]

@juanitobananas No problem, time is limited. Can you write about what remains to be done? Especially the complicated stuff, so maybe someone else can continue your work.

[juanitobananas]

@tuxayo :smile: I am very much afraid I didn't implement anything, I just took a look at the code to see if I could find the places where it'd have to be changed. I wasn't very successful at that.

[rugk]

Also here some nice analysis information of the tracking functions in Maps.me by @mnemonicflow: https://github.com/mapsme/omim/issues/5122#issuecomment-270690016

Direct link to gist: https://gist.github.com/mnemonicflow/41c6b7fbf9794a663d2bb7293bb6135e

[ghost]

0. User delete "maps.me" and install "LibreMaps.me"(internet connection permission completely removed edition).
1. I download MWM files from http://direct.mapswithme.com/direct/latest/ to my WWW server(by manual or auto script).
2. You point the URL to my WWW server.
3. You download the map.
4. You use it.

Why Maps.me must track all people just using this app? Can't this be this simple?

[Zverik]

With open source, "someone" almost always means you :)

[Zverik]

We haven't got any questions in mail about that, but you don't have to consult us about installing a server. There is no complex procedures to configure it, just publish a directory with mwm files, and that's all. You can do with any cheap hosting solution, although you would need at least 35 GB for the whole world.

[rugk]

Also there are many free CDNs, maybe there are also some hosters available, which offer free hosting for FLOSS projects.

[ghost]

According to maps.me support, "we don't support fdroid" "we don't need that niche audience".

Also, they don't(never) add proxy support to Fdroid client(e.g., Orbot support).

I think this issue can be closed.

[Zverik]

So what? We don't support F-Droid, that's true, but the code is open-source and anyone can make a custom build.

[rugk]

Indeed. This issue is only about where (& how) to host the files. One could maybe use BinTray - it's free for FLOSS projects (1TB download per month, 10GB storage, CDN & HTTPS).

And the files could just be pulled from the official server once day or so.

The only thing we then need is a F-Droid-acceptable fork, which uses the new server.

[Thrilleratplay]

@paride mentioned in August that there is the possibility of hosting the map files on F-Droid's servers.

[paride]

@Thrilleratplay but as an OBB, which would require modifying how maps.me deals with map data. It would be better to find a way to serve the mwm files directly via http, staying near to upstream.

cloudatcost.com is currently offering an 80% discount on their biggest VPS, with 80GB of storage and unmetered monthly transfer. It costs 224$, one time. Of course you get what you pay for, still it could be good enough for serving map data, as occasional downs won't affect the user experience too much.

If somebody is willing to buy the server I'm willing to administer it for free, keeping a mirror up to date. I won't work on the Android app. I can't offer more as I'm currently working on another project, which is already draining the resources I can devote to FLOSS stuff.

[bonanza123]

Frankly, I guess that finding someone that hosts the files is not the problem (I personally can offer ~20TB traffic and ~ 100GB data storage). The "real" work is making the fork.

[tuxayo]

Does a fork is really needed? These F-Droid related changes might not be the core devs priority (and that's fine) but it's totally possible that they would accept well written patches. Can we have a statement on the subject?

[rugk]

It's necessary, already for removing all tracking "features". (or at least making it possible to shut them off)

[Zverik]

I don't think you would need to fork the repo: without a set of private keys all of third-party tracking would be disabled, and the built-in statistics can be disabled in app settings.

[biodranik]

It’s always good to have binaries without any proprietary 3rd party code:

• You never know what are they collecting even without valid initialization keys.
• You waste less CPU/battery (and potentially network traffic).
• Binary size is smaller.
• Some 3party libs can crash at runtime if you don’t provide valid keys to them.

[rugk]

without a set of private keys

Which private keys?

In any case disabling the option in the settings is not enough as @mnemonicflow showed. So as even @deathbaba argues against 3rd party tracking libs, I assume none are used in Maps.me, are they?

[Zverik]

Check out the build dependencies

[rugk]

So @deathbaba if you dislike these 3rd party tracking libs, why not remove them from Maps.me?

[biodranik]

@rugk I don’t have time to do that once, and definitely don’t want to support it later with every new release.

Btw, it should be relatively easy for any developer.

[rugk]

No, your understanding me wrong. Why don't do it in the upstream version (here is this repo).

But, ah, you do not work for Maps.me any more?