search

mapsme / omim

🗺️ MAPS.ME — Offline OpenStreetMap maps for iOS and Android
Apache License 2.0
4.56k stars 1.15k forks source link

F-Droid...? #85

Open arkhan opened 8 years ago

arkhan commented 8 years ago

This application meets the requirements for repositories added to F-Droid ...?

Zverik commented 8 years ago

It doesn't on two counts:

But other than that, it does. As far as I understand, we do not object to publishing the app on f-droid, though we'd prefer only release versions to be published.

lorenzhs commented 8 years ago

See https://github.com/WhisperSystems/TextSecure/issues/127 for some possible arguments against distribution on F-Droid. Many do not apply here, I just wanted to list it for completeness.

Quick summary of those that might apply here:

These issues are a lot less critical here than they are in an app that people rely on to keep them secure, so I think it's entirely doable to distribute Maps.ME on F-Droid, but doing it well would probably require some work.

Zverik commented 8 years ago

Well, these are cons of downloading from F-Droid, not of publishing the app there. We definitely won't use it as an official distribution channel.

ghost commented 8 years ago

It's encouraging that devs are open to the idea of supporting F-Droid. Although I use it personally, I realize it is a very niche audience. Thank you for working with the community.

From my perspective I can see a few distribution options.

1) Third party binary repo

E.g. The Guardian Project

This would not be much different than the APKs you currently offer on your website, but with increased security of file downloads.

Pros:

Cons:

2) Reproducible builds All proprietary components must be removed/replaced.

Pros:

Cons:

3) F-Droid Source-built version

E.g. Fennec F-Droid, Telegram FOSS

May need a generic "white-label" branding to comply with copyright.

Pros:

Cons:

I recommend setting up a binary repo if you would like to support F-Droid users until a more permanent solution such as a reproducible build can be achieved. The first two have the advantage of being used as official distribution channels.

pizzamaker commented 8 years ago

A thread for its inclusion has been created at https://f-droid.org/forums/topic/maps-me/

Zverik commented 8 years ago

Thanks!

HalosGhost commented 8 years ago

I would just like to throw in my support for a custom repo. This makes it available to users through F-Droid but avoids almost all of the problems that Whisper ran into (frankly, I'd really love to see them do this too)!

Either way, keep up the rockin work!

mvdan commented 8 years ago

Just looked through the source code and it would indeed be easier for everyone if you just set up your own F-Droid repo. Which is just a few files on an HTTP server, really.

Once you do, you can set up a QR code for people to scan like this one: http://grobox.de/fdroid/

relan commented 8 years ago

Unfortunately, the source code of Android app is incomplete. I've found the following dubious modules:

Blobs are blockers for inclusion into F-Droid. Facebook's EULA will most probably be rejected too because it's unclear whether this is FOSS or not.

HalosGhost commented 8 years ago

@relan, well, that is a blocker for inclusion in the official F-droid repo. But as mentioned previously, hosting a repo specifically for this can allow anything so long as it is legal for distribution. Ideally, I would like to see a version of this lacking the blobs you mention, but they do not actually stop it from being distributable in a custom F-droid repo as far as I know.

mvdan commented 8 years ago

Both @relan and @HalosGhost are correct. I concur with the latter clarification of "inclusion into F-Droid", since that applies to our main repository. Third party repositories can have their own rules.

relan commented 8 years ago

@HalosGhost, you are absolutely right. I was talking about inclusion into the main F-Droid repo because I don't see any point in unofficial repo for MAPS.ME: if you accept proprietary software that tracks you, just use Google Play; if you don't, you cannot use MAPS.ME regardless of the installation source.

I've found another issue: private.h file is missing. It should contain addresses of servers used by the app. Obviously the app is useless without servers because you cannot even download a map.

Zverik commented 8 years ago

To make an empty private.h, run configure.sh from the repo root (see docs/CONTRIBUTING.md). You can copy maps to a device manually, or set up your own server.

mvdan commented 8 years ago

@relan I mostly agree, but there's a difference - you can freely fetch the apks without having to use Google Play and a Google account. You could do that by just uploading apks somewhere, but F-Droid adds the interface, security (index signing and apk checksums) and update notifications.

HalosGhost commented 8 years ago

@relan, I have to agree wtih @mvdan on this point. One of the obvious use cases for F-droid is to have as close to only free software on your device as possible. But another big one that many people forget is the ability to install applications without needing access to Google Play.

relan commented 8 years ago

@Zverik, thanks for clarification. Please consider publishing those addresses. They are not a secret anyway but their absence hinders development for newcomers.

Also, could you publish the source code of gson-altered?

joncamfield commented 8 years ago

Also, one can run f-droid repositories in offline locations; lots of valuable use cases if there's also a way to side-load maps and bookmark data.

ghost commented 8 years ago

@joncamfield Sounds like it would work well in Cuba or anywhere else with expensive or unreliable internet access.

Using F-Droid swap would be a great way to help those users avoid expensive data charges, use and possibly contribute to OpenStreetMap, and be exposed to FOSS.

The upshot for MAPS.ME is increased brand exposure and first-mover advantage. @Zverik what do you think about this?

joncamfield commented 8 years ago

I was actually thinking of things like national parks / hiking trails and off-the-beaten-path tourist spots (or -- with Swap, backpacker hostels where people could swap placemarks) Obviously, it would also be useful in low-connectivity environments around the world!

utack commented 8 years ago

Released as of today, can be closed?

Zverik commented 8 years ago

I wonder how does it do map downloading...

miclill commented 8 years ago

Great. Thanks! (It's >50mb. maybe this could be changed?)

ghost commented 8 years ago

@miclill The F-Droid version is an unofficial fork so don't expect it to be supported by the devs here. Questions about that specific build should be directed to its issue tracker.

relan commented 8 years ago

@Zverik,

I wonder how does it do map downloading...

It uses the same servers as the official builds (commit).

I hope you won't consider this fork as a competitor. It targets a different audience that wouldn't use MAPS.ME otherwise due to proprietary dependencies. It's a win for everyone: F-Droid users get a great navigation app, you get new users (many of which are quite advanced and socially active). I'd like to cooperate with the upstream as much as possible, so feel free to contact me.

Many thanks to the MAPS.ME team and Mail.ru for making all this possible!

biodranik commented 8 years ago

You have removed all analytics from your fork, and still using our servers in not a legal way, without our permission, putting our official users under a risk of servers overloading.

This is gray zone, we definitely can not guarantee anything. Please consider using your own servers for maps as soon as possible.

On Dec 2, 2015, at 08:08, relan notifications@github.com wrote:

@Zverik https://github.com/Zverik,

I wonder how does it do map downloading...

It uses the same servers as the official builds (commit https://github.com/relan/omim/commit/38e41f90f9be9bde5465c6c7f3017c999a1a6a0e).

I hope you won't consider this fork as a competitor. It targets a different audience that wouldn't use MAPS.ME otherwise due to proprietary dependencies. It's a win for everyone: F-Droid users get a great navigation app, you get new users (many of which are quite advanced and socially active). I'd like to cooperate with the upstream as much as possible, so feel free to contact me.

Many thanks to the MAPS.ME team and Mail.ru for making all this possible!

— Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161183204.

relan commented 8 years ago

@deathbaba,

You have removed all analytics from your fork, and still using our servers in not a legal way, without our permission

User tracking is a show stopper for F-Droid version because MAPS.ME uses proprietary libraries for that.

I'm quite confused by your statement. Could you please clarify

  1. Do users of the official builds loose the right to access your servers if they disable user tracking in preferences?
  2. If I make a fork with all the user tracking left but disabled by default (with the ability for a user to enable it easily in preferences), will users of this fork have the right to access your servers?

Please consider using your own servers for maps as soon as possible.

This introduces a very high barrier for F-Droid version. I'll try to find people who would like to run server infrastructure for the project, but most probably I'll have to shut it down. :(

biodranik commented 8 years ago

My user tracking comment was about your statement:

you get new users (many of which are quite advanced and socially active).

Without analytics libraries we can’t count these users as “ours”. If we can’t count, we can’t say that we got new users. If user installs MAPS.ME from Google Play or AppStore, at least we see download numbers and updates count, even if analytics was disabled by user.

But we actually get higher load on our servers.

We never gave any official permission to use our servers for a forked applications. Even more, you made our private server urls publicly available, without requesting any permission. Documentation states that you should use your own servers: https://github.com/mapsme/omim/blob/master/docs/INSTALL.md#map-servers

Please, don’t get me wrong. I’m a great fan of Open Source and want to spread good software everywhere. And it was feasible to grant an official permission in your case to use our servers, or even get a separate server, just for f-droid users.

Let’s discuss a possible option which can at least temporarily reduce risks of overloading our servers by f-droid users. If you append string “-fdroid” or “fdroid-“ to ‘res’ variable (after line 45, res = HashUniqueID(res); in file android/jni/com/mapswithme/platform/Platform.cpp ) at least we can block possible DDOS attempts from your fork. This ID is used when client tries to download any map from our servers. If you use your own server(s), it is not used at all.

On Dec 2, 2015, at 11:13, relan notifications@github.com wrote:

@deathbaba,

You have removed all analytics from your fork, and still using our servers in not a legal way, without our permission

User tracking is a show stopper for F-Droid version because MAPS.ME uses proprietary libraries for that.

I'm quite confused by your statement. Could you please clarify

• Do users of the official builds loose the right to access your servers if they disable user tracking in preferences? • If I make a fork with all the user tracking left but disabled by default (with the ability for a user to enable it easily in preferences), will users of this fork have the right to access your servers? Please consider using your own servers for maps as soon as possible.

This introduces a very high barrier for F-Droid version. I'll try to find people who would like to run server infrastructure for the project, but most probably I'll have to shut it down. :(

— Reply to this email directly or view it on GitHub.

pizzamaker commented 8 years ago

Why not use free analytics tools?

biodranik commented 8 years ago

Btw, Alohalytics was open source and free. But you have removed it too :)

On Dec 2, 2015, at 13:47, pizzaiolo notifications@github.com wrote:

Why not use free analytics tools?

— Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161256071.

mvdan commented 8 years ago

As @relan pointed out, the idea was to remove non-free parts of the app, not analytics in specific. And of course no offense/harm was meant toward upstream.

If those analytics are in fact open source, then there is no reason for removing them.

ghost commented 8 years ago

All the maps are available publicly, how can there be a risk of overloading? Or does the app use a different server? Since there are no analytics with the fork, map downloads should be the only communication with MAPS.ME servers. Either way, I wouldn't mind sideloading maps if in app server access was revoked.

P.S. As a user, removal of antifeatures such as tracking is a big advantage.

biodranik commented 8 years ago

This link is just for manual download by users who can’t do it in the app by some strange network issues or provider blocking. It is NOT intended for use in the code.

On Dec 2, 2015, at 17:14, twzkxkan notifications@github.com wrote:

All the maps are available publicly http://direct.mapswithme.com/direct/latest/, how can there by a risk of overloading? Or does the app use a different server?

— Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161308786.

LuccoJ commented 8 years ago

And yet it can be used by anyone with a web server. Look, when you release something as open source (under most OSI-approved licenses at least), you are releasing it to be used, modified and redistributed by anyone. It's that simple. Don't want this? Don't release it as open source. I'm tired of seeing people release things as open source only to later WHINE when the rights that open source grants are actually USED by others. That way, you're trying to make "open source" merely dead letter. As to "Even more, you made our private server urls publicly available, without requesting any permission", I'm very confused. If the "private" server URLs are listed in the source code, and the source code is open, then they're public. That's also a very linear concept.

mvdan commented 8 years ago

@LuccoJ your understanding of what free software is is plain wrong. Them releasing their software doesn't grant you rights to using their servers. Ranting like this, it seems to me like you'll just get them to regret ever open sourcing this app.

biodranik commented 8 years ago

Our private servers were (and are) NOT listed in our open sourced code. The author of f-droid fork has reverse-engineered urls from the client and put them into an open source without even asking us.

Servers are not an open code. It is a limited resource, which we are paying for and taking care of. And if anything goes wrong with our private servers, millions of our users will suffer. And I want to avoid it by simple preventive measures.

On Dec 2, 2015, at 18:21, LuccoJ notifications@github.com wrote:

And yet it can be used by anyone with a web server. Look, when you release something as open source (under most OSI-approved licenses at least), you are releasing it to be used, modified and redistributed by anyone. It's that simple. Don't want this? Don't release it as open source. I'm tired of seeing people release things as open source only to later WHINE when the rights that open source grants are actually USED by others. That way, you're trying to make "open source" merely dead letter. As to "Even more, you made our private server urls publicly available, without requesting any permission", I'm very confused. If the "private" server URLs are listed in the source code, and the source code is open, then they're public. That's also a very linear concept.

— Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161330589.

LuccoJ commented 8 years ago

@deathbaba if they were not in the code, then shame indeed on the person who put that in F-Droid. But, apparently, as @twzkxkan pointed out, the maps are available publicly on well-known servers, too...? You say those servers are just for manual download when users have problems, but if the reality is that anyone can point an HTTP client there and download them, I don't see the issue.

@mvdan I understand it fine, and I'm sadly aware that in some countries the law requires one to respect server ToS even when you're accessing an unauthenticated, public HTTP server - which is pretty messed up, if you ask me. But regardless of what a specific jurisdiction is like, I strongly believe that someone telling you that you can only use their (open source) code to access their servers if you don't make certain modifications to it (like removing analytics, or whatever) is blatantly contrary to the free software spirit, and should at the VERY least give you a very big red-letter warning on any F-Droid entry. Furthermore, in my view, you're basically never guaranteed you'll be able to use such a piece of software in virtual perpetuity; instead, it's all in the hands of the original developers, and that is ALSO a deal-breaker for free software ideals.

mvdan commented 8 years ago

@LuccoJ I agree that the situation isn't ideal, but again, ranting at upstream devs will accomplish nothing.

biodranik commented 8 years ago

You can play around and point code to this one public server. And if it will be DDOSed because of that, other servers will continue to work, and users will not suffer.

We intentionally made one specific server publicly available for our users to manually download maps, and for developers to play around with maps.

I repeat again: my only concern is that millions of our users should always have our map servers accessible. If you can guarantee it in any way, you can get our permission to use our private servers.

And just in case if you missed our documentation which says how to set up your own server: https://github.com/mapsme/omim/blob/master/docs/INSTALL.md#map-servers https://github.com/mapsme/omim/blob/master/docs/INSTALL.md#map-servers And how to generate maps to put on this server: https://github.com/mapsme/omim/blob/master/docs/MAPS.md https://github.com/mapsme/omim/blob/master/docs/MAPS.md

P.S. Free software spirit should not make any harm to real users. They don’t understand it.

LuccoJ commented 8 years ago

@mvdan you see, it's that every time that I have a peek on #fdroid on freenode, it's always about some app that I had noticed the previous day and seemed good news to have in "the free world", being discussed because their developer has gotten MAD about it being released on F-Droid, for one reason or the other. That gets a bit tiresome, as I'm sure you are first in line to understand, and the way I tend to see things at least, I end up preferring software like that not to be released as open source, instead of being "teased" and then realizing it's even harder in practice to keep a hold on it than it is with some proprietary software.

@deathbaba honestly do you really think the small minority of Android users who source their software from F-Droid instead of Google Play will have an impact on your servers' availability? And in any case, when someone downloads your software, generally they'll start downloading maps (or, exactly one map) almost immediately, so what great gain do you have by having "advance warning" from your analytics in terms of a couple of minutes?

biodranik commented 8 years ago

@LuccoJ I suppose that apk from f-droid can be uploaded into any Chinese store and we easily can get our servers DDOSed, without even seeing these users and without any easy way to block them not harming ‘legal’ users. That’s why I offered at least a patch which can help us to control such situations.

If analytics shows that we have more users from some specific country/app store, we can rescale our server’s load. Or put more efforts/resources to support this new source of users and traffic. We can buy additional servers and traffic to scale it for our new users. And we can use new users in our marketing texts (we just got XXX installs from YYY country!) And of course it’s not about realtime reaction.

On Dec 2, 2015, at 19:02, LuccoJ notifications@github.com wrote:

@mvdan https://github.com/mvdan you see, it's that every time that I have a peek on #fdroid on freenode, it's always about some app that I had noticed the previous day and seemed good news to have in "the free world", being discussed because their developer has gotten MAD about it being released on F-Droid, for one reason or the other. That gets a bit tiresome, as I'm sure you are first in line to understand, and the way I tend to see things at least, I end up preferring software like that not to be released as open source, instead of being "teased" and then realizing it's even harder in practice to keep a hold on it than it is with some proprietary software.

@deathbaba https://github.com/deathbaba honestly do you really think the small minority of Android users who source their software from F-Droid instead of Google Play will have an impact on your servers' availability? And in any case, when someone downloads your software, generally they'll start downloading maps (or, exactly one map) almost immediately, so what great gain do you have by having "advance warning" from your analytics in terms of a couple of minutes?

— Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161345123.

LuccoJ commented 8 years ago

If you ask me, I'd say that "We subscribe to the free software ideals and we are shipped by F-Droid as transparent open source software" is a more effective marketing text than "We got X installs from country Y", but maybe this is just my own wishful thinking. I still don't get it, anyway. When people download your maps, your server will see how many IP addresses are downloading and what countries they come from. What's the difference? The only difference I can see is that you can't know that one IP address corresponds to exactly one (and always the same) user, since addresses can be dynamic.

But that's about tracking users, not simply analytics.

ghost commented 8 years ago

@deathbaba 's concerns are reasonable. The reverse engineering risk exists regardless of whether or not the code is present in the F-Droid fork. But it's still best to incorporate the changes proposed if their servers are used and the app is being widely distributed.

It looks like we have three options:

1.)make the changes @deathbaba proposed (unique ID) 2.)setup a community run server 3.)remove internet permissions from the fork and manually download and copy map files onto a device

I'm not sure which server @deathbaba was referencing "You can play around and point code to this one public server", but if they are willing to let one of their servers be used in code that is a fourth option.

biodranik commented 8 years ago

@LuccoJ We can not distinguish between F-Droid users and all other users in the case you described.

And it’s a good question, what is more effective marketing text :)

On Dec 2, 2015, at 19:17, LuccoJ notifications@github.com wrote:

If you ask me, I'd say that "We subscribe to the free software ideals and we are shipped by F-Droid as transparent open source software" is a more effective marketing text than "We got X installs from country Y", but maybe this is just my own wishful thinking. I still don't get it, anyway. When people download your maps, your server will see how many IP addresses are downloading and what countries they come from. What's the difference? The only difference I can see is that you can't know that one IP address corresponds to exactly one (and always the same) user, since addresses can be dynamic.

But that's about tracking users, not simply analytics.

— Reply to this email directly or view it on GitHub https://github.com/mapsme/omim/issues/85#issuecomment-161350276.

Thrilleratplay commented 8 years ago

@LuccoJ Please be respectful. Regardless of your views on what open source software is and interpretation of the Apache License, you must admit releasing the Mape.me code is a monumental gift to the FOSS community. Like Whisper Systems, wanting to have control in the distribution of their software is very reasonable when you think about brand integrity. As far as I know, Whisper Systems issue with Fdroid is that they did not have control over the builds offered by a third party; important for company that sells itself on security. You and I both trust Fdroid implicitly. My phone does not have gApps and install apps exclusively from Fdroid. It is voluntarily run community but, like any online community, its integrity is contingent on its members and kickass moderators which could change one day. Just look at SourceForge. You have the option to uninstall Fdroid but developers may not have the option to remove their app if used for malicious purposes before their name has been tarnished. Also as a software developer, I understand that bugs unfortunately happen. The Maps.me team can test to ensure a logic flaw does not accidentally DDOS their servers but they do not have this level control in a third party fork. Open software is about the openness of sharing of ideas and concepts, that goes beyond just the code but also the people who use and develop it. Open source, open minds.

@deathbaba I will apologize for the rudeness here. As a member of the FOSS Android community, I want to thank you for what Maps.me provided and hope that this has not soured the decision to release the source.

As far as analytics is concerned for Fdroid, if the analytics library is open source I beleive it can be included as long as there is an option to opt-in/opt-out.

Is Github's Large File Storage an option to mirror the map files for Fdroid?

LuccoJ commented 8 years ago

@deathbaba I'm not really sure why you'd want to distinguish F-Droid users if the goal is simply to know how many resources to allocate. But anyway, you could distinguish quite simply: just have a (secret) ID that your proprietary built sends to the servers, and the F-Droid build would have to come up with a different ID, so you could tell which is which. I'm not talking about unique IDs for user, but just an ID for the build.

@Thrilleratplay without implying that this is a similar case, the Trojan horse was a monumental gift, too. "Having control in the distribution of your software" and "developers having the option to remove their app (from repositories)" are concepts that run directly contrary to software freedom. If you want direct control on, including the ability to REVOKE license to use, your code, then you don't license it under a free software license, because that's EXACTLY what free software licenses are made to prevent.

patcon commented 8 years ago

:-1: @LuccoJ

biodranik commented 8 years ago

@Thrilleratplay Thanks for your support! MAPS.ME has a setting in menu to turn off analytics. About Github LFS, we never consider it because nobody will host petabytes/month traffic for free. But in theory, if they have HTTP links to files and support HTTP partial download it can work. HTTPS probably needs some tuning/testing in the client code, we didn’t tried it yet (see #817).

@twzkxkan @relan Options I see sorted by preference:

  1. Use community servers for maps (no any dependency from us, no risks for F-Droid users). Cons: someone needs to generate and update maps on a regular basis. Or at least sync it with our direct link.
  2. Do not remove open sourced Alohalytics statistics ( https://github.com/deathbaba/Alohalytics ) so we can count F-Droid users as ours and scale servers if needed.
  3. Add “fdroid-“ prefix or “-fdroid” suffix to UniqueClientID so we can block unwanted traffic by user agent in an emergency case.
  4. Leave METASERVER_URL empty (I would prefer to clear commit history too), put our direct server into DEFAULT_URLS_JSON and replace OMIM_OS_NAME in Storage::GetFileDownloadUrl to "direct”. At least we lose only one node if DDOS happens.

Some options can also be combined.

And in any case, please add the following flavor into android/build.gradle to make your apk more customized for F-Droid, and build it with ./gradlew assembleFdroidRelease

    fdroid {
      versionName = android.defaultConfig.versionName + ‘-fdroid'
      buildConfigField 'String', 'SUPPORT_MAIL', ‘"fdroid@maps.me"'
      android.sourceSets.fdroid.assets.srcDirs = ['flavors/mwm-ttf-assets']
    }

(Yes, I’ve just created fdroid@maps.me email for support).

agilob commented 8 years ago

@deathbaba w8 a sec, what do you mean:

in not a legal way,

  1. Were there any securities or rules that were broken?
  2. Your position is currently against the licence file you published in the repository. You should consider changing the licence.

All the following redistribution rules are satisfied by Fdroid:

 4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:
      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and
      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and
      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and

3 In NOTICE file you say: Please refer to their LICENCE, COPYING or NOTICE files for terms of use.

There is no COPYING file in the repository.

f-droid can be uploaded into any Chinese store and we easily can get our servers DDOSed

You know... I can do the same with any apk by using apk downloader or backuping apk from playstore.

LuccoJ commented 8 years ago

@agilob Use of their servers has legally nothing to do with the license the source code is under.

mvdan commented 8 years ago

Let's drop the licensing discussion. They are not violating their own license, nor are we by publishing it. But they are in their right to ask us to use their servers on their terms.

@deathbaba option 2 can be done. Option 3 can be done along with it, but if 2 is done, is there really any need?