mapsme / omim

🗺️ MAPS.ME — Offline OpenStreetMap maps for iOS and Android
Apache License 2.0
4.58k stars 1.15k forks source link

direct links to maps down #10387

Open gganz opened 5 years ago

gganz commented 5 years ago

I'm travelling frequently and have the maps of the whole wold on my phone and tablet (> 1000 in total). I used to download the maps from http://direct.mapswithme.com/direct/latest/ to deploy them to my two devices without having to download them twice. Now I just wanted to update all my maps but the server locked me out (403 Forbidden). Can you please take care of this. Thx

tatiana-yan commented 5 years ago

You can not update maps without updating app (new things appearing in maps, old code does not know about it and will crash). So the easiest and less traffic consumable way is to update all maps on one device (update can be started right after app update: it'll ask you to update everything or manually). It is less traffic consumable way cause in app we use diff when updating maps -- you do not download all maps, just diffs, it's much smaller. And then transfer maps to another device with the same updated version of app.

RussianNeuroMancer commented 5 years ago

@tatiana-yan but what if someone need to do similar deployment on new devices? He have to re-download all maps on every device or you permit download files from http://direct.mapswithme.com/direct/latest/ ?

gganz commented 5 years ago

It was very convenient to download from http://direct.mapswithme.com/direct/latest/ and I have really no idea why they blocked it. No user will benefit from this blocking. Maybe another case of paternalism which is so popular among software companys.

dimqua commented 5 years ago

@gganz probably they trying to decrease the amount of outbound traffic from their servers.

axelsimon commented 5 years ago

It sounds like using a less server-centric solution for file distribution and moving to something like IPFS could be interesting then.

tatiana-yan commented 5 years ago

If you want to place maps directly to device it means that you have an access to device filesystem. So you can take from one device and put to another one. You do not need to download twice.

We want app to control download process because maps version must correspond app version, world.mwm version, world coasts version, countries.txt, configs, etc. App has callbacks on map download and does some job. Incorrect maps download and placement produces hard to reproduce bugs, support reports etc. So I recommend to download maps with app. And diff mechanism makes it less traffic consumable: update of maps on two devices takes less than one complete download.

rlucassen2 commented 5 years ago

@tatiana-yan: there is an option called "Auto-download". What's the purpose of this option? Does it get map updates automagically?

R.

DirtYiCE commented 5 years ago

This also broke the map download within the fdroid version of this app. https://gitlab.com/axet/omim/issues/78

tatiana-yan commented 5 years ago

@rlucassen2 yes, maps update will start automatically after application update.

tatiana-yan commented 5 years ago

@DirtYiCE actually applications based on maps.me should use they own map servers. One reason is that we provide sources but not server infrastructure, another reason is maps and code should correspond: some changes should be done in maps and code at the same time. We do not control forks code and our maps should not correspond it.

This guys even use our support e-mail in they app and our support team often spend lots of time to find out that user uses another app and error is not related to maps.me. Do you think it's right to do so?

linux019 commented 5 years ago

This guy removed all ads and all analytics code from the app. This is only one reason why I switched to open source version of app instead of Google play version. I agree that you are paying for servers infrastructure but not all people like to see ads

rlucassen2 commented 5 years ago

They pay their servers with ads I suppose. So either you use their app and accept the ads / pay 7 dollar per year to be add free, or you use the OSS version. But using the OSS version also means that you set up your own network. I'm willing to contribute to such a network, but apparently I'm the only one. This is not how OSS works. An OSS project can only survive when it is supported by the community. If it's not, it will die.

R.

DirtYiCE commented 5 years ago

@tatiana-yan I couldn't find any support email in the app, only support.maps.me, which looks like more of a knowledge base than a support address. Maybe he removed it in the meantime?

Updating the server and client code at the same time works in case of a webpage, in case of a downloadable application, less. Forcing the users to download xMB of updates right now on a metered connection just because the developers didn't bother about compatibility is a lame move (even if many companies do it all the time).

I can understand that a company does not want to pay the server costs of users using a random third party fork of their app. But don't forget, the base of your maps is openstreetmap, a community made map, available for free. If you're going to make a profit from it, at least give something back.

tatiana-yan commented 5 years ago

@DirtYiCE just search https://gitlab.com/axet/omim/blob/master/data/strings/strings.txt for support@maps.me

tatiana-yan commented 5 years ago

I am developer and came to this topic to help with technical things, to clarify that the best way to update maps is just to allow app to update it: it needs less traffic than complete download even if you have several app installations and it allows app to control all maps and configs correspond each other and properly located. I'm not a manager or someone translating company position. I know much more about technical things. But, as I know we never provided any serverside to another apps. I have no idea why do these guys use our servers we never shared to them. We have a free opensource app with very flexible licence. I think it's a good contribution.

juanitobananas commented 5 years ago

We have a free opensource app with very flexible licence. I think it's a good contribution.

I believe so too. Thank you very much, for this cool piece of software! :smile:

We are planning on hosting a copy of the map files (https://gitlab.com/axet/omim/issues/85) so that the F-Droid version doesn't need to use your servers. Is it okay if I download the maps for every release and host them? Are you aware of any legal/licensing issues?

Again, thank you!

EDIT: typo

RussianNeuroMancer commented 5 years ago

We are planning on hosting a copy of the map files

I guess this is still not resolve this issue for people like @gganz who need to download all maps manually for usage with original app, which is what this issue about .

In this comment @tatiana-yan suggest copying downloaded maps from one device to another or download diffs via app. Which is valid advices for people who downloaded many or all maps to their devices, and still have these devices on hands.

However, as I understand, there is no way to do bootstrap manually anymore, i.e. no way to download maps manually for start on fresh device. So, @tatiana-yan this particular use-case will be ignored and nobody going to fix this, right?

Correct me, if I understand something wrong.

rlucassen2 commented 5 years ago

Axet, the author of maps isn't responding anymore. I consider the project as dead. Since two days on gitlab a new project has appeared called "maps" which is empty for the moment.

R.

tatiana-yan commented 5 years ago

@juanitobananas We are going to build osm-only mwms free of any proprietary data. It will be legal to use it. It will take about a week. I'll post the update to this topic.

@RussianNeuroMancer yes, if you install the app to new device you should download maps from the app. And it's the only supported way to get the maps for official app.

juanitobananas commented 5 years ago

We are going to build osm-only mwms free of any proprietary data. It will be legal to use it. It will take about a week. I'll post the update to this topic.

@tatiana-yan, yay! :fireworks: Thank you so much for that! Looking forward to it!

dimqua commented 5 years ago

@RussianNeuroMancer it's still possible to download maps manually by using the direct links.

burivuh commented 5 years ago

Any use of our official release data with proprietary layers outside of the official application is illegal. Here is the link for OSM-only maps:

http://opensource-data.mapswithme.com/regular/weekly/

Starting 26th of April we'll setup a cron job to generate them weekly on one of our servers.

juanitobananas commented 5 years ago

@burivuh, Thank you very much for this!

http://opensource-data.mapswithme.com/regular/weekly/

Is it okay with you if the F-Droid maps app downloads maps directly from there? Or do you prefer if we try to setup mirrors for the maps.

Again, thank you very much!

burivuh commented 5 years ago

Well, we have no problem with that, at least in the near future. Data is hosted on several nodes, but all of them are in one country, so at least backup mirror will be good.

juanitobananas commented 5 years ago

Cool! Well, we will try to setup mirrors anyway.

Starting 26th of April we'll setup a cron job to generate them weekly on one of our servers.

As far as I've understood from reading several GitHub and GitLab issues, the maps' versions are important. Is it safe to assume that the version (v) in countries.txt will change only when the maps change? Or might the maps change more often?

Or asked differently: Would it be enough for us to monitor countries.txt in order to download the maps?

tatiana-yan commented 5 years ago

As far as I've understood from reading several GitHub and GitLab issues, the maps' versions are important. Is it safe to assume that the version (v) in countries.txt will change only when the maps change? Or might the maps change more often?

Or asked differently: Would it be enough for us to monitor countries.txt in order to download the maps?

We have backward compatibility for maps. It means you can use maps which older than app but not maps which are newer than app. If you want to build the app from our release branch take the last maps version. Countries.txt and World.mwm should be from this version and it'll be compatible with old mwms users already have on their devices.

gganz commented 5 years ago

Hi @all Does that mean I can use http://opensource-data.mapswithme.com/regular/weekly/ as a replacement for the old http://direct.mapswithme.com/direct/latest/ ? I dont need any non-OSM layers. Thx

burivuh commented 5 years ago

Yes, you can.

Zverik commented 5 years ago

Seems like these maps were last updated in March. Could you please reinstate weekly updates?

burivuh commented 5 years ago

Plan is all the same, we just hit a bit of delay in establishing automatic osm-only data pipeline and, you know, long May holidays. I hope in a week or so we'll return to the schedule.

gganz commented 5 years ago

Yes, you can.

Great, thanks a lot!

burivuh commented 5 years ago

First update has been rolled out

Zverik commented 5 years ago

Awesome, thank you!

dbf256 commented 5 years ago

@burivuh No updates for ~1 month? http://opensource-data.mapswithme.com/regular/weekly/

burivuh commented 5 years ago

Well, it takes some time to fix all the problems with a new data pipeline. Current version sometimes down, sometimes broken, sometimes without server resources. We are fixing it. For now it is about one update in about 3 weeks.

pataquets commented 5 years ago

Thanks everyone for the work. Some ideas/suggestions/doubts:

tatiana-yan commented 5 years ago

Would it be possible to share the map generation pipeline files, scripts, etc. in a repo? This would allow interested people contribute back improvement and also help by easing setting up alternate map servers. Seems to me that there is enough interest and it would be a win-win.

See https://github.com/mapsme/dockerfiles https://github.com/mapsme/omim/tree/master/tools/python/maps_generator

pataquets commented 5 years ago

Thanks for sharing! Tried downloading an updated version and it worked. One small caveat: the map does not have terrain height info, so height/altitude map for routes always displays a 0m elevation, flat route. I don't know if it's just an omission of something expected.

tatiana-yan commented 5 years ago

Hello. It is osm-only build so some information is absent cause we get it from other sources (traffic information, altitudes, booking hotels, opentable restaurants). For altitudes we use SRTM and you can set SRTM_PATH in map_generator.ini

dimqua commented 4 years ago

@burivuh maps are out of date again.

http://opensource-data.mapswithme.com/regular/weekly/

tarasis commented 4 years ago

Current date of weekly files is 23-Dec-2019.

You need to update INSTALL.md as it still talks about direct download from http://direct.mapswithme.com/direct/latest/ it took searching through issues to find out what to do instead.

dbf256 commented 4 years ago

Looks like "weekly" maps generator from mapsme team is broken again again, but there is some end-user weekly updates site: https://hide.webhop.me/mapsme/. New regions could be requested here https://forum.openstreetmap.org/viewtopic.php?id=68433 (but there is no guarantee that they will be added).

M-Murat commented 4 years ago

In INSTALL.md there is a link to http://direct.mapswithme.com/direct/latest/ It doesn’t work. I would like to use it as a developer. Please, update the file or make the link working.

h1z1 commented 4 years ago

This appears to be down again or some other protection is preventing updates? Keep getting 403s.

nobodyisherecu commented 2 years ago

Almost 403 😐

cscode2000 commented 2 years ago

Still getting 403. Very sad. I need to transfer maps to a device with no internet connection. It's no problem to download the latest apk, so there should be no problem to download the latest mwm, too.