Identify heavy app tile users by User-Agent

pnorman commented 4 years ago

The OWG could use help identifying these high-volume User Agents. We can contact them if necessary, but could use help with the identifying.

I'll be editing this post to keep the information consolidated, and hiding messages as I integrate them.

High usage app User-Agents

Daily hits	User-Agent	Contact
2560964	GDAL WMS driver (http://www.gdal.org/frmt_wms.html)		Allow for now
2483953	ArcGIS Client Using WinInet		Allow for now
1836191	Maperitive/2.4.3		Allow for now
1146725	cgeo	https://github.com/cgeo/cgeo/issues
684191	sunriseinnovations.WIS.beta	https://www.sunriseinnovations.ie/
503447	Nemo Outdoor	https://www.keysight.com/main/contactInformation.jspx?nid=-11143.0.00&lc=eng&cc=US
391522	ir.pakcharkh.bdood/19	https://cafebazaar.ir/app/ir.pakcharkh.bdood?l=en
379844	com.taxiadmins.client	https://play.google.com/store/apps/details?id=com.taxiadmins.client
377245	uz.smartlab.itaxi.client	https://play.google.com/store/apps/details?id=uz.smartlab.itaxi.client
371303	GeliosMobile/98 CFNetwork/1121.2.2 Darwin/19.3.0	https://www.geliossoft.com/contacts
350733	com.rsm.muoversiaroma	https://play.google.com/store/apps/details?id=com.rsm.muoversiaroma
335876	staticmaps@1.1.4 (Node.js v12.14.1)		Someone using https://github.com/StephanGeorg/staticmaps
334280	com.qualteh.mobtaxi.android.client	https://play.google.com/store/apps/details?id=com.qualteh.mobtaxi.android.client
317557	OruxMaps v.7.5.8 GP	https://play.google.com/store/apps/details?id=com.orux.oruxmapsDonate
310438	com.helleniccomms.makedonia.driver	https://play.google.com/store/apps/details?id=com.helleniccomms.makedonia.driver
300515	in.kipo	https://play.google.com/store/apps/details?id=in.kipo
290211	it.Beta80Group.whereareu	https://play.google.com/store/apps/details?id=it.Beta80Group.whereareu
287639	1.71.7
281412	OfflineMaps/3.4 (om86; ar; +https://www.offline-maps.net)	https://play.google.com/store/apps/details?id=net.psyberia.offlinemaps
277342	Other
272811	GeoTiler/0.14.3
266006	ArcGISRuntime-Android/100.2.1 (Android 5.1; armeabi-v7a; SAMSUNG-SM-T365)
264563	RT3000host
261361	com.tiskel.terminal
259940	OruxMaps v.7.4.22	https://play.google.com/store/apps/details?id=com.orux.oruxmapsDonate
254269	de.schildbach.oeffi	https://schildbach.de/
244848	com.helleniccomms.glyfada.driver
231130	Nokia (Qt) Graphics Dojo 1.0
224615	ArcGIS.JavaSE.10.1.1
224314	at.tbirn.lfkapp/21230501
216037	com.caynax.sportstracker/20601
214502	com.mivselltech.mobilesales2
206937	com.taxi505.driver/135
204403	ru.taxomet.tadriver
197363	com.softlogics.client

Contacted

Daily hits	User-Agent	Contact
870173	pl.itaxi.driver	https://itaxi.pl/english/
576751	net.uztaxi.driver	https://play.google.com/store/apps/details?id=net.uztaxi.driver
498789	OSMDroid/2.1 (its; rutaxi 3.28.0)	https://play.google.com/store/apps/details?id=com.its.rto
482874	com.helleniccomms.mercedes.driver	https://play.google.com/store/apps/details?id=com.helleniccomms.mercedes.driver
469157	ru.taximaster.www	https://play.google.com/store/apps/details?id=ru.taximaster.www
453734	com.arobs.trackgps	https://play.google.com/store/apps/details?id=com.arobs.trackgps
409986	com.helleniccomms.asteras.driver	https://play.google.com/store/apps/details?id=com.helleniccomms.asteras.driver

Newly blocked UAs

Daily hits	User-Agent	Contact
1784800	Where my children/2.0.9 (iPhone; iOS 13.3.1; Scale/2.00)		Blocked
1730899	Where my children/2.0.9 (iPhone; iOS 13.3.1; Scale/3.00)		Blocked
1530710	nossoonibusjp.android.crosswalk	https://www.jampabus.com.br/	No attribution
711132	br.com.concisoti.potybus	https://play.google.com/store/apps/details?id=br.com.concisoti.potybus&hl=en_US	No attribution, same as Jampabus
784944	com.soft373.taptaxi	https://taptaxi.su/
770529	com.kradac.ktxcore	https://kradac.com/
709664	ru.crowdsystems.topcontrol.knd	https://crowdsystems.ru/en
700755	C# TilesDownloader
688465	okhttp/4.2.2		Generic android HTTP framework
543909	kc_android
507506	Android
497577	MapDownloader
486335	/n software IPWorks HTTP/S Component - www.nsoftware.com	https://www.nsoftware.com/company/contact.aspx
324496	Where my children/2.0.9 (iPhone; iOS 13.3; Scale/2.00)		Blocked
285467	Where my children/2.0.9 (iPhone; iOS 13.3; Scale/3.00)
241033	Where my children/2.0.9 (iPhone; iOS 12.4.5; Scale/2.00)
221438	okhttp/3.12.2		Generic android HTTP framework
213015	Microsoft-ATL-Native/12.00

Total requests without referer: 529910207

scaidermern commented 4 years ago

Nemo Outdoor seems to be the software Nemo Outdoor by Keysight: https://www.keysight.com/en/pd-2765544-pn-nta00000a/nemo-outdoor. It features an OSM-based map, see https://about.keysight.com/en/newsroom/images/NBIoT/. Contact: https://www.keysight.com/main/contactInformation.jspx?nid=-11143.0.00&lc=eng&cc=US

OSMDroid/2.1 (its; rutaxi 3.28.0) is the Android app Rutaxi.Online https://play.google.com/store/apps/details?id=com.its.rto, contact: vezetaksi@gmail.com

com.helleniccomms.mercedes.driver is the Android app TaxiClub Driver https://play.google.com/store/apps/details?id=com.helleniccomms.mercedes.driver, contact: helleniccomms@gmail.com

ru.taximaster.www is the Android app TMDriver https://play.google.com/store/apps/details?id=ru.taximaster.www, contact: support@bitmaster.ru

uz.smartlab.itaxi.client is the Android app iTaxi.uz https://play.google.com/store/apps/details?id=uz.smartlab.itaxi.client, contact: info@itaxi.uz

com.arobs.trackgps is the Android app TrackGps https://play.google.com/store/apps/details?id=com.arobs.trackgps, contact: trackmobile@arobs.com

com.helleniccomms.asteras.driver is the Android app ΡΑΔΙΟΤΑΞΙ ΑΣΤΕΡΑΣ https://play.google.com/store/apps/details?id=com.helleniccomms.asteras.driver, contact: support@asteras1.gr

com.taxiadmins.client is the Android app ТА Водитель™ такси https://play.google.com/store/apps/details?id=com.taxiadmins.client, contact: info@taxiadmins.com

com.rsm.muoversiaroma is the Android app Roma Mobilità https://play.google.com/store/apps/details?id=com.rsm.muoversiaroma, contact: app@romamobilita.it

com.qualteh.mobtaxi.android.client is probably the Android app Tudo Taxi https://play.google.com/store/apps/details?id=com.qualteh.mobtaxi.android.client. However the screenshots don't show OSM tiles, looks like MapBox? Contact: info@qualteh.com

OruxMaps v.7.5.8 GP and OruxMaps v.7.4.22 is the Android app OruxMaps https://play.google.com/store/apps/details?id=com.orux.oruxmapsDonate, contact: oruxuro@gmail.com

com.helleniccomms.makedonia.driver is the Android app Radiotaxi Makedonia Driver https://play.google.com/store/apps/details?id=com.helleniccomms.makedonia.driver, contact: helleniccomms@gmail.com

it.Beta80Group.whereareu is the Android app 112 Where ARE U https://play.google.com/store/apps/details?id=it.Beta80Group.whereareu, contact: where@areu.lombardia.it

dorotheakazazi commented 4 years ago

com.helleniccomms.mercedes.driver com.helleniccomms.makedonia.driver com.helleniccomms.asteras.driver .. com.helleniccomms.mercedes.driver is the Android app TaxiClub Driver https://play.google.com/store/apps/details?id=com.helleniccomms.mercedes.driver, contact: helleniccomms@gmail.com .. com.helleniccomms.asteras.driver is the Android app ΡΑΔΙΟΤΑΞΙ ΑΣΤΕΡΑΣ https://play.google.com/store/apps/details?id=com.helleniccomms.asteras.driver, contact: support@asteras1.gr

@pnorman anything hellenic* is Greek-related, so if you want a Greek translation of the text when contacting them, let me know.

scaidermern commented 4 years ago

/n software IPWorks HTTP/S Component - www.nsoftware.com is obviously https://www.nsoftware.com/company/contact.aspx although I'm not sure where this software uses a map.

ir.pakcharkh.bdood/19 is the Android app بیدود (BDOOD) https://cafebazaar.ir/app/ir.pakcharkh.bdood?l=en

GeliosMobile/98 CFNetwork/1121.2.2 Darwin/19.3.0 is the Android/iOS app GeliosMobile by GeliosSoft https://www.geliossoft.com/contacts, screenshots with OSM-based maps can be found at https://apps.apple.com/th/app/geliosmobile/id999837601#?platform=ipad

OfflineMaps/3.4 (om86; ar; +https://www.offline-maps.net) is the Android app All-In-One Offline Maps https://play.google.com/store/apps/details?id=net.psyberia.offlinemaps

GeoTiler/0.14.3 are users of the GeoTiler library https://github.com/wrobell/geotiler

com.tiskel.terminal is a taxi management app by Tiskel https://www.tiskel.com/produkty?lang=en

com.helleniccomms.glyfada.driver is the Android app Radiotaxi Glyfadas Driver https://play.google.com/store/apps/details?id=com.helleniccomms.glyfada.driver

tastrax commented 4 years ago

I suspect that ArcGIS.JavaSE.10.1.1 is ArcGIS Runtime SDK for Android development

https://topic.alibabacloud.com/a/an-overview-of-the-arcgis-runtime-sdk-for-android-development-note-arcgis-runtime-sdk_1_12_30779938.html

tastrax commented 4 years ago

Maperitive/2.4.3 is likely http://maperitive.net/docs/

Lineflyer commented 4 years ago

Hello openstreetmap-team,

we (the c:geo team) were made aware about this issue recently. c:geo is an opensource free geocaching app for Android.

Some months ago we changed the c:geo app in a way that it provides a dedicated user-agent (instead of some default set by the mapsforge library) when requesting tiles to allow identification for your team. See this posting and the corresponding issue for details. AFAICS there has also been some contact to your team at this point of time.

From your table above I do now see, that we are under the top 5 of tile requests (as far as I understand your table), which I never would have expected... While we are just a small opensource team, without any commercial background, I would still like to ask, how you evaluate the situation of OSM tile usage by c:geo, and whether we can do anything to improve the load situation on your side or any other measure we can take to help you? Ideas and constructive discussion is always welcome.

pnorman commented 4 years ago

c:geo is identifying itself with a User-Agent, is open-source and a related activity, mentions OSM when you select the OpenStreetMap Carto basemap, and isn't using it as a default, so we're not looking at blocking it at this time. As our needs and load changes, this could change but we would attempt to contact you if it wasn't urgent.

Could you change the "OSM: Map" to "OpenStreetMap Standard" or something that has the full name?

From your table above I do now see, that we are under the top 5 of tile requests (as far as I understand your table), which I never would have expected...

This is surprising to me too. I'll open up an issue on your tracker with some ideas.

Firefishy commented 4 years ago

Offline mass down of tiles is not permitted, even by Open Source apps.

pnorman commented 4 years ago

Thanks everyone for helping identify the big users. We've been able to identify the big users and reach out to them. The remaining uses are into the long tail of small users, and if they stop being small users, we've got this to look back on and get their contact information.

I'm going to close this, noting that there's about half a dozen apps where we're giving them time to respond before blocking.

moving-bits commented 4 years ago

@pnorman
~Beside other means of reducing the load, like increased usage of offline map files, is there a way to make the MapsForge lib use a different tile server (eg cyclosm.openstreetmap.fr) to spread the load?~

sorry, wrong repo - will redirect to mapsforge repo

hbogner commented 4 years ago

Maperitive/2.4.3 is likely http://maperitive.net/docs/

Yes it is. I used it few years ago.

ghost commented 4 years ago

Maperitive/2.4.3 is likely http://maperitive.net/docs/

Yes it is.

No, http://maperitive.net/docs/ is Maperitive's docs pages.

Contact page (mail list) is here:

http://groups.google.com/group/maperitive

hbogner commented 4 years ago

Or contact the author directly: https://twitter.com/breki74

openstreetmap / operations