MapQuest geocoding does not work

[Lineflyer]

Describe your problem!

Edit: ticket 831920

A user reported, that on his devices without GooglePlayServices he is not able to perform a adress search (reverse geocoding required). The problem appeared recently and worked before on same devices.

AFAIK we use MapQuest as a fallback if no Google Services are available. Looking at the users debug log I do see a 401 (Unauthorized) response from MapQuest.

How to reproduce?

• Device without Google Play Services
• Search for address like "Köln"

Actual result after these steps?

01-17 19:14:48.126 22741 22741 D cgeo    : [main] HTTP-GET: https://www.mapquestapi.com/geocoding/v1/address?location=Berlin&maxResults=20&thumbMaps=false&key=Fmjtd%7Cluurn1u2n9%2Cbs%3Do5-9wynua {Network.request:363/Network.requestJSON:541/MapQuestGeocoder.get:52/MapQuestGeocoder.getFromLocationName:38/AddressListActivity.lambda$lookupAddressInBackground$0:48/AddressListActivity.$r8$lambda$oZE_3W1ddQ8AfrqsXubhSW5RSBA:0/AddressListActivity$$ExternalSyntheticLambda0.apply:0/ObservableOnErrorNext$OnErrorNextObserver.onError:82}
01-17 19:14:48.127 22741 23023 D cgeo    : [OkHttp] HTTP-REQ:GET https://www.mapquestapi.com/geocoding/v1/address?location=Berlin&maxResults=20&thumbMaps=false&key=Fmjtd%7Cluurn1u2n9%2Cbs%3Do5-9wynua, headers=[Accept:application/json, text/javascript, */*; q=0.01;Accept-Charset:utf-8,iso-8859-1;q=0.8,utf-16;q=0.8,*;q=0.7;Accept-Language:en-US,*;q=0.9;User-Agent:Mozilla/5.0 (X11; Linux x86_64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1 cgeo/2024.01.06;X-Requested-With:XMLHttpRequest] {Network$LoggingInterceptor.intercept:404/RealInterceptorChain.proceed:109/Network$HeadersInterceptor.intercept:392/RealInterceptorChain.proceed:109/RealCall.getResponseWithInterceptorChain$okhttp:201/RealCall$AsyncCall.run:517/ThreadPoolExecutor.runWorker:1137/ThreadPoolExecutor$Worker.run:637}
01-17 19:14:48.127 22741 23023 D cgeo    : [OkHttp] HTTP-COOKIES: SEND for https://www.mapquestapi.com/geocoding/v1/address?location=Berlin&maxResults=20&thumbMaps=false&key=Fmjtd%7Cluurn1u2n9%2Cbs%3Do5-9wynua:  {Cookies$InMemoryCookieJar.loadForRequest:77/BridgeInterceptor.intercept:74/RealInterceptorChain.proceed:109/RetryAndFollowUpInterceptor.intercept:76/RealInterceptorChain.proceed:109/Network$LoggingInterceptor.intercept:408/RealInterceptorChain.proceed:109/Network$HeadersInterceptor.intercept:392}
01-17 19:14:48.718 22741 23023 D cgeo    : [OkHttp] HTTP-RESP:401 [] (591 ms) GET https://www.mapquestapi.com/geocoding/v1/address?location=Berlin&maxResults=20&thumbMaps=false&key=Fmjtd%7Cluurn1u2n9%2Cbs%3Do5-9wynua (h2), headers=[access-control-allow-origin:*;content-length:50;content-type:text/plain; charset=UTF-8;date:Wed, 17 Jan 2024 18:14:49 GMT;x-powered-by:MapQuest] {Network$LoggingInterceptor.intercept:414/RealInterceptorChain.proceed:109/Network$HeadersInterceptor.intercept:392/RealInterceptorChain.proceed:109/RealCall.getResponseWithInterceptorChain$okhttp:201/RealCall$AsyncCall.run:517/ThreadPoolExecutor.runWorker:1137/ThreadPoolExecutor$Worker.run:637}
01-17 19:14:48.765 22741 22741 D cgeo    : [main] [cgeo.geocaching.address.AddressListActivity].showToast(c:geo konnte keine Koordinaten für diese Adresse ermitteln){1} {ActivityMixin.showCgeoToast:87/ActivityMixin.lambda$postShowToast$0:102/ActivityMixin.$r8$lambda$L4ZH2fiOtFB_ifd1mDwiDWNUYZs:0/ActivityMixin$$ExternalSyntheticLambda0.run:0/Activity.runOnUiThread:7368/ActivityMixin.postShowToast:102/ActivityMixin.showToast:121/AbstractActivity.showToast:88}

Expected result after these steps?

Valid response

Reproducible

Unclear

c:geo Version

2024.01.06

System information

No response

Additional Information

Maybe the API conditions changed for MapQuest or we hit a limit (but I would assume another error code in such case)

[Lineflyer]

Support ticket 729689

[devemux86]

See here:

• https://github.com/cgeo/cgeo/issues/15057

[moving-bits]

We might use open source Nominatim as fallback for Android's integrated geocoding. Usage policy can be found here.

[SammysHP]

No heavy uses (an absolute maximum of 1 request per second). […] Note that the usage limits above apply per website/application: the sum of traffic by all your users should not exceed the limits.

Apps must make sure that they can switch the service at our request at any time (in particular, switching should be possible without requiring a software update). If at all possible, set up a proxy and also enable caching of requests.

[moving-bits]

Yes, both limitations have to be taken into account.

No heavy uses (an absolute maximum of 1 request per second). […] Note that the usage limits above apply per website/application: the sum of traffic by all your users should not exceed the limits.

Default option is (and should stay at) internal Android geocoding (using Play services), so MapQuest (and Nominatim, if implemented) are only used as fallback, thus for a much smaller user group.

Apps must make sure that they can switch the service at our request at any time (in particular, switching should be possible without requiring a software update). If at all possible, set up a proxy and also enable caching of requests.

I guess we will receive some http 429 or similar if the limit is hit, which c:geo could react on by temporarily disabling geocoding fallback.

If it is about a call from them "stop using our geocoder NOW", then we will have to implement some online flag (or implement the mentioned proxy - not sure how much work that would be).

[Lineflyer]

As mapquest is now completely unavailable I guess we should implement Nominatim as fallback for non-Google devices...shall we?

[moving-bits]

I'm playing around a bit with the Nominatim API currently - seems to be easy enough to use for our use-cases. I can dig a bit deeper into it.

To comply with their usage policy we should set up a caching proxy.

@Lineflyer: Are we able to run a small PHP script and a MySQL/MariaDB database somewhere? This server should be reachable 24/7 (more or less).

[Lineflyer]

@Lineflyer: Are we able to run a small PHP script and a MySQL/MariaDB database somewhere? This server should be reachable 24/7 (more or less).

I can offer my hosted webspace for that. Its PHP and SQL capable. Would only need to check if I have free SQL-DB instances in my package, but I guess so. You are thinking of a request cache / proxy?

Just tell me, what I shall do and I can make it available on short notice.

[moving-bits]

You are thinking of a request cache / proxy?

Yeah, c:geo would query our server - our server would look up in its own db and report back, if found, or otherwise query the remote server, store the result locally and report back to c:geo. Some automatic maintenance could be added, eg. "delete all results that are older than one month" or so, to keep the database in shape.

AFAICS we have two places where we need geocoding:

• home page: coords => address (if enabled)
• search by address: string to address (and its coords)

Do you know of any other places?

[Lineflyer]

Do you know of any other places?

No, thats all AFAIK.

Some automatic maintenance could be added, eg. "delete all results that are older than one month" or so, to keep the database in shape.

Lets see how many entries we get. As those entries usually valid almost forever, we could create a growing databse ourselves with this.

That makes me think: Isn't there some database + code publicly available to completely host it ourselves?

[moving-bits]

That makes me think: Isn't there some database + code publicly available to completely host it ourselves?

Yes, it is. For self-hosting of a world file they recommend 128 GB of RAM :-)

[fm-sys]

Why do you think a caching proxy would help reducing the calls in any way? How likely is it that people ever search for exactly the same location a significant number of times?

Another question (sorry if it was already discussed): Would we use it for address search and/or also for geopoint to city name convertion?

moving-bits @.***> schrieb am Sa., 16. März 2024, 19:19:

I'm playing around a bit with the Nominatim API currently - seems to be easy enough to use for our use-cases. I can dig a bit deeper into it.

To comply with their usage policy we should set up a caching proxy.

@Lineflyer https://github.com/Lineflyer: Are we able to run a small PHP script and a MySQL/MariaDB database somewhere? This server should be reachable 24/7 (more or less).

— Reply to this email directly, view it on GitHub https://github.com/cgeo/cgeo/issues/15265#issuecomment-2002075781, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMW4ZXPUU4JJCNGQVWCNVTYYSEJNAVCNFSM6AAAAABCT2K4L6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGA3TKNZYGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[moving-bits]

Why do you think a caching proxy would help reducing the calls in any way? How likely is it that people ever search for exactly the same location a significant number of times?

That depends on how specific the search is: Searching for "Tante Emmas Brötchenladen, Sylt" will probably being searched for only rarely, whereas searches for cities will be quite frequently (eg. "Berlin", "London", ...)

Besides that, implementing a proxy is a prerequisite for using the OSM nominatim server, defined by their usage policy.

Another question (sorry if it was already discussed): Would we use it for address search and/or also for geopoint to city name convertion?

Yes, both cases will be covered.

I'm in contact with @Lineflyer wrt to the server-side details, PR for the c:geo-side client will follow after the proxy is live.

[moving-bits]

FYI: Just found out that reverse geocoding ("coords to address" on home screen) always uses the builtin AndroidGeocoder as of today, with no fallback to MapQuest. Will be fixed with the switch from MapQuestGeocoder to OsmNominatumGeocoder.

[Lineflyer]

@moving-bits I chatted with @mucek4 and he will create a CNAME for api.cgeo.org to the URL we discussed. So you shall use that as the endpoint in c:geo code and we could direct it to whatever server we have in background in future.