AtlisInc / Nominatim-API

Java library for reverse geocoding for Nominatim
http://nominatim.openstreetmap.org
GNU Lesser General Public License v3.0
14 stars 3 forks source link

Broken down #6

Closed azaSilaS closed 5 years ago

azaSilaS commented 5 years ago

It seems to getAddressFromMapPoint give every time NULL since 2 months ago.

String endpointUrl = "https://nominatim.openstreetmap.org/"; Address address = NominatimAPI.with(endpointUrl).getAddressFromMapPoint(point);

The lat,lon is perfect. The host is not disabled, because outuse! A simple wget does the job on the same host.

SilaS

michaelassraf commented 5 years ago

https://nominatim.openstreetmap.org/ is just a public server which generously allowed public access from the outside, if it doesn't work you can build you our own nominatim server.

azaSilaS commented 5 years ago

Hi Michael,

Thnx for quick response! As I wrote to You it works from the same host with wget (simple http client) in any way, but dont worked with atlisinc api!!!! So it seems, not the nominatim server is the caiuse of error! If I build an own one, it will producnig the same error!

SilaS

2018-10-26 16:00 keltezéssel, Michael Assraf írta:

https://nominatim.openstreetmap.org/ is just a public server which generously allowed public access from the outside, if it doesn't work you can build you our own nominatim server http://nominatim.org/release-docs/latest/admin/Installation/.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433417807, or mute the thread https://github.com/notifications/unsubscribe-auth/AAe8ew3KLHwXwWYQfNvTkWY9hIzdznnRks5uoxWKgaJpZM4X8Rnn.

michaelassraf commented 5 years ago

Can you set level log to "debug" so we will see the difference in the url structrue between your wget request and line 107 here?

azaSilaS commented 5 years ago

Can You help me how to do it? I am working with netbeans and glassfish

2018-10-26 16:21 keltezéssel, Michael Assraf írta:

Can you set level log to "debug" so we will see the difference in the url structrue between your wget request and line 107 here https://github.com/AtlisInc/Nominatim-API/blob/master/src/main/java/com/atlis/location/nominatim/NominatimAPI.java?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433424319, or mute the thread https://github.com/notifications/unsubscribe-auth/AAe8e4F30JGJuLEgwt6GldvBfpJjZ8sqks5uoxpMgaJpZM4X8Rnn.

michaelassraf commented 5 years ago

You can also do that using debug breakpoint, just download the sources using netbeans and place the breakpoint.

On Fri, Oct 26, 2018, 5:26 PM azaSilaS notifications@github.com wrote:

Can You help me how to do it? I am working with netbeans and glassfish

2018-10-26 16:21 keltezéssel, Michael Assraf írta:

Can you set level log to "debug" so we will see the difference in the url structrue between your wget request and line 107 here < https://github.com/AtlisInc/Nominatim-API/blob/master/src/main/java/com/atlis/location/nominatim/NominatimAPI.java ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433424319>,

or mute the thread < https://github.com/notifications/unsubscribe-auth/AAe8e4F30JGJuLEgwt6GldvBfpJjZ8sqks5uoxpMgaJpZM4X8Rnn .

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433426224, or mute the thread https://github.com/notifications/unsubscribe-auth/AKzsXIJKjZJTonH1A8bPmQgess2tIvoEks5uoxulgaJpZM4X8Rnn .

azaSilaS commented 5 years ago

So: YOur URL is this: https://nominatim.openstreetmap.org//reverse.php?osm_type=N&format=json&zoom=18&lon=19.301463&accept-language=en&lat=46.262944

If I place it into wget without "", then:

wget -d https://nominatim.openstreetmap.org//reverse.php?osm_type=N&format=json&zoom=18&lon=19.301463&accept-language=en&lat=46.262944 ---request begin--- GET //reverse.php?osm_type=N HTTP/1.1 User-Agent: Wget/1.17.1 (linux-gnu) Accept: / Accept-Encoding: identity Host: nominatim.openstreetmap.org Connection: Keep-Alive

---request end--- HTTP request sent, awaiting response... accept-language=en: command not found

---response begin--- HTTP/1.1 400 Bad Request Date: Fri, 26 Oct 2018 15:35:40 GMT Server: Apache/2.4.29 (Ubuntu) Access-Control-Allow-Origin: * Access-Control-Allow-Methods: OPTIONS,GET Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly" Upgrade: h2 Connection: Upgrade, close Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8

---response end--- 400 Bad Request Registered socket 3 for persistent reuse. Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true Updated HSTS host: nominatim.openstreetmap.org:443 (max-age: 31536000, includeSubdomains: true) URI content encoding = ‘utf-8’ Skipping 369 bytes of body: [

Bad Request

Nominatim has encountered an error with your request.

Details: Need coordinates or OSM object to lookup.

If you feel this error is incorrect feel file an issue on Github. Please include the error message above and the URL you used.

] done. 2018-10-26 15:35:40 ERROR 400: Bad Request.

Saving HSTS entries to /home/silas/.wget-hsts

If I place it into "":

wget -d "https://nominatim.openstreetmap.org//reverse.php?osm_type=N&format=json&zoom=18&lon=19.301463&accept-language=en&lat=46.262944" ---request begin--- GET //reverse.php?osm_type=N&format=json&zoom=18&lon=19.301463&accept-language=en&lat=46.262944 HTTP/1.1 User-Agent: Wget/1.17.1 (linux-gnu) Accept: / Accept-Encoding: identity Host: nominatim.openstreetmap.org Connection: Keep-Alive

---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Date: Fri, 26 Oct 2018 15:37:35 GMT Server: Apache/2.4.29 (Ubuntu) Access-Control-Allow-Origin: * Access-Control-Allow-Methods: OPTIONS,GET Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Expect-CT: max-age=0, report-uri="https://openstreetmap.report-uri.com/r/d/ct/reportOnly" Upgrade: h2 Connection: Upgrade, close Transfer-Encoding: chunked Content-Type: application/json; charset=UTF-8

---response end--- 200 OK Registered socket 3 for persistent reuse. Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true Updated HSTS host: nominatim.openstreetmap.org:443 (max-age: 31536000, includeSubdomains: true) URI content encoding = ‘UTF-8’ Length: unspecified [application/json] Saving to: ‘reverse.php?osm_type=N&format=json&zoom=18&lon=19.301463&accept-language=en&lat=46.262944’

reverse.php?osm_type=N& [ <=> ] 530 --.-KB/s in 0s

2018-10-26 15:37:35 (30.0 MB/s) - ‘reverse.php?osm_type=N&format=json&zoom=18&lon=19.301463&accept-language=en&lat=46.262944’ saved [530]

Saving HSTS entries to /home/silas/.wget-hsts

Yes this is blaming "accept-language"! But in "" not blames and works?!?!?!?!?!??!?!

Is'nt possible that the Agent reporting something like this is missing? connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11");

SilaS

2018-10-26 16:38 keltezéssel, Michael Assraf írta:

You can also do that using debug breakpoint, just download the sources using netbeans and place the breakpoint.

On Fri, Oct 26, 2018, 5:26 PM azaSilaS notifications@github.com wrote:

Can You help me how to do it? I am working with netbeans and glassfish

2018-10-26 16:21 keltezéssel, Michael Assraf írta:

Can you set level log to "debug" so we will see the difference in the url structrue between your wget request and line 107 here <

https://github.com/AtlisInc/Nominatim-API/blob/master/src/main/java/com/atlis/location/nominatim/NominatimAPI.java

?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <

https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433424319>,

or mute the thread <

https://github.com/notifications/unsubscribe-auth/AAe8e4F30JGJuLEgwt6GldvBfpJjZ8sqks5uoxpMgaJpZM4X8Rnn

.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub

https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433426224, or mute the thread

https://github.com/notifications/unsubscribe-auth/AKzsXIJKjZJTonH1A8bPmQgess2tIvoEks5uoxulgaJpZM4X8Rnn .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433430215, or mute the thread https://github.com/notifications/unsubscribe-auth/AAe8ewt-QfRWb3yzJ4BT0tvZUVEPmSCEks5uox5ZgaJpZM4X8Rnn.

michaelassraf commented 5 years ago

Hi,

I just ran a test method inside the library with your latitude and longtitude, and I got a perfecally fine result:

    @Test
    public void testMapPointToAddress() {
        List<Pair<Double, Double>> points
                = Arrays.asList(
                        new Pair<>(46.262944, 19.301463));
        for (Pair<Double, Double> point : points) {
            MapPoint mapPoint = new MapPoint().buildMapPoint(point.getKey(), point.getValue());
            Address address = NominatimAPI.with(endpointUrl).getAddressFromMapPoint(mapPoint);
            assertNotNull("Address object is bad.", address);
        }
    }

This is the result:

precision=0,
county=Bács-Kiskun
postcode=6440
country_code=hu
countryCode=HU
country=Hungary
city=Jánoshalma
displayName=Jánoshalma, Jánoshalmai járás, Bács-Kiskun, Southern Great Plain, Great Plain and North, 6440, Hungary

As you can see it's really not null.

So can you please tell me again what's the issue, this time with no question and exclamation marks.

Michael

azaSilaS commented 5 years ago

Hi Michael,

and here is my source, which producing null:

 public void flightGeoLocGenerate() throws InterruptedException {
     String endpointUrl = "https://nominatim.openstreetmap.org/";
     Integer num = 0;
     Integer fail = 0;
     for (Flight f : flightFacade.findAll()) {
         if ((f.getPlaceId().getLongitude() != null) && (f.getPlaceId().getLatitude() != null)) { // van koordinátája
             if ((!"".equals(f.getPlaceId().getLongitude())) && (!"".equals(f.getPlaceId().getLatitude()))) { // van koordinátája
                 if (f.getPlaceId().getDisplay_name() == null) { // ha már beazonosítottuk hagyjuk békén
                     try { // simán elszállhat formátum hibával.
                         Double lon = Double.parseDouble(f.getPlaceId().getLongitude());
                         Double lat = Double.parseDouble(f.getPlaceId().getLatitude());
                         if ((lon > 1) && (lon < 180) && (lat > 1) && (lat < 90)) { // ne foglalkozzunk a beltéri vagy GPS hibás helyek visszafejtésével
                             num++;
                             MapPoint point = new MapPoint().buildMapPoint(lat, lon);
                             Address address = NominatimAPI.with(endpointUrl).getAddressFromMapPoint(point);
                             if (address != null) {
                                 LOG.log(Level.INFO, "GeoLocGen : {0} ", f.getPlaceId().getLatitude() + " " + f.getPlaceId().getLongitude() + "->" + address.getDisplayName() + " " + address.getCity() + " " + address.getTown() + " " + address.getCountryCode());

I did not see any stamp of Agent in header or initialization. Nominatim say they need it! Is'nt be a caouse that, wget do the job, but the API missed?

SilaS

2018-10-27 15:13 keltezéssel, Michael Assraf írta:

Hi,

I just ran a test method inside the library with your latitude and longtitude, and I got a perfecally fine result:

|@Test public void testMapPointToAddress() { List<Pair<Double, Double>> points = Arrays.asList( new Pair<>(46.262944, 19.301463)); for (Pair<Double, Double> point : points) { MapPoint mapPoint = new MapPoint().buildMapPoint(point.getKey(), point.getValue()); Address address = NominatimAPI.with(endpointUrl).getAddressFromMapPoint(mapPoint); assertNotNull("Address object is bad.", address); } } |

This is the result:

|precision=0, county=Bács-Kiskun postcode=6440 country_code=hu countryCode=HU country=Hungary city=Jánoshalma displayName=Jánoshalma, Jánoshalmai járás, Bács-Kiskun, Southern Great Plain, Great Plain and North, 6440, Hungary |

As you can see it's really not null.

So can you please tell me again what's the issue, this time with no question and exclamation marks.

Michael

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/AtlisInc/Nominatim-API/issues/6#issuecomment-433619177, or mute the thread https://github.com/notifications/unsubscribe-auth/AAe8e_GCg2dVlVgcigMZzSxrtrMUwWkBks5upFv-gaJpZM4X8Rnn.

michaelassraf commented 5 years ago

The API uses Java http client not underlying wget, as you can see in the source code the user-agent is set to Firefox.

I couldn't reproduce your issue which may indicate you hit a rate limit of https://nominatim.openstreetmap.org so the fact that you manage to make a call from wget is because the user-agent or the source IP are different, but in both cases I can't see how changing anything in the library will help you with this hurdle.

azaSilaS commented 5 years ago

If it is, then a variable to set the agent..... A Huawei mobilomról küldve-------- Eredeti üzenet --------Tárgy: Re: [AtlisInc/Nominatim-API] Broken down (#6)Feladó: Michael Assraf Címzett: AtlisInc/Nominatim-API Másolatot kap: azaSilaS ,Author The API uses Java http client not underlying wget, as you can see in the source code the user-agent is set to Firefox. I couldn't reproduce your issue which may indicate you hit a rate limit of https://nominatim.openstreetmap.org so the fact that you manage to make a call from wget is because the user-agent or the source IP are different, but in both cases I can't see how changing anything in the library will help you with this hurdle.

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or mute the thread.

azaSilaS commented 5 years ago

Finaly I found the causes of error:

  1. It needs cacerts in cacerts.jks if you have -Djavax.net.ssl.trustStore in java environment openssl x509 -in <(openssl s_client -connect nominatim.openstreetmap.org:443 -prexit 2>/dev/null) -out ~/nominatim.openstreetmap.org.crt keytool -importcert -file ~/nominatim.openstreetmap.org.crt -alias example -keystore /opt/glassfish5.0.1/glassfish/domains/domain1/config/cacerts.jks -storepass changeit
  2. It need "email=somebody@somewhere.com" in query string.

SilaS