Closed jperedadnr closed 4 years ago
Sorry, but faking the user-agent is a bad idea. This app seems to violate OSM's tile usage policy. OSM servers run entirely on donated resources. Please set a valid user-agent pointing to Gluon Maps, not to Mozilla.
Hi @scaidermern, thanks for reporting.
We don't intend to violate any policy of course. We just found out about the issue with the http response errors a few days ago, and while we don't have a definitive fix yet, we are just trying some workarounds.
When running on Desktop there is no http.agent
defined, while on Android default agent, which is already is giving something like:
Dalvik/2.1.0 (Linux; U; Android 9; Pixel XL Build/PQ3A.190505.001)
based on the user's device.
So we can set for instance:
static {
String httpAgent = System.getProperty("http.agent");
if (httpAgent == null) {
httpAgent = "(" + System.getProperty("os.name") + " / " + System.getProperty("os.version") + " / " + System.getProperty("os.arch") + ")";
}
System.setProperty("http.agent", "Gluon Mobile/1.0.3 " + httpAgent);
}
Would that work?
. Of course, we can add to that:
static {
System.setProperty("http.agent", "Gluon Mobile/1.0.3 /" + System.getProperty("http.agent"));
}
Yes, looks fine as long as the string "Gluon Mobile" will be included in the user-agent.
Dear Jose Perada I'm very happy to find your issue. I'm in trouble because I cannot display OpenStreetMap in MATSim. MATSim is a traffic simulation written in Java. There is a description that reads OSM file in TileFactoryBuilder class in MATSim. I would be grateful if you coud give me advice on how to add the your proposal method that add http.agent. The contents of TileFactoryBuilder.class are described below.
/* *** *
package org.matsim.evacuationgui.jxmapviewerhelper;
import org.jdesktop.swingx.mapviewer.DefaultTileFactory; import org.jdesktop.swingx.mapviewer.TileFactory; import org.jdesktop.swingx.mapviewer.TileFactoryInfo; import org.jdesktop.swingx.mapviewer.wms.WMSService;
public abstract class TileFactoryBuilder {
public static TileFactory getOsmTileFactory() {
final int max=17;
TileFactoryInfo info = new TileFactoryInfo(0, 17, 17,
256, true, true,
"http://tile.openstreetmap.org",
"x","y","z") {
@Override
public String getTileUrl(int x, int y, int zoom) {
zoom = max-zoom;
String url = this.baseURL +"/"+zoom+"/"+x+"/"+y+".png";
return url;
}
};
TileFactory tf = new DefaultTileFactory(info);
return tf;
}
public static TileFactory getWMSTileFactory(String baseURL, String layer) {
WMSService service = new WMSService(baseURL, layer);
return new WMSTileFactory(service,17);
}
private static final class WMSTileFactory extends DefaultTileFactory {
public WMSTileFactory(final WMSService wms, final int maxZoom) {
super(new TileFactoryInfo(0, maxZoom, maxZoom,
256, true, true, // tile size and x/y orientation is r2l & t2b
"","x","y","zoom") {
@Override
public String getTileUrl(int x, int y, int zoom) {
int zz = maxZoom - zoom;
int z = (int)Math.pow(2,(double)zz-1);
return wms.toWMSURL(x-z, z-1-y, zz, getTileSize(zoom));
}
});
}
}
}
@kazshimda Did you see the changes in this commit https://github.com/jperedadnr/maps/commit/1471ef434e5bbf2feda4de656b99f739e542a770 ? You could add something like that, replacing Gluon Maps with your MatSim or alike.
Thank you for your quick reply. The place where I display OSM is not in the web browser, but in the Graphical User Interface of MATSim. I'm looking for a setting of user-agent notation instead of Gluon Maps that you use.
On the other hand, the OSM tile web site may change around June this year, which has changed the way to get OSM tile from java programs. Do you know if there was any announcement from the OSM administrator? Or Do you know which OSM representative to contact about this issue?
Gluon Maps doesn't use a web browser either. We simply add that commit I linked before to the Java code.
By the way, have you tried using https
instead of http
(as in https://tile.openstreetmap.org
)? Probably this will solve your issues. I recall this being the main change from a few months ago that you also mentioned.
@jperedadnr Thank you for your reply I changed to https and run MATSim program, but OSM was not displayed in MATSim's GUI.
The error message of MATSim is described below. I'm thinking about and trying which class of program and where in the program to add about http.agent like you. It would be helpful if you could give me some advice.
Failed to load a tile at url: http://tile.openstreetmap.org/15/8635/12957.png
java.io.IOException: Server returned HTTP response code: 403 for URL: http://tile.openstreetmap.org/15/8635/12957.png
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at org.jdesktop.swingx.mapviewer.AbstractTileFactory$TileRunner.cacheInputStream(AbstractTileFactory.java:348)
at org.jdesktop.swingx.mapviewer.AbstractTileFactory$TileRunner.run(AbstractTileFactory.java:307)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
It doesn't look like you applied https, it shows you are trying to load from http. I've already mentioned the fix we have done in Gluon Maps, that still uses http. With https it shouldn't even be necessary.
@jperedadnr Sorry the error message I posted earlier was an error in http status. The error when changing to https is described below. The error content looks almost same without at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
.
Failed to load a tile at url: https://tile.openstreetmap.org/15/8635/12956.png
java.io.IOException: Server returned HTTP response code: 429 for URL: https://tile.openstreetmap.org/15/8635/12956.png
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at org.jdesktop.swingx.mapviewer.AbstractTileFactory$TileRunner.cacheInputStream(AbstractTileFactory.java:348)
at org.jdesktop.swingx.mapviewer.AbstractTileFactory$TileRunner.run(AbstractTileFactory.java:307)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Error code 429 means that you are blocked most likely because of a rate limit that has been exceeded. This happens when you try to download too many tiles in a short period of time. See https://operations.osmfoundation.org/policies/tiles/ for the tile usage policy.
@tiainen Thank you for telling me what my error is. Error number 429 indicates that there is a lot of data, so I tried using small OSM data. But again, the same 429 error was displayed. From this result, I think that the size of the map imported from OSM is not the cause. I will continue the investigation. Thank you all.
Hi @kazshimda,
I will close this issue now. If the issue still persists, please free to reopen.
Recently OpenStreetMap rejects the request to retrieve tiles. For
http://tile.openstreetmap.org/
, it returns on every request:Changing to
https
doesn't solve the issue, though, as it rejects it now with a different error:As the URL (http or https) works fine on a browser, it seems that adding
user-agent
to the URLConnection request could solve it. However, the tiles return blank.A working solution for Desktop is this: setting
http.agent
system property:For Android:
Related StackOverflow question: https://stackoverflow.com/questions/56397206/gluon-maps-doesnt-load-the-map-and-throws-an-exception