Closed zstadler closed 1 year ago
Reverse engineering overpass-turbo website:
nw
[amenity="drinking_water"]
(area:3606195356);
out meta geom;
// rel (6195356);out geom;
So basically, if I'm understanding this correctly I need to do the following steps:
Bottom line, I don't think this can be implemented with the short link due to how overpass works, please let me know what else is possible from your perspective as a user.
Thanks for the analysis!
https://stackoverflow.com/a/33380848 and responseURL doc suggest it is possible to capture the redirect URL in JavaScript:
const xhr = new XMLHttpRequest();
xhr.open('GET', 'https://overpass-turbo.eu/s/1nyV');
xhr.onload = () => {
console.log(xhr.responseURL);
};
xhr.send(null);
prints
https://overpass-turbo.eu/?Q=nw%0A%20%20%5Bamenity%3D%22drinking_water%22%5D%0A%20%20(area%3A3606195356)%3B%0Aout%20meta%20geom%3B%0A%0A%2F%2F%20rel%20(6195356)%3Bout%20geom%3B&C=31.40054;35.04639;7&R
The Q
parameter can then be sent directly to the https://overpass-api.de/api/interpreter
API as the data
in an HTTP POST request. For example:
curl -d 'data=nw%0A%20%20%5Bamenity%3D%22drinking_water%22%5D%0A%20%20(area%3A3606195356)%3B%0Aout%20meta%20geom%3B%0A%0A%2F%2F%20rel%20(6195356)%3Bout%20geom%3B' https://overpass-api.de/api/interpreter
I tired that, but since the overpass site doesn't support CORS the xhr request doesn't get to the onload
but rather gets to the onerror
, in this method the responseURL
field is empty and I couldn't find a field in the xhr object that holds the redirected address.
Any thoughts or links to SO to solve this would help, I couldn't find anything :-(
Moreover, I keep reading the following line in all the answers I read:
To answer your question: You can't handle redirects with XHR callbacks because the browser takes care of them automatically. You will only get back what at the redirected location.
https://stackoverflow.com/questions/15996779/cannot-handle-302-redirect-in-ajax-and-why
Discussion summary: let's jump over the current problem with the short URL and focus on getting this tested assuming the user pastes the full query URL so we can see how this looks and feels...
After skipping the URL redirect issue, I hard coded a lot of things and got the following to work:
I've added a source with the protocol of overpass://some-url-i-made-up-to-see-that-I-get-it
and I've added a very simple circle layer just to make sure the data is presented:
I think a fuller less hard coded example is needed in order to better understand the manual work that is needed in order to create a custom layer.
@zstadler can you please create a style somewhere (any public github repo will do) that has a style that should be constructed as follows:
...
sources: {
...
overpass-example: { // or any other key you want to give it
type: 'geojson',
data: "overpass://query-to-overpass-url-encoded" // the example we used in this issue is nw%0A%20%20%5Bamenity%3D%22drinking_water%22%5D%0A%20%20(area%3A3606195356)%3B%0Aout%20meta%20geom%3B%0A%0A%2F%2F%20rel%20(6195356)%3Bout%20geom%3B
}
},
layers: [
... // any way you would like to style this data that uses the overpass-example source
]
If this will work the only question left is if we insist on the short URL or not.
Here is a style file using circles similar to Overpass-Turbo: Overpass-overlay-example.json
Thanks!! Can you please remove the source-layer
from the layer definition? it shouldn't be there for geojson source.
Thanks!! this is how it looks :-)
Two things to consider
This is less than 15 lines of code in total :-)
There is a free un-shorten API - https://unshorten.me/api.
For example, https://unshorten.me/s/https://overpass-turbo.eu/s/1nyV
returns
https://overpass-turbo.eu/?Q=nw%0A%20%20%5Bamenity%3D%22drinking_water%22%5D%0A%20%20(area%3A3606195356)%3B%0Aout%20meta%20geom%3B%0A%0A%2F%2F%20rel%20(6195356)%3Bout%20geom%3B&C=31.40054;35.04639;7&R
One other thing to consider is the Overpass-Turbo feature that prevents small features from vanishing when zooming-out.
Overpass Turbo seems to do so by temporarily replacing the geometry of the feature with a point geometry when the size of the bounding box is below a threshold. The original geometry is restored when the appropriate zoom-in is done.
Their code is made for Leaflet.
The above doesn't support CORS unfortunately... :/
Access to XMLHttpRequest at 'https://unshorten.me/s/https://overpass-turbo.eu/s/1nyV' from origin 'http://localhost:5000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
In the style users are free to define min zoom, max zoom etc and style however they see fit, I don't think there's a need here to complicate things...
Bottom line, at this point I have no more intentions on pursuing the short URL issue.
The style editor should take the query, url encode it using an online tool like https://www.urlencoder.org/, add a prefix of overpass://
and set that as the value for the data
field in the geojson source, much like can be see in the following example from here:
{
"version": 8,
"name": "Overpass-overlay-example",
"metadata": {
"owner": "Zeev Stadler",
"copyright": "Copyright Zeev Stadler 2022. All Rights Reserved."
},
"sources": {
"overpass-points": {
"type": "geojson",
"data": "overpass://nw%0A%20%20%5Bamenity%3D%22drinking_water%22%5D%0A%20%20(area%3A3606195356)%3B%0Aout%20meta%20geom%3B%0A%0A%2F%2F%20rel%20(6195356)%3Bout%20geom%3B"
}
},
"layers": [
{
"id": "overpass-point",
"type": "circle",
"metadata": {"IHM:overlay": true},
"source": "overpass-points",
"paint": {
"circle-radius": 10,
"circle-color": "#f22",
"circle-stroke-width": 2,
"circle-opacity": 0.3,
"circle-stroke-color": "#d0f",
"circle-stroke-opacity": 0.7
}
}
]
}
I've added a proxy for the unshorten API at https://israelhiking.osm.org.il/unshorten
For example, the body of the response to https://israelhiking.osm.org.il/unshorten/overpass-turbo.eu/s/1nyV is
https://overpass-turbo.eu/?Q=nw%0A%20%20%5Bamenity%3D%22drinking_water%22%5D%0A%20%20(area%3A3606195356)%3B%0Aout%20meta%20geom%3B%0A%0A%2F%2F%20rel%20(6195356)%3Bout%20geom%3B&C=31.40054;35.04639;7&R
Some Overpass-Turbo query links are LZW-compressed, as noted by a ?q=
parameter rather than the ?Q=
parameter used for plain text queries.
For example, https://overpass-turbo.eu/s/1ocA is a link to same "drinking water" query above, but it is compressed. The body of the response to https://israelhiking.osm.org.il/unshorten/overpass-turbo.eu/s/1ocA is
https://overpass-turbo.eu/?q=bncKICBbYW1lbml0eT0iZHJpbmvEkWdfd2F0ZXIiXcSCIChhcmVhOjM2MDYxOTUzNTYpOwpvdXQgxId0YSBnZW9txLEKLy8gxKJsxJ_EqcSrxK3ErzvEs8S1xLvEvTs&c=BYAznW9WWH&R
Luckily, the de-compression code is written in JavaScript and it is available in the Overpass-Turbo repository:
@HarelM How I add label to the point? I want to add data from tag of the node.
Using the style.json. overpass is just the underlying data source.
I tried to add a layer but I must have made a mistake. I'm not a geojson expert.
https://gist.github.com/amiad/e4e7187e906a716ec279fdd0bb28b051
@amiad,
Maputnik is your friend.
Also, out geom;
would be sufficient.
תודה, אחרי העזרה ממך ומהראל הצלחתי להגדיר הכול כמו שרציתי.
Great!
Would you be interested in publishing your work
כן, רק אני לא בטוח מה העניין לציבור. המטרה האישית שלי היא להציג את סוג העץ גם במקרה שהוא לא קיים ב־name (או שמשתמש חרוץ מחק אותו משם)
דוגמא לשימוש ביכולת, אולי מישהו אחר ירצה דוגמה שעובדת...
Feature
The IHM custom layer allows everyone to define a layer to be added to IHM. However, the process is technically challenging:
Adding a protocol that receives the data from Overpass-Turbo would greatly simplify the creation of data for a custom layer. The Overpass-Turbo wizard can further simplify the process. As a result, the layer data will also be kept up-to-date w.r.t. the underlying OSM data.
To address the complexity of defining a style, the protocol would have a default style. Potentially, this style will be similar to the Overpass-Turbo style.
It would be even better if the protocol would accept simple overrides to the style, such as a URL to an icon form point features or a color for line features.
Example Overpass-Turbo query:
I could not think of a layer of line features that would be of interest to hikers or bikers. If that's so, it would be fine for the protocol to show icons only - for point and area features.
Things I think the developers should know, images, links etc.
The code that translates Overpass API results into geojson is maintained by the Overpass-Turbo team, and it is available on Github and as an npm package osmtogeojson
For example, the above drinking water query can be defined as a layer using this syntax:
A possible extension could be adding formatting parameters, such as size and icon, as optional URL parameters.