Unexpected outcome when using QL query and `build=False`

[pe-perry]

It is expected the following codes should produce the same output, but it turns out they didn't.

Code 1

import overpass
api = overpass.API()
result = api.get('[out:json];rel(3676782);out;', build=False)
print(result)

Output:

{"features": [], "type": "FeatureCollection"}

Code 2

import requests
resp = requests.get(
    'http://overpass-api.de/api/interpreter', 
    params={'data': '[out:json];rel(3676782);out;'})
print(resp.json())

Output:

{'version': 0.6, 'generator': 'Overpass API 0.7.54.13 ff15392f', 'osm3s': {'timestamp_osm_base': '2018-04-10T07:39:02Z', 'copyright': 'The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.'}, 'elements': [{'type': 'relation', 'id': 3676782, 'members': [{'type': 'way', 'ref': 45161703, 'role': 'outer'}, {'type': 'way', 'ref': 9768812, 'role': 'outer'}, {'type': 'way', 'ref': 9770315, 'role': 'outer'}, {'type': 'way', 'ref': 45161682, 'role': 'outer'}, {'type': 'way', 'ref': 10041616, 'role': 'outer'}, {'type': 'way', 'ref': 9767801, 'role': 'outer'}, {'type': 'way', 'ref': 262802435, 'role': 'outer'}, {'type': 'way', 'ref': 9560555, 'role': 'outer'}, {'type': 'way', 'ref': 246658002, 'role': 'outer'}, {'type': 'way', 'ref': 9562448, 'role': 'outer'}, {'type': 'way', 'ref': 9560531, 'role': 'outer'}, {'type': 'way', 'ref': 9763360, 'role': 'outer'}, {'type': 'way', 'ref': 9761979, 'role': 'outer'}, {'type': 'way', 'ref': 9768478, 'role': 'outer'}, {'type': 'way', 'ref': 9770660, 'role': 'outer'}, {'type': 'way', 'ref': 271093834, 'role': 'outer'}], 'tags': {'name': '大嶼山 Lantau Island', 'name:en': 'Lantau Island', 'name:zh': '大嶼山', 'place': 'island', 'type': 'multipolygon', 'wikidata': 'Q502379', 'wikipedia': 'en:Lantau Island'}}]}

The reason should be the default value of responseformat (geojson). Although my query said I want a json, the .get() method still convert it to geojson (default responseformat). To obtain the desired output, I have to add responseformat='json'.

api.get('[out:json];rel(3676782);out;', build=False, responseformat='json')

This looks redundant. Not sure if it is what you desired.

[mvexel]

That is confusing, you're right.

Will address in next release unless you beat me to it with a PR ✊

[mvexel]

Would you propose to detect the format from the raw query? In that case we'd need to assume that the query always starts with [out:...] -- or employ a regex to detect the [out:...] statement. What do you think?

[pe-perry]

I prefer the former option, users are required to write a complete and valid overpass QL query if they set build=False. What do you think?

[mvexel]

Makes sense. We would then ignore outputformat altogether (or even throw an exception when it is used in combination with build=False) and just return the raw response body -- whatever that is. This basically would bypass any of the logic we use to parse the output. Sounds good?

[pe-perry]

It's one option, or should we keep the content_type checking part and return appropriate Python object?

if build is False,

• content_type == text/csv: return str or list of lists (like what you did)
• content_type == "text/xml" or content_type == "application/xml": return str or tree object (in xml, lxml etc)
• content_type == "application/json": return dict

[pe-perry]

I have pushed a branch, see if it is fine. :)

[pe-perry]

It looks like the bug re-appear again.

q = '[out:json];rel(336);out;'
data1 = api.get(q, build=False)
data2 = api._get_from_overpass(q).json()

Output:

# >>> print(data1)
# {
#     "features": [], 
#     "type": "FeatureCollection"
# }
# >>> print(data2)
# {
#     'generator': 'Overpass API 0.7.55.5 2ca3f387', 
#     'version': 0.6, 
#     'osm3s': {
#         'copyright': 'The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.', 
#         'timestamp_osm_base': '2018-12-22T13:51:02Z'
#     }, 
#     'elements': [
#         {
#             'type': 'relation', 
#             'members': [
#                 {
#                     'ref': 43759879, 
#                     'type': 'way', 
#                     'role': 'inner'
#                 }, 
#                 {
#                     'ref': 30663172, 
#                     'type': 'way', 
#                     'role': 'outer'
#                 }
#             ], 
#             'id': 336, 
#             'tags': {
#                 'type': 'multipolygon', 
#                 'water': 'lake', 
#                 'name': 'Trollsvannet', 
#                 'natural': 'water'
#             }
#         }
#     ]
# }