ajnisbet / opentopodata

Open alternative to the Google Elevation API!
https://www.opentopodata.org
MIT License
312 stars 69 forks source link

Feature request: adding POST support for query many locations. #49

Closed iisri-vu closed 2 years ago

iisri-vu commented 3 years ago

The GET request has a limitation with uri length. We need a POST to increase this limitation.

ajnisbet commented 3 years ago

Thanks for raising this, I've held off because it doesn't align with http semantics, but I could see it being useful. I'll have a look at what other APIs do.

I have some questions that would help me prioritise this issue:

In the meantime there's a couple of things you can do:

iisri-vu commented 3 years ago

Thanks Andrew,

I'm currently running my Flask server with blue print tech for GET and POST to stream road information.

I'm trying to get elevation of roads geometry, which may need a fair number of points.

I would like to host your API on our server to serve large number of locations query.

What you could do is have option on the POST query to limit the number of points. If we could set this limit value on our own build. This way it won't change your previous implementation or overload your server.

I have used open elevation. But it takes 20GB of rams for just one map and runs very slow on my high end laptop, running the docker service.

I'm currently evaluate your API. I saw it uses smaller map junks, which is great for tree search optimization and memory.

I'm currently downloading the 90m files. This seems to take for ever.

iisri-vu commented 2 years ago

This is a minimal example of POST, without changing to the existing code. Sorry I did not check for the exact error in the code.

the method accept: json: {"dataset_name":"srtm90m", "locations": latlngs, "interpolation":"cubic"} latlngs = [[lat, lon], [lat, lon], [lat,lon],...]

By the way, I think returning a dataset name in the response is a waste of bandwidth. Maybe we can remove that. Thanks

@app.route("/v1/elevation",  methods=["POST"])
def post_elevation():
    """Calculate the elevation for the given locations.
    Returns:
        Response.
    """
    try:
        if(request.method == 'POST'):
            obj = request.get_json()
            dataset_name = obj['dataset_name']
            latlons = obj['locations']
            if (obj['interpolation'] == None):
                interpolation = DEFAULT_INTERPOLATION_METHOD 
            else: 
                interpolation = obj['interpolation']

            if(len(latlons) > _load_config()["max_locations_per_request"]):
                return (
                    jsonify({"status": "SERVER_ERROR", "error": "Config Error: {}".format("too many location specified")}),
                    400,)

            #Convert list of (lat, lon) tuples to lat and lon lists.
            lats = [p[0] for p in latlons]
            lons = [p[1] for p in latlons]
            nodata_value = None

            # Get the z values.
            datasets = _get_datasets(dataset_name)
            elevations, dataset_names = backend.get_elevation(
                lats, lons, datasets, interpolation, nodata_value
            )

            # Build response.
            results = []
            for z, dataset_name, lat, lon in zip(elevations, dataset_names, lats, lons):
                results.append(
                    {
                        "elevation": z,
                        "dataset": dataset_name,
                        "location": {"lat": lat, "lng": lon},
                    }
                )

            data = {"status": "OK", "results": results}
            return jsonify(data)

    except (ClientError, backend.InputError) as e:
        return jsonify({"status": "INVALID_POST", "error": str(e)}), 400
    except config.ConfigError as e:
        return (
            jsonify({"status": "SERVER_ERROR", "error": "Config Error: {}".format(e)}),
            500,
        )
    except Exception as e:
        if app.debug:
            raise e
        app.logger.error(e)
        msg = "Unhandled server error, see server logs for details."
        return jsonify({"status": "SERVER_ERROR", "error": msg}), 500
ajnisbet commented 2 years ago

Added in https://github.com/ajnisbet/opentopodata/commit/920f110de06d616695eb969e4d1169d6b21a632e