Dropping POST support in 5.0

[TheMarex]

URL based POST encoding doesn't work with the new API anymore, since the coordinates are part of the path now, e.g. http://127.0.0.1/route/v1/driving/lon,lat;lon,lat;lon,lat?option=value. As such there is little sense to remove the (short) option part to POST.

Since we can't support two APIs that means POST is out for osrm-routed. For big requests we have been advocating using the C++ library directly or using node-osrm since some time now. As such this is conscious removal of a feature.

• [x] Remove POST tests
• [x] Remove code that handles POST

[daniel-j-h]

I can take a look at that.

Also for the record: here is an example project integrating libosrm with a battle-tested HTTP server (e.g. providing KeepAlive amongst other features). That should get you started:

https://github.com/daniel-j-h/libosrm-http-casablanca

I wanted to do the same with Facebook's Proxygen but haven't had the time so far.

[kostadin24]

What about distance matrix? Without POST it is not possible to solve matrix with 5000 elements. URL become too long.

[daniel-j-h]

You can either split your request and re-assemble the responses (I think I saw a project that does that transparently), or use the node-osrm bindings which we recommend for production use anyway, or use libosrm for table queries without the HTTP overhead.

---

Here's a quick brain dump of how to use libosrm in the upcoming v5 release (currently in branch rewrite/new-api). I will add this and more details to the Wiki at some point in time.

Take a look at the example code (that lives in example/example.cc). Here is all you ever wanted to know about libosrm, that is a short description of what the types do and where to find documentation on it:

• EngineConfig - for initializing an OSRM instance we can configure certain properties and constraints. E.g. the storage config is the base path such as france.osm.osrm from which we derive and load france.osm.osrm.* auxiliary files. This also lets you set constraints such as the maximum number of locations allowed for specific services.
• OSRM - this is the main Routing Machine type with functions such as Route and Table. You initialize it with a EngineConfig. It does all the heavy lifting for you. Each function takes its own parameters, e.g. the Route function takes RouteParameters, and a out-reference to a JSON result that gets filled. The return value is a Status, indicating error or success.
• Status - this is a type wrapping Error or Ok for indicating error or success, respectively.
• TableParameters - this is an example of parameter types the Routing Machine functions expect. In this case Table expects TableParameters. You can see it wrapping two vectors, sources and destinations --- these are indices into your coordinates for the table service to construct a matrix from (empty sources or destinations means: use all of them). If you ask yourself where coordinates come from, you can see TableParameters inheriting from BaseParameters.
• BaseParameter - this most importantly holds coordinates (and a few other optional properties that you don't need for basic usage); the specific parameter types inherit from BaseParameters to get these member attributes. That means your TableParameters type has coordinates, sources and destination member attributes (and a few other that we ignore for now).
• Coordinate - this is a wrapper around a (longitude, latitude) pair. We really don't care about (lon,lat) vs (lat, lon) but we don't want you to accidentally mix them up, so both latitude and longitude are strictly typed wrappers around integers (fixed notation such as 13423240) and floating points (floating notation such as 13.42324).
• Parameters for other services - here are all other *Parameters you need for other Routing Machine services.
• JSON - this is a sum type resembling JSON. The Routing Machine service functions take a out-ref to a JSON result and fill it accordingly. It is currently implemented using mapbox/variant which is similar to Boost.Variant (Boost documentation is great). There are two ways to work with this sum type: either provide a visitor that acts on each type on visitation or use the get function in case you're sure about the structure. The JSON structure is written down in the v5 spec.

To summarize: 1/ create an OSRM instance initialized with a EngineConfig 2/ call the service function on the OSRM object providing *Parameters 3/ check the return code and use the JSON result.

---

Here you can see a Route query, by first creating RouteParameters and then calling the Route function which fills a result JSON object from the example server integration I posted above:

https://github.com/daniel-j-h/libosrm-http-casablanca/blob/9e72296dd23035fe90c66294fb2bf1a689c62047/main.cc#L85-L96

[danpat]

@kostadin24 Note that the URL length limits are entirely client-side - osrm-routed will accept any length URL you feed it. The only limits are client-side.

If you're requesting from a browser, then the maximum URL size will be governed by that browser (http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers).

For command-line tools like curl, the maximum size will be limited by the maximum parameter size enforced by the operating system: for Linux, it's about 130kb (http://www.in-ulm.de/~mascheck/various/argmax/#maximum_number)

If you programatically control curl, or implement your own HTTP client code, there is no effective limit.

@TheMarex @daniel-j-h We should probably put a limit on this on the OSRM demo server so that it's not a vector for a denial-of-service attack.

[grudolf]

@danpat, the problems with GET size limits start when osrm-routed is accessed through a http reverse proxy.

[kostadin24]

@grudolf Because of GET size limit - I'm using POST. Now POST support will be dropped.

@danpat I made requests from program in same machine. Browsers aren't involved here. With POST works fine with 5000x5000 matrix. But with GET after URL pass 32768 size - problem. I didn't make research. Just switched to POST and it was OK.

[daniel-j-h]

This landed in the v5 rc.

Have a look at the C++ libosrm or if you need experimental C and Python bindings head over to

https://github.com/daniel-j-h/libosrmc

and speak out now.

[kostadin24]

Hello, I wasn't arround for a little :) @daniel-j-h you wrote "This landed in the v5 rc.". Does this means that version 5.7 handles GET requests with url size bigger than 32768?

[daniel-j-h]

We just removed POST support from the osrm-routed binary, nothing else should have changed. There should also be no limit on GET request size coming from osrm-routed.

I still recommend using either libosrm directly from C++ or node-osrm via Node.js for larger matrices. Good luck!

[kostadin24]

Direct use from c++ is best option. Unfortunatelly I wouldn't have free time soon to write code for this. Will try again with big URL for GET request to fix situation for first time. My previous unsuccesfull attempt was with URL > 32768 and request was from same machine, so no browser or network involved here.

[kostadin24]

Good work! After testing v 5.7.0 with GET request/URL length 40k/ - everything is FINE :) Will replace old 4.x version with new one.

[kostadin24]

Hmm! Are you sure that in v5.7.0 POST is dropped? I just made mistake to use POST instead of GET. Request for table with 2000 points and.........it works. Here is code:

String url contains table request with 2000 points and it is about 40kb.

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
            request.Method = "POST";
            request.Timeout = 10000000;

HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            using (TextReader tr = new StreamReader(response.GetResponseStream()))
            {
                return Newtonsoft.Json.JsonConvert.DeserializeObject<TableResponse>(tr.ReadToEnd());
            }

This code works with both GET or POST.

[daniel-j-h]

Yes https://github.com/Project-OSRM/osrm-backend/pull/2182 removes POST support from the osrm-routed's request parser. Maybe your library has a GET fallback?

[kostadin24]

Probably GET falback is involved. If I move parameters from URL to request.content POST request fails. If POST have url with params and no data in request.content - fails as expected