Dynamic routing

[3noch]

Is there a way (currently) to build a dynamic route at runtime out of static routes? I'm imagining a simple use case where you want to serve the same API with different URL roots, say /production/api/... and /test/api. You could parameterize your server on top-level path and server the API from there. But I can't think of a way to dynamically build a route.

[fizruk]

Well, I guess you can alway add a middleware for that... but I don't quite understand what you're trying to achieve. Could you elaborate a bit more?

[3noch]

Let's say I have some server to run a calculation (based on data) and return an estimate. I can serve it at:

<host>/analysis-calculator

Now my client services can use that URL to get important data.

But let's say I want to simultaneously serve the exact same server, but with some tweaks. However, I still want my clients to have access to it (maybe a beta pool). I might serve it here:

<host>/beta/analysis-calculator

And maybe I have another (less stable) version of that served at

<host>/test/analysis-calculator

Now, of course, I could serve all three of these from the same server executable. But that would require that each deployment to any of these endpoints requires a deployment to all of them. It'd be much nicer if I could configure my deployment to add and remove servers at will (behind some reverse proxy, of course). In that case I might take the unstable test server and configure my deployment to start it like this:

server --port 8080 --root test

The beta server would be deployed as

server --port 8081 --root beta

Etc.

But since both "test" and "root" are not known at compile time, I have no way of creating a route in Servant that uses them. At least not that I know of.

Could SomeSymbol be of help somehow? I.e. a dependently typed route.

[alpmestan]

Maybe something like the reflection package could make this somewhat simple?

[3noch]

@alpmestan I've never used the reflection package. Any ideas how it would work to solve this problem?

[christian-marie]

@3noch is there something preventing you from simply adding an extra capture, then handling the dynamic part as a string? I don't see why this needs to be dynamic, perhaps you can reframe it?

[christian-marie]

Although if just want to know if it's possible, that's an interesting question by itself.

[alpmestan]

@3noch Sorry it took me so long, but I finally got around to giving a reflection based approach a shot. I'll give a very lightweight overview here but you can find the code & the example in a nice little cabal project at https://github.com/alpmestan/servant-prefix. Feel free to ask for pushing rights if you want to improve it and release it to hackage, in case you think this could be useful to others.

Alright, so if we are to take a reflection based path, then our approach should rely on reifySymbol :: forall r. String -> (forall n. KnownSymbol n => Proxy n -> r) -> r. This allows us to turn a good old String value into a type "representing it", behind a Proxy. However, we see that n can't be returned in any way, given that it's only ever in scope during the lifetime of the second argument. Whatever we need to do with that n, we have to do it in the body of that second argument.

A piece that we'll need is a function that takes a Proxy api, a Proxy str (representing our String at the typevel) and that returns Proxy (str :> api), i.e an API equivalent to the one we gave, but that's exposed under some static string prefix.

prefix :: forall s api.
          KnownSymbol s -- not necessary, but makes sure we only get type-level strings
       => Proxy s
       -> Proxy api
       -> Proxy (s :> api)
prefix Proxy Proxy = Proxy

Yes, this is a "stupid" function, it does nothing interesting at the value-level, but notice how its sole purpose is to give us an API with the string prefix. We want the result of this function to be fed to serve, which generates the router, so that the resulting server will only answer queries under the given string prefix.

Now, we need to wrap reifySymbol in a function so that we give it 1/ a String prefix, 2/ a Proxy api and that then allows us to work with a Proxy (s :> api) in some "delimited scope". The following will do.

prefixing :: String
          -> Proxy api
          -> (forall s. KnownSymbol s => Proxy (s :> api) -> r)
          -> r
prefixing s api f = reifySymbol s $ \sProxy -> f (prefix sProxy api)

OK, this is all nice and interesting, but does it actually work? How can we use it? Here's an example.

type API = "foo" :> Get '[JSON] Int

api :: Proxy API
api = Proxy

server :: Server API
server = return 10

main :: IO ()
main = prefixing "testing" api $ \apiProxy -> run 8080 (serve apiProxy server)

Let's check that it does what we want:

$ curl http://localhost:8080/foo -v
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /foo HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.50.1
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Transfer-Encoding: chunked
< Date: Sun, 08 Jan 2017 11:41:48 GMT
< Server: Warp/3.2.8
< 
* Connection #0 to host localhost left intact
$ curl http://localhost:8080/testing/foo -v
> GET /testing/foo HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.50.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Date: Sun, 08 Jan 2017 11:41:56 GMT
< Server: Warp/3.2.8
< Content-Type: application/json
< 
* Connection #0 to host localhost left intact
10

Full code at https://github.com/alpmestan/servant-prefix/blob/master/servant-prefix.hs

[alpmestan]

I've actually realized we can write these two very helpful functions:

serveUnder :: HasServer api '[] => String -> Proxy api -> Server api -> Application
serveUnder prefix api server = prefixing prefix api $ \apiProxy ->
  serve apiProxy server

serveUnderWithContext :: HasServer api ctx
                      => String
                      -> Proxy api
                      -> Context ctx
                      -> Server api
                      -> Application
serveUnderWithContext prefix api ctx server =
  prefixing prefix api $ \apiProxy -> serveWithContext apiProxy ctx server

The former considerably simplifies our example's main:

main :: IO ()
main = run 8080 (serveUnder "testing" api server)

I've pused them to the repo.

[3noch]

@alpmestan This is exactly what I was hoping for. It looks like using the type-safe URL creators would just need to use prefixing to generate URLs in the same "sub-API".

@christian-marie Yes, I think the idea is more what I'm after. You're right that my specific example could easily have other (even better) solutions than using reflection.

[alpmestan]

Yeah, you really want docs/clients/links/etc to use the "prefixed proxy" (handed to us in the body of prefixing's third argument) otherwise your queries won't reach their target endpoints anymore.

[3noch]

@alpmestan Thank you so much for this work. Is this something that deserves to be in the servant core libraries? I have little preference either way, but I'd love for your work to be highly "discoverable" since most people coming from other API systems will likely be surprised to find that dynamic route creation is not "built-in" to Servant.

[alpmestan]

Hmm, it could indeed be added to the core libraries I guess, basically adding variants to serve, client, etc. If anyone wants to look into that... :)

[aravindgopall]

Are there any updates on this. @alpmestan your changes are throwing a few errors and not able to use it. Can you help

[alpmestan]

Can you show us the errors?

[aravindgopall]

Please find the logs:

• Couldn't match type ‘k’ with ‘*’ ‘k’ is a rigid type variable bound by the type signature for: serveUnder :: forall k (api :: k). HasServer api '[] => String -> Proxy api -> Server api -> Application at src/Newton/Utils/Routes.hs:(33,1)-(34,74) Expected type: Proxy api0 Actual type: Proxy api • In the second argument of ‘prefixing’, namely ‘api’ In the expression: prefixing prefix api In the expression: prefixing prefix api $ \ apiProxy -> serve apiProxy server • Relevant bindings include server :: Server api (bound at src/Newton/Utils/Routes.hs:35:23) api :: Proxy api (bound at src/Newton/Utils/Routes.hs:35:19) serveUnder :: String -> Proxy api -> Server api -> Application (bound at src/Newton/Utils/Routes.hs:35:1) | 36 | prefixing prefix api $ \apiProxy -> serve apiProxy server

• Couldn't match type ‘ServerT api Handler’ with ‘ServerT api0 Handler’ Expected type: Server (s :> api0) Actual type: Server api NB: ‘ServerT’ is a non-injective type family The type variable ‘api0’ is ambiguous • In the second argument of ‘serve’, namely ‘server’ In the expression: serve apiProxy server In the second argument of ‘($)’, namely ‘\ apiProxy -> serve apiProxy server’ • Relevant bindings include apiProxy :: Proxy (s :> api0) (bound at src/Newton/Utils/Routes.hs:36:27) server :: Server api (bound at src/Newton/Utils/Routes.hs:35:23) api :: Proxy api (bound at src/Newton/Utils/Routes.hs:35:19) serveUnder :: String -> Proxy api -> Server api -> Application (bound at src/Newton/Utils/Routes.hs:35:1) | 36 | prefixing prefix api $ \apiProxy -> serve apiProxy server

[aravindgopall]

@alpmestan any update?

[alpmestan]

Can we see the contents of the entire module too? It might be because of some changes in GHC since the version that I used to write that.

Also, I'm afraid it is not so reasonable to expect answers in the hours following your last message:

• all the servant contributors participate to the issue tracker and development for free
• it's the middle of the (work) day
• you didn't bother formatting the error messages correctly (using code blocks) nor asking politely

I would nonetheless be glad to help, once I have the code. When I get a chance. :-)