Overriding server handlers with efficient Raw implementations

[fizruk]

At work I faced the need to abandon nice Servant API types in favour of Raw for some critical endpoints to enhance handler performance by avoiding "unnecessary" JSON (de)serialisation. However I also wanted to retain Haskell client functions and Swagger documentation, which I lose if I go with just Raw.

I think I've come up with a solution with is lightweight and allows me to override any endpoint whenever there's a need to and without sacrificing client functions or Swagger documentation:

-- | A value that can be overriden with a value of a different type.
data OverridableAs raw api
  = Overriding  raw
  | Overridable api

type SampleAPI
    = "send" :> OverridableAs Raw SendItem
      -- ^ we want to be able to provide a more efficient
      -- Raw implementation for this endpoint
      -- specifically to avoid unnecessary ToJSON/FromJSON
      -- conversions and validations
      -- but we want client functions and Swagger documentation
      -- to still be derived from SendItem API
 :<|> "list" :> ListItems

The full gist is available at https://gist.github.com/fizruk/59c54f849941306b1bd50dd276debb64

I believe this is a usable, but still raw idea that can probably be improved upon:

• In route I am relying on RawRouter and serveWithContext. I think it can be done better, but should introduce little-to-no overhead for Raw (which is the case I'm most interested in).
• There might be other uses for this. For instance OverridableAs NotImplemented where NotImplemented has some trivial implementation.
• OverridableAs is essentially Either which is just a sum type operator in disguise. A natural theoretical question to ask is does ever makes sense to "sum" more than two values? How about zero?

@phadej @alpmestan I would appreciate your comments on this one!

[fizruk]

I have updated the gist to make it self-contained (with nix-shell shebang) and added Swagger UI for demonstration.

[alpmestan]

Interesting. Before discussing your actual idea/suggestion, would you mind answering the following question: while I understand how bypassing the JSON encoding/decoding part can help, can you get your performance improvements (or a fraction of them) by taking a ReqBody '[OctetStream] LBS.ByteString for example? IIRC "decoding" this is almost a no-op (depending on how far GHC will unroll the content type stuffs).

[fizruk]

@alpmestan I would say yes, but I can't run benchmarks now.

One problem with OctetStream is that it has a different Content-Type and I have to make my own version (which is fine). Another is that I perform content type check at all. For some endpoints I trust clients with content they send and can also skip content type check. And lastly I would still use OverridableAs SendItemOctetStream SendItemJSON to have nice Swagger UI and clients based on SendItemJSON.

[phadej]

I understand the problem. I do not like that solution clutters type API (exposes implementation details). I'd like a solution leaving in servant-server alone, so other interpretations don't need to care about it.

Could some kind of zipping of routers be possible, so we can "take left" or "take right" implementation, where left could be Maybe Raw, and right would be the default implementation.

[fizruk]

I do not like that solution clutters type API (exposes implementation details).

I agree. Although clutter is minimal I think.

Also, it might have pros too. In theory you might want to override clients too and mention implementation detail in the documentation (e.g. that this endpoint is less safe or does not respond correctly to errors in request since it does not perform some checks).

I'd like a solution leaving in servant-server alone, so other interpretations don't need to care about it.

I just came up with another solution, using Replace type family:

-- | Replace a @old@ sub-API with @new@ everywhere in @api@.
type family Replace old new api where
  Replace old new old = new
  Replace old new (param :> api) = param :> Replace old new api
  Replace old new (left :<|> right)
    = Replace old new left :<|> Replace old new right
  Replace old new api = api

-- | Like 'SampleAPI', but with 'Raw' for more efficient 'SendItem' implementation.
type EfficientSampleAPI
  = Replace SendItem Raw SampleAPI

-- | Like 'sampleServer', but with 'efficientSendItem'.
efficientServer :: Server EfficientSampleAPI
efficientServer
    = efficientSendItem
 :<|> serveListItems

main :: IO ()
main = do
  -- we have to replace SampleAPI with EfficientSampleAPI
  Warp.run 8080 $ serve (Proxy @(Replace SampleAPI EfficientSampleAPI API)) $
    swaggerSchemaUIServer sampleSwagger :<|> efficientServer

The full gist is at https://gist.github.com/fizruk/442d93cd8c324b366919630bc4e02771

[fizruk]

A solution with Replace has a couple of minor downsides I see so far:

• For nested servers you'll have to run Replace SendItem Raw at every level;
• A few "replaced" Raw endpoints can be easy to mess up, would be nice to have Tagged api Raw;
• Replace type family can potentially result in unclear type error messages.