Allow policies to edit the nginx config

[mikz]

Some policies (like the mutual TLS auth until https://github.com/3scale/apicast/issues/440 is solved) need a way to edit the nginx configuration. Part of https://github.com/3scale/apicast/issues/427

There are several options how to do this.

Options

1. Pattern matching and text transformation

Like https://github.com/spree/deface can edit a template with set of filters we could add stuff before / after other directives.

2. Parsing nginx config

Using nginx internals ngx_conf_parse could parse the whole config, edit it and print back. This is pretty FFI heavy and really hard approach.

3. Predefined extension points

We could offer few extension points that policies can use. This is quite limiting but so far we have just one policy that needs this. We could try to automate this by getting list of all directives from the nginx repository: https://trac.nginx.org/nginx/browser/nginx_org/xml/en/docs/http/ngx_http_ssl_module.xml#L369

[nmasse-itix]

Hi @mikz !

This custom extension also needs to modify the nginx config file: https://github.com/nmasse-itix/apicast-dynamic-router

[mikz]

Hi @nmasse-itix !

Thanks for reminder. What directives it needs from nginx?

I see it has a catalog as new server and dynamic router.

The catalog could be implemented in plain lua no? Or does it need to be exposed externally? Or is it just for consumption by the dynamic router?

And the router defines own upstram and also some locations. If this is meant to replace how APIcast routes requests the upstream is not necessary and can use the one defined by APIcast. And rest of the code in those locations can be easily done in Lua.

I'm not saying it is well documented how to do so, but those things should be doable. Adding nginx configuration should be necessary only for cases that have no Lua counterpart - like setting the TLS client certificate or defining new server that starts on different port.

I'm happy to help and discuss how to convert the dynamic router to pure Lua policy.

[nmasse-itix]

Hi @mikz,

In this module, I need to call an external HTTP server (the catalog server). Based on the answer, it determines the private base URL and Apicast routes the request as usual.

I used for that an upstream nginx directive as it is the mechanism we also use to communicate between the apicast and the 3scale backend.

I found the ngx.location.capture API in LUA but it seems limited to internal nginx subrequests. How would you call an external HTTP server from LUA ? (before the balancing phase)

Regarding the catalog server, it is meant to be replaced by the customer by an external implementation so it can stay as an nginx sample configuration.

[mikz]

APIcast embeds an HTTP client wrapper that supports several libraries. The wrapper is called resty.http_ng. By default it is going to use https://github.com/pintsized/lua-resty-http library that implements HTTP client using openresty ngx.socket.tcp.

You can use it pretty easily:

local http_ng = require 'resty.http_ng'
local http = http_ng.new()
local res = http.get('http://example.com')

There are several uses of this within APIcast. You can search for http_ng. The backend option passed to .new() controls which library to use and is very helpful when testing, because you can inject test backend and test outgoing calls.

That is the recommended way of making http calls from APIcast.

One of the good examples is https://github.com/3scale/apicast/blob/632d0f38a436b95c864be0c901713ff0cd3dbc5c/gateway/src/apicast/backend_client.lua which is using this wrapper to connect to the 3scale backend (but actually using the ngx backend which uses upstream). Another example is https://github.com/3scale/apicast/blob/632d0f38a436b95c864be0c901713ff0cd3dbc5c/gateway/src/apicast/configuration_loader/remote_v2.lua which downloads the configuration from the API manager and uses the default library.

[nmasse-itix]

Ho great ! I didn't know... :)

Now I can simplify a lot my code !