X-Sendfile / X-Accel Support

[oneiros]

It would be super nice if the file-Method in actions supported sending X-SENDFILE (Apache) and X-Accel-* (nginx) HTTP headers.

Background: Instead of delivering the file through the app, front-end webservers like Apache and nginx support the aforementioned HTTP headers. The app does not actually send the whole file, but rather uses a header to tell the front-end webserver which file to send out. The front-end webserver then does what it is really good at, deliver static files, while the application process is free to process the next request.

I am currently developing an app that will need to send out video files. I have a hunch that it might benefit a lot from this feature. But it will be some time before I can really look at this kind of optimization.

When I come around to it, I am happy to give it a try myself. Until then I thought I would leave this here, so I will not forget. (Of course, if someone wants to tackle this before I get around to it, I would not mind that at all :grin: )

[jwoertink]

I'm not too familiar with this, but would you just do something like..

class SomeAction < Lucky::Action
  before set_nginx_headers

  get "/some-file.txt" do
    file "some-file.txt", disposition: "attachment", content_type: "plain/text"
  end

  private def set_nginx_headers
    context.response.headers["X-Accel-Redirect"] = "/whatever"
  end
end

Or would something else need to go in to this? Since this is specific to your webserver, how would we know which one to add in? Like if you use Caddy?

[oneiros]

Or would something else need to go in to this?

Yes. In this case, file should not actually send the file, just the header. The webserver will then take care of delivering the file to the user agent, freeing the crystal process to attend to other requests.

Since this is specific to your webserver, how would we know which one to add in? Like if you use Caddy?

Caddy seems to support the X-Accel-* variant of this, if I understand https://caddyserver.com/docs/internal correctly.

[jwoertink]

Ah interesting. So really it'd be more like:

class SomeAction < Lucky::Action
  before set_caddy_headers

  get "/some-file.txt" do
    text "OK"
  end

  private def set_caddy_headers
    context.response.headers["X-Accel-Redirect"] = "/whatever"
    #.. add a few other X-Accel-* headers
  end
end

But the main thing I'm trying to understand here is what Lucky would actually add internally for this. Like, is this something we add in configurations for, or would this work as a separate shard? Or do we just leave it up to the individual developers to add the code (provided my sample is sufficient)?

If you also have an idea of what this implementation needs to look like, I'd be curious to see. If it's as simple as what I think, maybe we just add some documentation to the site to cover that instead of some configuration stuff... Let me know your thoughts.

[oneiros]

But the main thing I'm trying to understand here is what Lucky would actually add internally for this. Like, is this something we add in configurations for, or would this work as a separate shard? Or do we just leave it up to the individual developers to add the code (provided my sample is sufficient)?

I think this should be a per-environment configuration just like in rails, because in development and test environments you usually do not have a front-end webserver. So sending out a header instead of a file only ever really makes sense in production envs.

Edit: And of course even in production this should be totally optional, default off.