serve static file

[Qinka]

If it is possible that pingora can serve static file, just like nginx?

[yonas]

Make sure to use sendfile for performance.

[hikaricai]

I think pingora is a framework, we can wait for river

[smsilva98]

This is the feature I'd like to see most.

[mustafasegf]

I have figureout to to serve static file. Here's a small example that I use for my personal reverse proxy https://github.com/mustafasegf/ora-ping/blob/e98601f661b26d58d215c84808702a1e10828f4d/src/main.rs I could make a pr for docs on how to serve static file

[dgagn]

I have figureout to to serve static file. Here's a small example that I use for my personal reverse proxy https://github.com/mustafasegf/ora-ping/blob/e98601f661b26d58d215c84808702a1e10828f4d/src/main.rs I could make a pr for docs on how to serve static file

Serving static files is so much more than sending the response with the static file. For instance, nginx generates etags and uses sendfile to send the file faster. I would like to see something similar in Pingora. In the code, there's a MemCache, but it is marked as For testing only, not for production use which shouldn't be used. We can use something like redis or such, but I would like to see something similar to nginx in terms of caching mechanism. There's something promising in the pingora-memory-caching package, but doesn't implement Storage. Documentation would be key I think.

[smsilva98]

I have figureout to to serve static file. Here's a small example that I use for my personal reverse proxy https://github.com/mustafasegf/ora-ping/blob/e98601f661b26d58d215c84808702a1e10828f4d/src/main.rs I could make a pr for docs on how to serve static file

You have the content-type hard-coded to text/html, I'd like for Pingora to dynamically set the content-type header to its proper value based on the file it's reading and also set the content-encoding header for compressed files.

[mustafasegf]

yes, having direct static file suport with sendfile/splice would be great. but i believe we need ktls before that

[bestgopher]

expect river

[palant]

Reading through the river announcement, it doesn’t sound like this is one of their goals. The focus does not appear to be adding missing features to pingora but rather allowing WASM apps to run. Writing a WASM application to serve static files isn’t exactly easier than writing it in Rust.

As has been noted before, supporting static files is far from trivial. That’s why I tried making ServeDir from tower-http work, it already has most of the necessary functionality implemented. While it sort of works, two issues make this not really ready for production:

1. Tower services expect &mut self. While this mutability is unnecessary for ServeDir, it makes things quite a big harder given that pingora doesn’t seem to have a mutable global state and we probably don’t want to have a copy of the service for each request. Solving this IMHO requires messing with RwLock which likely degrades performance unnecessarily.
2. Producing responses in request_filter is currently suboptimal because it will skip all other processing. For example, compression will not be applied (you can still use pre-compressed files with ServeDir), and logging phase will be skipped.

At this point the best solution seems to be having a different server do the static file handling and proxy its response in pingora. That’s unless pingora-proxy changes “upstream” to be a more flexible concept that includes custom handling (e.g. within the same application).

[jellybobbin]

This is the feature I'd like to see most.

[palant]

I’ve implemented this functionality in a crate that can be used as a request filter: https://crates.io/crates/static-files-module

There is a complete example, a web server based on pingora-proxy and serving static content from a directory. The default command line options and config file are extended to include configuration for the static-files-module crate. That’s currently more complicated than it should be, hopefully the relevant Pingora issues will be fixed soon. I intend to add a more elaborate example (virtual hosts with separate configurations) soon.

This should support all the relevant functionality: GET/HEAD requests, conditional requests, byte range requests. Compression works either via pre-compressed files or Pingora’s downstream compression. A custom 404 error page can be configured.

One notable exception is sendfile – my understanding is that this currently isn’t possible within the Pingora framework, the relevant issue being #160.

[eaufavor]

I’ve implemented this functionality in a crate that can be used as a request filter: https://crates.io/crates/static-files-module

This is great work! Part of the reason we are not rushing to provide static file support is because we believe with the APIs we provide, the community should be able to make it happen all by yourselves.

Regarding sendfile, because of the wide adoption of of https (user space encryption) and compression, sendfile is not practically used in many real life workloads. This is also we are not too concerned about the lack of support of it.