Extend root to work like Nginx location

[guilhermebr]

I want to use root like Nginx location:

  location /static/ {
      alias /var/www/project/public/static/;
    }

So in caddy we can extend root to do this, something like:

www.domain.com {
    root /var/www/project
    root /static/ /var/www/project/public/static 
}

root {path} root {url_path} {path}

Or someone has a better solution?

[pedronasser]

Wouldn't be better to do almost like Nginx?

www.domain.com {
    /static/ {
        root /static/        
    }
}

@mholt is it possible?

[mholt]

I wondered when this would come up. In the early days of designing the Caddyfile, I had considered something called location contexts (or path scopes, whatever), which are exactly what you've proposed @pedronasser - they even had the same syntax. Basically, anything in a path scope only applied to those paths.

This turned out to be difficult to implement properly, and gets complicated quickly. Desiring to reduce overhead, I wanted to compile a separate middleware chain for each path scope so that every request wouldn't have to be switching on which path scopes matched the request and which ones didn't. Then the question of whether path scopes are inclusive or exclusive. Then the question of priority if they're inclusive - should it be user-defined by the order it appears in the file or by length of the path?

Caddy does have partial support for location scopes built in. I would like to decide positively whether or not we will implement this feature and then fill it out and finish it or delete it entirely and simplify some code. (Honestly, I prefer to keep it simple and not implement them, unless the reasons are really compelling otherwise.)

Now, the matter of variable root paths.

When the server starts, each virtualhost is jailed to the root directory using a 'fake' file system (http.Dir in fact) to prevent any possibility of exploiting the Caddy file server component to serve files outside of site root.

Given that each virtualhost has just one of these, it seems infeasible to support multiple depending on the request path.

However, if this is sorely needed, we could look into changing the way the root path is implemented. It's very important that A) the code stays simple, and B) the code stays fast. Maybe the right answer is to implement path scopes, or maybe it's by having root accept a second argument and restructuring the virtualhosts somewhat.

[pedronasser]

How about using different configurations (contexts) for different location requests. Let me try to explain what I mean with an example:

Considering the following configuration file:

www.domain.com {
    root /tmp/

    /static/ {
        search ^/ /search
    }
    /other/ {
        search ^/other/ /otherSearch
    }
}

Would result in the following process:

• Load all directives
• Load each virtual host's configuration.
• Load each location's configuration and merge with the parent virtual host configuration
• Create a fake default location for requests that not matches any configured location using only current virtual host configuration.
• For each location, start configured directives using current location's configuration
• Wait requests
  • If request URL matches any location, send to the processing stack using location's configuration.
  • Else send request to the processing stack using default location's configuration.

Configurations results:

www.domain.com/static/ {
     root /tmp/
     search ^/ /search
}

www.domain.com/other/ {
     root /tmp/
     search ^/other/ /otherSearch
}

(default) www.domain.com/* {
      root /tmp/
}

Would that process be heavy? Just throwing that idea.

[mholt]

As discussed (thoroughly) in Slack, this isn't trivial.

This is about two things:

1. Changing the site root depending on the request path
2. Enabling path scopes in the Caddyfile

Number 1 might be possible, but requires re-architecting the fileServer somehow, like creating the http.Dir for every request rather than at server startup. Another problem is that variable root paths cause problems with all the other paths that are relative to site root.

Attempting to solve that with number 2 results in a new paradigm for the virtualhosts. No longer would virtualhosts just be host names, they would be host+path combinations. It also changes the paradigm of the Caddyfile, which is feature-centric, not location-centric like nginx conf.

I feel like the path scopes (number 2) introduces more complexity than is worth it. We did seem to agree that there were very few (if any) use cases that we've seen where combining many directives under the same path scope was beneficial.

For my own benefit and understanding, can somebody tell me why a site is not in one directory - why is it split up in different folders not under the same root path?

[jpoehls]

@mholt: Here's my simple use case. I have a Django app and want to Caddy to serve it. This means:

1. Proxy anything under / to my Django app
2. Serve anything under /static directly from Caddy as static files

So for that to work I need to have different directives for different paths under the same host. I think it is a pretty necessary use case as well since, at least in the Django case, it isn't recommended to serve static files from the Django app itself.

Here is a Gist that has full example of what I'm going for. https://gist.github.com/jpoehls/a40db92ad2033ab7316f

The gist doesn't let me represent sub-directories so here is the expected tree:

# FILE TREE
#
# │ Caddyfile
# │ server.go
# │
# └───static_files/
#         styles.css

[mholt]

@jpoehls So if I understand correctly, you need to proxy everything not in /static to Django?

What if your site was structured so that the dynamic part was in /django or something, then you could just proxy that?

[jpoehls]

That'd work of course but doesn't seem practical. If the entire app is Django then the default route will be a Django view, /, and with your proposal you couldn't have it served by Caddy at all unless you redirected / to /app or something silly like that.

[jpoehls]

This seems like it may work. Probably needs tweaked to rewrite all paths under / properly.

localhost:2015 {
    startup "go run ./server.go" &
    root ./static_files
    rewrite / /server
    proxy /server localhost:2016
}

[mholt]

Hmm, would using without /server work (in the proxy directive), instead of that rewrite? Or does that do the inverse of what you intend? (Sorry, not a Django programmer.)

[mholt]

Oops, somehow I didn't see your gist above. :see_no_evil: That looks like a good example - I'll take a closer look at it.

[mholt]

@jpoehls Sorry for my confusion. I had a chance to try out your gist, now I see what you mean. Do you think a "not" command for proxy, being able to exclude certain paths (space-separated), would be able to satisfy this common request?

proxy / localhost:2016 {
    not /static_files
}

How useful would this be?

[jpoehls]

Yep, I think that would address my needs. My setup is super simple. Proxy everything to X while also serving static files directly. not seems like a workable solution for that.

I prefer the idea of Caddy supporting the idea of "path specific directives" at a higher level but don't have any specific use case at the moment that requires it.

[jpoehls]

If you go the not route, it occurred to me this morning that other directives that do proxy-like things should get the same feature. fastcgi is what came to mind but there may be others.

[mholt]

I'm working on this right now and would like some feedback. @jpoehls @guilhermebr and @pedronasser

#default site root
root /home/a/www

# different roots if request path starts with /b or /c
root /b /home/b/www
root /c /home/c/www

Or maybe allow them to be combined:

root /home/a/www {
    /b /home/b/www
    /c /home/c/www
}

^ I can support both forms if that's more intuitive. What do you think?

Also, what about overlapping base paths:

root /foo ...
root /foo/bar ...

Which root path has precedence for a request like /foo/bar/fun? I imagine we choose the closest match (i.e. longest prefix)?

[coolaj86]

root {
    / /path/to/crazyeight.net
    /login /path/to/example.com/login
    /photoalbum /path/to/photoalbumapp/
    /radioapp /path/to/radioapp/
}

[mholt]

This gets even harder when you consider that some directives run functions at startup and need to know the root directory of the site. For example, markdown can generate a static site when the server starts.

I don't know a good way to solve this yet.

[andyjeffries]

I have a similar problem with Ruby on Rails. I want everything proxied to the Rails application, EXCEPT the /assets path (which should go to the root.

I think for now I can around this by using https://github.com/heroku/rails_serve_static_assets but I think it'll take a performance hit (going through my non-threading single process Rails server for all assets).

[abiosoft]

@andyjeffries You can do that already. https://caddyserver.com/docs/proxy.

proxy / 1270.0.0.1:3000 {
    except /assets
}

[andyjeffries]

@abiosoft That's awesome! Don't know how I'd missed that in the docs (or in my copious Googling around). I've just redeployed all my apps to support static file serving :-)

Thanks mate, really appreciated.

[mholt]

Thanks for everyone's feedback! Closing this issue to funnel the discussion into #619.

[henriquechehad]

:+1:

already started the codes?

[mholt]

Yep.