Incorrect ordering of middleware in default project

[MikeBeller]

Hi -- I'm pretty sure the stack of middleware in the default template project is incorrect. Currently the template has it as:

(def app (as-> routes/app ?
               (handler ?)
               (layout ? layout/app)
               (logger ?)
               (csrf-token ?)
               (session ?)
               (extra-methods ?)
               (query-string ?)
               (body-parser ?)
               (server-error ?)
               (x-headers ?)
               (static-files ?)))

But the problem is (1) the server returns "internal error" instead of "not found" for a normal not found case (because not found checking should be after static files lookup, right?) And (2), there is no logging provided for not found cases because, well logging should be at the end when all the handlers, including static-files, have had a chance to fail.

I believe the ordering should be as follows:

(def app (as-> routes/app ?
               (handler ?)
               (layout ? layout/app)
               (csrf-token ?)
               (session ?)
               (extra-methods ?)
               (query-string ?)
               (body-parser ?)
               (server-error ?)
               (x-headers ?)
               (static-files ?)
               (not-found ?)
               (logger ?)))

The same fix should probably provided in joy/docs/middleware.md and anywhere else a default stack is provided, especially one which includes static-files.

[swlkr]

This is timely, because I did finally wrap the default stack in a function here: https://github.com/joy-framework/joy/blob/master/src/joy/router.janet#L215 so definitely need to hash some of this stuff out.

not-found is just plain missing from the old default stack, I usually add it myself but yeah that's super confusing. I did manage to add it to the new default stack though 😅

When it comes to whether or not it's below or above static-files it doesn't really matter since I tried it both ways and it still returns a 404 either way, but it makes logical sense on the bottom, so I'll change that.

As far as the logger on the bottom goes, I'm not sure why I had it way up there, now that I think about it, but I did have it above static files because every static file gets logged and it gets noisy if you have a few. I'll probably add an exclusion thing for static-files 200 responses or something like log levels to get rid of the noise.

The middleware should be fixed on master now.

[MikeBeller]

Oh -- interesting! I looked at the code you mention and it appears you are developing a sort of "auto app" capability. Cool! Is that described in the docs anywhere? (A quick note -- I didn't test it but in auto-routes code doesn't your filter at the top of auto-routes risk catching the "/" and "/=" built in operators?)

As to static file logging: I hear what you are saying about too many log entries, but believe log levels or logging options make the most sense. I found it helpful to have all logging enabled because I found the static file handler surprising -- app.js is found at /app.js and not at /public/app.js, which is different than I have experienced in other frameworks. Until I had logging I couldn't figure out what was going on. So with options could "turn it up to 11" for debugging (https://en.wikipedia.org/wiki/Up_to_eleven) and then turn it back down for production.

[swlkr]

Hmm that's a good point about auto-routes, probably should have thought that one through more 🤦‍♂️

Really, what I was doing was looking for a way to have functions and route definitions live next to each other, so I didn't have to update code in two places when defining a route. auto-routes turns out to have serious drawbacks like what you're describing. I did come up with something and now that looks like this:

(use joy)

(route :get "/" :home)
(defn home [request]
  (text/plain "home"))

(def app (app))

(server app (env :port))

This one comes with a downside too that you can't run the server behind janet's main.janet situation, but that's a trade off I'm willing to make, since defroutes/routes still works.

Ah, that's a good point about the static files being at root by default, you should be able to change that here: https://github.com/joy-framework/joy/blob/master/src/joy/middleware.janet#L22-L23

If you change it to ./ static files should then be served from /public/app.css which at that point I believe all source files might be served statically, I haven't tried this yet.

For the logging, yeah I'll definitely add logging levels

[swlkr]

Oh and the auto-app, bundled middleware thing isn't documented anywhere yet, was going to try to get that done today

[MikeBeller]

I like the new approach you suggest:

(route :get "/" :home)
(defn home [request]
  (text/plain "home"))

I tried it out and it's intuitive and works great!

On the other hand -- I'm still thrown off a bit by static-files. I feel like it should be possible for it to work the way you have it (all files in ./public are served as if they were in the local directory) but also possible to make it so that all files in public are served from a specific root uri -- e.g. :get /static/app.js will retrieve ./public/app.js and return it to the client.

The go "express" web framework has these abilities for example-- https://expressjs.com/en/starter/static-files.html

[swlkr]

Oh interesting, the virtual path /static -> /public seems confusing, but I guess there's a use for it? Here's a quick middleware that does it:

(defn virtual-static-files [handler &opt options]
  (def default-options {:root "./public" :prefix "/"})
  (default options default-options)
  (def options (merge default-options options))

  (fn [request]
    (let [{:root root :prefix prefix} options
          {:method method :uri uri} request
          filename (path/join root (string/replace prefix "" uri))]
      (if (and (some (partial = method) ["GET" "HEAD"])
            (path/ext filename)
            (file-exists? filename)
            (string/has-prefix? prefix uri))
        {:file filename}
        (handler request)))))

Here's how you can use it

(def app (as-> routes ?
               (handler ?)
               (csrf-token ?)
               (session ?)
               (extra-methods ?)
               (query-string ?)
               (body-parser ?)
               (server-error ?)
               (x-headers ?)
               (not-found ?)
               (virtual-static-files ? {:prefix "/static"})
               (logger ?)))

(server app 9001)

I might make a breaking change 😱 and change the built-in static-files middleware or I might just add a better named with-files middleware and use that quick middleware from above in core instead of a breaking change, not sure yet.

[swlkr]

I should add for anyone following along that in prod it's always better to NOT use joy's underlying http server halo to serve static files, always use nginx or caddy or something for that.

[MikeBeller]

Thanks @swlkr for the response. I'm not sure my case is common, so it may not be worth you making any changes for it. I can clearly always use the above workaround for my case.

[swlkr]

I'm going to close this since I believe it's been solved