Example, or documentation, for integration of middleware(s)

[kravemir]

I'm considering to move to httptreemux, from go-gin. However, I'm missing middleware support. The basic use case is handling of authorization (extract information from request, validate, and set context parameters).

The README says, that it's easy:

This package provides no middleware. But there are a lot of great options out there and it's pretty easy to write your own.

However, I have no clue, how to add any middleware to httptreemux. Can you please add any example, how to do that for some group of routes?

[dimfeld]

It’s pretty similar to writing middleware for the standard Go http stack. Your middleware function takes a handler function as an argument and then returns a new handler function that does the middleware functionality and, if desired, calls the original handler.

[kravemir]

@dimfeld thank you for the answer.

Your middleware function takes a handler function as an argument and then returns a new handler function that does the middleware functionality and, if desired, calls the original handler.

Do you mean it as function wrapper? Or, as some form of interceptor?

Following example creates middleware as a function wrapper:

package main

import (
  "log"
  "net/http"
)

func middlewareOne(next http.Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    log.Println("Executing middlewareOne")
    next.ServeHTTP(w, r)
    log.Println("Executing middlewareOne again")
  })
}

func middlewareTwo(next http.Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    log.Println("Executing middlewareTwo")
    if r.URL.Path != "/" {
      return
    }
    next.ServeHTTP(w, r)
    log.Println("Executing middlewareTwo again")
  })
}

func final(w http.ResponseWriter, r *http.Request) {
  log.Println("Executing finalHandler")
  w.Write([]byte("OK"))
}

func main() {
  finalHandler := http.HandlerFunc(final)

  http.Handle("/", middlewareOne(middlewareTwo(finalHandler)))
  http.ListenAndServe(":3000", nil)
}

However, gin allows to define a middleware as some form of interceptor, which can be used by all routes withing route group:

func main() {
    // Creates a router without any middleware by default
    r := gin.New()

    // Global middleware
    // Logger middleware will write the logs to gin.DefaultWriter even if you set with GIN_MODE=release.
    // By default gin.DefaultWriter = os.Stdout
    r.Use(gin.Logger())

    // Recovery middleware recovers from any panics and writes a 500 if there was one.
    r.Use(gin.Recovery())

    // Per route middleware, you can add as many as you desire.
    r.GET("/benchmark", MyBenchLogger(), benchEndpoint)

    // Authorization group
    // authorized := r.Group("/", AuthRequired())
    // exactly the same as:
    authorized := r.Group("/")
    // per group middleware! in this case we use the custom created
    // AuthRequired() middleware just in the "authorized" group.
    authorized.Use(AuthRequired())
    {
        authorized.POST("/login", loginEndpoint)
        authorized.POST("/submit", submitEndpoint)
        authorized.POST("/read", readEndpoint)

        // nested group
        testing := authorized.Group("testing")
        testing.GET("/analytics", analyticsEndpoint)
    }

    // Listen and serve on 0.0.0.0:8080
    r.Run(":8080")
}

Question 1: do I need to define middleware as a function wrapper, therefore if I want to wrap some group of routes, then I must manually wrap all these handling functions?

Question 2: is it possible to define a middleware for whole httptreemux? For example, authentication resolution can be applied for all requests, and requests without provided authentication are left as unauthenticated.

[dimfeld]

Yes, it's a function wrapper model, as in your first example. There isn't any built-in support for sharing middleware across multiple routes. Usually what I do in this case is create general functions that wraps a handler in all the middleware I want and then for each endpoint use whichever function I like best. Something like this:

func WrapAllRoutes(handler httptreemux.HandlerFunc) httptreemux.HandlerFunc {
   return middlewareOne(middlewareTwo(handler))
}

var AuthRequiredEndpoint = func(method string, path string, handler httptreemux.Handlerfunc) httptreemux.HandlerFunc{
  router.Handle(method, path, authRequiredMiddleware(WrapAllRoutes(handler))
}

AuthRequiredEndpoint('POST', '/login', loginEndpoint)
AuthRequiredEndpoint('POST', '/submit', submitEndpoint)
router.GET('/analytics', WrapAllRoutes(analyticsEndpoint))

So that's pretty much it. I've purposefully kept middleware out of the scope of this package in the name of speed and simplicity, so if you're looking for more comprehensive built-in support for middleware like the gin example you gave, then you might be happier with a different router package. There are a lot of good ones out there now :)

[kravemir]

@dimfeld thank you for explanation, I understand middlewares now. IMHO, it would be useful (also for others) to add some of provided examples as documentation (maybe linked "docs/EXTRAS.md" file?).

So that's pretty much it. I've purposefully kept middleware out of the scope of this package in the name of speed and simplicity, so if you're looking for more comprehensive built-in support for middleware like the gin example you gave, then you might be happier with a different router package. There are a lot of good ones out there now :)

I chose to go for httptreemux, because of:

1. focus on speed (nice results on go-http-routing-benchmark),
2. supports catch-all routes,
3. supports route "collisions" / "subsets", with prioritization based on specificity (static wins, then wildcard, then catch-all)

So, except for missing convenience about middlewares, it's exactly what I was looking for.

Now, that I understand the concept, middlewares are simple, only helper/convenience methods are missing. However, I can live with such helper methods, which I will need to write on my own. The most probably, I'll build some httptreemux builder/wrapper to make routes+middlewares registration more convenient.

[dimfeld]

Sounds good! I agree that will be useful to have some middleware examples in the documentation. When I have some time I'll look at putting that in.

[kravemir]

@dimfeld

... so if you're looking for more comprehensive built-in support for middleware like the gin example you gave, then you might be happier with a different router package. There are a lot of good ones out there now :)

You're maybe right :-)

In the end, I'm going to refactor to https://github.com/go-chi/chi, because it supports catch-all parameters (compared to gin), and mentioned features like: lightweight, 100% compatible with net/http, no external dependencies...

Though,.. it doesn't beat other best routers with performance (echo, gin, httprouter, yours httptreemux),.. it offers best compromise between convenience, lightweight-ness, performance, and sticking as close as possible to go std library.

Btw,.. Wishing you a Happy New Year and all the best, and success, in software development and creations.

[dimfeld]

Thanks! I think a lot of people use chi and like it, so I hope it works well for you!

[vmihailenco]

@dimfeld what do you think about https://github.com/vmihailenco/httptreemux/commit/8e8ced4ba145cf2e69bfb2c5813955562cb8b86b . Usage is following:

api := router.NewGroup("/api")

// No middlewares so far - zero overhead
api.GET("/a", a)

api.Use(firstMiddleware)

// This handler will use only firstMiddleware
api.GET("/b", b)

api.Use(secondMiddleware)

// This handler will use firstMiddleware and secondMiddleware
api.GET("/c", c)

Such design requires adding a middleware before registering a route and I think it is a reasonable requirement in most cases. Middleware stack implementation itself is rather simple and small. Thoughts?

[dimfeld]

This looks great, thank you!

I'd love to integrate it into this package with some tests. Off the top of my head, these tests would be useful:

• Test a handler with one middleware.
• Multiple middlewares in a stack should execute them in the expected order.
• If a middleware sends a response instead of calling next, nothing bad happens.
• In an example like the one you wrote above, test that handlers a, b, and c execute the expected middlewares.
• Create a group with some middleware, and create a second group within that group. Test that a handler in the second group executes the parent group's middleware.
• Same as above, but with additional middleware on the second group.

I don't have much time to work on this package, so if you can implement these tests I would really appreciate it. If you can't, I'll try to get them done within the next few months. Thanks again!

[dimfeld]

One other thing we may want to do is wrap the ResponseWriter with a structure that allows middleware to see some information about the response. This is useful for things like logging the status code written by a response. I've used a structure like this in my own code:

type ResponseWriter struct {
    http.ResponseWriter
    WroteStatus bool
    StatusCode  int
}

func NewResponseWriter(w *http.ResponseWriter) ResponseWriter {
    return &ResponseWriter{
        ResponseWriter: w,
        WroteStatus:    false,
        StatusCode:     0,
    }
}

func (w *ResponseWriter) GetStatus() int {
    return w.StatusCode
}

func (w *ResponseWriter) WriteHeader(code int) {
    w.ResponseWriter.WriteHeader(code)
    w.StatusCode = code
    w.WroteStatus = true
}

[vmihailenco]

One other thing we may want to do is wrap the ResponseWriter

I think this is best to leave outside of the package. Even though that requirement is very common - most of us already have such wrappers and it is not very realistic to expect that world will start using httptreemux implementation. E.g. there is https://github.com/open-telemetry/opentelemetry-go/blob/master/plugin/othttp/wrap.go#L50 - doing the same in httptreemux will duplicate the effort and the cost.

And it is trivial to do with middleware, e.g.

router.Use(func(next HandlerFunc) HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request, params map[string]string) {
        w = wrap(w)
        next(w, r, params)
    }
})

But since you mentioned it - I would like to add a wrapper around r *http.Request, params map[string]string to make handler signature smaller, e.g.

func(w http.ResponseWriter, req httptreemux.Request) {
    id := req.Params.Uint64("id") // helper for strconv.ParseUint
    id, ok := req.Params.Get("id") // same as params["id"]
}

type Request struct {
    *http.Request
    Ctx context.Context // to allow setting context without req.WithContext() which allocates
    Params Params
}

type Params struct {
    kvs []param // I believe this should be faster than map[string]string for low number of params
}

type param struct {
    key string
    value string
}

All of that are micro-optimizations that break backwards compatibility so I will understand if you think they are not worth releasing v6. But with middlewares some functionality can be moved outside of the package, e.g. PanicHandler:

router.Use(func(next HandlerFunc) HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request, params map[string]string) {
        defer func() {
            if err := recover(); err != nil {
                panicHandler(w, r, err)
            }
        }
        next(w, r, params)
    }
})

So it may be worth considering...

[dimfeld]

Yeah, at this point in the library's life I would prefer not to break backwards compatibility unless absolutely necessary. And I know there are people using this package who aren't on a recent Go version with modules support too.

Good point about just wrapping the ResponseWriter in the middleware itself as needed.

I'll try to take a look at the PR later today. Thanks again!

[dimfeld]

Closed by #71. Thank you!