Built-in Memory session leaks?

[Allendar]

Loading a simple c.Session() (not even a get/set on it) on the context and then running wrk -d10s -t3 -c100 https://localhost:8443 benchmark spikes my initial app's 50MB memory instantly to 450MB+. Turning the session-access off on a test page in my application and the whole overhead is instantly gone and even after running multiple runs the memory stays super steady the same low usage.

Is this virtual growth or really a bug? I'm using github.com/gorilla/securecookie as the encoder/decoder, but even if I just use native unmodified built-in memory session storage it keeps on leaking. Every re-run on the same process just add another 350~450MB to the process' memory usage.

This was a note I already had on my list for a long time using Iris, but as the benchmark will never be a use-case in my situation (yet) I kind of postponed it. I hope maybe we can look at this together and figure out what causes this and if the concern should be valid.

[Allendar]

I noticed this is also the reason I'm only achieving 38k~ requests per second. When I turn session-load/init off and literally keep all my other full production code running (with middleware/etc.) I easily get 47~51k per benchmark-run.

[Dexus]

Do you use the default sessions? MemStore? Then you need to know each request will generate a new sessionId which will for sure blow the memory to your 350-450MiB.

I'm right? Maybe you try also redis as session storage and compare the usage of your program after that.

[Allendar]

Yes, but with securecookie. Without it's the same issue tho. The same without a custom SessionDB too actually. So even just running the example basic memstore gives the issue.

    hashKey := []byte(config.Values.Security.CookieHashKey)
    blockKey := []byte(config.Values.Security.CookieBlockKey)
    secureCookie := securecookie.New(hashKey, blockKey)
    SessionDB = sessiondb.New(filepath.FromSlash(config.Values.Paths.Store + "/session"))
    sess := sessions.New(sessions.Config{
        Cookie: config.Values.Security.SessionCookieName,
        DisableSubdomainPersistence: !config.Values.Security.AllowSubdomainCookieAccess,
        Expires: (time.Duration(config.Values.Security.SessionCookieExpiration) *
            time.Second),
        Encode: secureCookie.Encode,
        Decode: secureCookie.Decode,
    })
    sess.UseDatabase(SessionDB)
    M.AttachSessionManager(sess)

Redis is not an option. Once I link Redis (even over socket) my req/sec go down below 1000 per/second.

[Allendar]

Maybe the benchmark keeps making new cookies (and new sessions), or has none to say so, and that's the issue?

[Dexus]

Securecookie use the same memstore. So its no bug when the memory is the store. When you use redis or an other storage backend, you dont blow your memory usage. But i have not yet tried it, i'm only write you from my bed 😅 My wife is to loud... 😴

[Allendar]

Haha no worries. Go to sleep! It's late here too 😆.

I think we need to research this a bit more tho. Maybe it's an added effect of the benchmark. I'll try to see tomorrow (movie-night first :D) or in the weekend if I can run a benchmark that injects a cookiejar.

[Dexus]

Year the wrk dont use cookies as far as i know. But with a lua script you can set your session cookie: example: https://www.darklaunch.com/2014/02/11/use-wrk-benchmark-tool-to-stress-test-a-website

[Allendar]

Thank you I will look into it too! This issue is currently the only anomly I have in my whole project performance/memory-wise, so if I can be sure it's really not an issue on bigger scaled environments, then I can sleep better too :).

[Dexus]

I have now checked it myself i only get 10 random sessionids and don't blow my memory ;) so looks all fine fore me.

If you have other thoughts, let me know. Else please close this yourself. Thanks.

[Allendar]

This reproduces it for me on macOS Sierra 10.12.5 on Go 1.8.3 with Siris 7.3.4:

package main

import (
    "github.com/go-siris/siris"
    "github.com/go-siris/siris/context"
    "github.com/go-siris/siris/sessions"
)

func main() {
    app := siris.New()

    app.AttachSessionManager(sessions.New(sessions.Config{}))

    app.Get("/", func (c context.Context) {
        c.Session()
        c.WriteString("Content")
    })

    app.Run(siris.Addr(":8080"))
}

Start says the memory is 3,3MB.

Then running wrk -d10s -t3 -c100 http://localhost:8080 once and it's 195,0MB (not giving back memory, even after waiting the max 7 minutes on GC).

Running it again the memory just goes up to 395,0MB. Third run 628,9MB.

Details of the process say:

Physical memory: 633,2 MB
Virtual memory: 530,86 GB
Shared memory: 248 KB
Private memory: 628,7MB

To be clear; It really is c.Session(). Quoting out that line just increases the memory from 3,3MB to 7,7MB once. And then no matter how many runs of wrk is run, it stays super stable on it.

[Dexus]

If no Expire is set, it is unlimited read source sessions.go

Von meinem iPhone gesendet

Am 08.07.2017 um 16:16 schrieb Allendar notifications@github.com:

This reproduces it for me on macOS Sierra 10.12.5 on Go 1.8.3 with Siris 7.3.4:

package main

import ( "github.com/go-siris/siris" "github.com/go-siris/siris/context" "github.com/go-siris/siris/sessions" )

func main() { app := siris.New()

app.AttachSessionManager(sessions.New(sessions.Config{}))

app.Get("/", func (c context.Context) {
    c.Session()
    c.WriteString("Content")
})

app.Run(siris.Addr(":8080"))

} Start says the memory is 3,3MB.

Then running wrk -d10s -t3 -c100 http://localhost:8080 once and it's 195,0MB (not giving back memory, even after waiting the max 7 minutes on GC).

Running it again the memory just goes up to 395,0MB. Third run 628,9MB.

Details of the process say:

Physical memory: 633,2 MB Virtual memory: 530,86 GB Shared memory: 248 KB Private memory: 628,7MB — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

[Allendar]

What would be a solution for this problem that won't cause massive leaks on high-velocity servers?

[Allendar]

If I set the sessions.Config.Expires to 15 seconds as a test the peak just stays high too. Even after waiting a few minutes.

Update: Have it running for a while now and did some random browser visits in the meanwhile too and it seems the memory never gets freed again.

[Dexus]

Okay, I will make a pprof to this...

[Dexus]

So after profiling and debugging I can't find any reason that the implementation has a memory leak.

(pprof) top10
54.54MB of 54.54MB total (  100%)
Dropped 19 nodes (cum <= 279.23kB)
Showing top 10 nodes out of 40 (cum >= 1024.30kB)
      flat  flat%   sum%        cum   cum%
   21.50MB 39.43% 39.43%    36.53MB 66.99%  github.com/go-siris/siris/sessions.(*provider).newSession
       8MB 14.67% 54.10%     9.53MB 17.48%  time.AfterFunc
    6.50MB 11.92% 66.02%     6.50MB 11.92%  runtime.hashGrow
       6MB 11.00% 77.02%        6MB 11.00%  runtime.rawstringtmp
       6MB 11.00% 88.02%        6MB 11.00%  runtime.makemap
       2MB  3.67% 91.69%        2MB  3.67%  runtime.malg
    1.53MB  2.81% 94.49%     1.53MB  2.81%  runtime.addtimerLocked
    1.50MB  2.75% 97.25%        8MB 14.67%  runtime.mapassign
    0.50MB  0.92% 98.17%     0.50MB  0.92%  net/http.newBufioWriterSize
    0.50MB  0.92% 99.08%        1MB  1.83%  net/http.readRequest

Total: 54.54MB
ROUTINE ======================== github.com/go-siris/siris/sessions.(*provider).newSession in /root/go/src/github.com/go-siris/siris/sessions/provider.go
   21.50MB    36.53MB (flat, cum) 66.99% of Total
         .          .     45:
         .          .     46:   sess := &session{
         .          .     47:           sid:      sid,
         .          .     48:           provider: p,
         .          .     49:           values:   p.loadSessionValuesFromDB(sid),
      16MB    21.50MB     50:           flashes:  make(map[string]*flashMessage),
         .          .     51:   }
         .          .     52:
         .          .     53:   if expires > 0 { // if not unlimited life duration and no -1 (cookie remove action is based on browser's session)
    5.50MB     5.50MB     54:           time.AfterFunc(expires, func() {
         .          .     55:                   // the destroy makes the check if this session is exists then or not,
         .          .     56:                   // this is used to destroy the session from the server-side also
         .          .     57:                   // it's good to have here for security reasons, I didn't add it on the gc function to separate its action
         .          .     58:                   p.Destroy(sid)
         .          .     59:           })
         .          .     60:   }
         .          .     61:
         .     9.53MB     62:   return sess
         .          .     63:}
         .          .     64:
         .          .     65:// can return nil
         .          .     66:func (p *provider) loadSessionValuesFromDB(sid string) memstore.Store {
         .          .     67:   var store memstore.Store

(pprof) tree
80.62MB of 80.62MB total (  100%)
Dropped 19 nodes (cum <= 0.40MB)
----------------------------------------------------------+-------------
      flat  flat%   sum%        cum   cum%   calls calls% + context
----------------------------------------------------------+-------------
                                           41.53MB   100% |   github.com/go-siris/siris/sessions.(*provider).Init
      22MB 27.29% 27.29%    41.53MB 51.52%                | github.com/go-siris/siris/sessions.(*provider).newSession
                                           12.53MB 64.16% |   time.AfterFunc
                                               7MB 35.84% |   runtime.makemap
----------------------------------------------------------+-------------
                                           12.53MB   100% |   github.com/go-siris/siris/sessions.(*provider).newSession
      11MB 13.64% 40.94%    12.53MB 15.54%                | time.AfterFunc
                                            1.53MB   100% |   time.startTimer
----------------------------------------------------------+-------------
                                           12.50MB 80.64% |   github.com/go-siris/siris/sessions.(*provider).Init
                                            1.50MB  9.68% |   net/http.Header.clone
                                               1MB  6.45% |   net/textproto.MIMEHeader.Add
                                            0.50MB  3.23% |   net/textproto.(*Reader).ReadMIMEHeader
       9MB 11.17% 52.10%    15.50MB 19.23%                | runtime.mapassign
                                            6.50MB   100% |   runtime.hashGrow
----------------------------------------------------------+-------------
                                               9MB   100% |   runtime.slicebytetostring
       9MB 11.16% 63.27%        9MB 11.16%                | runtime.rawstringtmp
----------------------------------------------------------+-------------
                                            7.50MB   100% |   runtime.newproc1
    7.50MB  9.31% 72.57%     7.50MB  9.31%                | runtime.malg
----------------------------------------------------------+-------------
                                               7MB   100% |   github.com/go-siris/siris/sessions.(*provider).newSession
       7MB  8.68% 81.25%        7MB  8.68%                | runtime.makemap
----------------------------------------------------------+-------------
                                            6.50MB   100% |   runtime.mapassign
    6.50MB  8.06% 89.32%     6.50MB  8.06%                | runtime.hashGrow
----------------------------------------------------------+-------------
                                               2MB   100% |   bytes.(*Buffer).grow
       2MB  2.48% 91.80%        2MB  2.48%                | bytes.makeSlice
----------------------------------------------------------+-------------
                                            1.53MB   100% |   runtime.addtimer
    1.53MB  1.90% 93.70%     1.53MB  1.90%                | runtime.addtimerLocked
----------------------------------------------------------+-------------
                                               1MB   100% |   net/http.(*conn).serve
       1MB  1.25% 94.94%        1MB  1.25%                | net/http.newBufioWriterSize
----------------------------------------------------------+-------------
                                               2MB 57.14% |   net/http.SetCookie
                                            1.50MB 42.86% |   github.com/go-siris/siris/sessions.AddCookie
       1MB  1.24% 96.18%     3.50MB  4.34%                | net/http.(*Cookie).String
                                               2MB 80.00% |   bytes.(*Buffer).WriteString
                                            0.50MB 20.00% |   runtime.slicebytetostring
----------------------------------------------------------+-------------
                                            0.58MB   100% |   runtime.newproc1
    0.58MB  0.72% 96.90%     0.58MB  0.72%                | runtime.allgadd
----------------------------------------------------------+-------------
                                            1.50MB   100% |   net/http.(*conn).readRequest
    0.50MB  0.62% 97.52%     1.50MB  1.86%                | net/http.readRequest
                                            0.50MB 50.01% |   net/textproto.(*Reader).ReadMIMEHeader
                                            0.50MB 49.99% |   net/textproto.(*Reader).ReadLine
----------------------------------------------------------+-------------
                                            2.50MB   100% |   net/http.(*conn).serve
    0.50MB  0.62% 98.14%     2.50MB  3.10%                | net/http.(*conn).readRequest
                                            1.50MB 75.00% |   net/http.readRequest
                                            0.50MB 25.00% |   context.WithCancel
----------------------------------------------------------+-------------
                                            0.50MB   100% |   net/http.(*connReader).backgroundRead
    0.50MB  0.62% 98.76%     0.50MB  0.62%                | net.(*conn).Read
----------------------------------------------------------+-------------
                                            8.58MB   100% |   runtime.mstart
    0.50MB  0.62% 99.38%     8.58MB 10.64%                | runtime.systemstack
                                            8.08MB   100% |   runtime.newproc.func1
----------------------------------------------------------+-------------
                                            0.50MB   100% |   net/http.(*conn).readRequest
    0.50MB  0.62%   100%     0.50MB  0.62%                | context.WithCancel
----------------------------------------------------------+-------------
                                               2MB   100% |   net/http.(*Cookie).String
         0     0%   100%        2MB  2.48%                | bytes.(*Buffer).WriteString
                                               2MB   100% |   bytes.(*Buffer).grow
----------------------------------------------------------+-------------
                                               2MB   100% |   bytes.(*Buffer).WriteString
         0     0%   100%        2MB  2.48%                | bytes.(*Buffer).grow
                                               2MB   100% |   bytes.makeSlice
----------------------------------------------------------+-------------
                                           68.04MB   100% |   github.com/go-siris/siris/core/router.(*routerHandler).HandleRequest
         0     0%   100%    68.04MB 84.39%                | github.com/go-siris/siris/context.(*context).Do
                                           68.04MB   100% |   main.main.func1
----------------------------------------------------------+-------------
                                           66.54MB   100% |   main.main.func1
         0     0%   100%    66.54MB 82.53%                | github.com/go-siris/siris/context.(*context).Session
                                           66.54MB   100% |   github.com/go-siris/siris/sessions.(*Manager).Start
----------------------------------------------------------+-------------
                                            1.50MB   100% |   main.main.func1
         0     0%   100%     1.50MB  1.86%                | github.com/go-siris/siris/context.(*context).WriteString
                                            1.50MB   100% |   github.com/go-siris/siris/context.(*responseWriter).WriteString
----------------------------------------------------------+-------------
                                            1.50MB   100% |   github.com/go-siris/siris/context.(*context).WriteString
         0     0%   100%     1.50MB  1.86%                | github.com/go-siris/siris/context.(*responseWriter).WriteString
                                            1.50MB   100% |   github.com/go-siris/siris/context.(*responseWriter).tryWriteHeader
----------------------------------------------------------+-------------
                                            1.50MB   100% |   github.com/go-siris/siris/context.(*responseWriter).WriteString
         0     0%   100%     1.50MB  1.86%                | github.com/go-siris/siris/context.(*responseWriter).tryWriteHeader
                                            1.50MB   100% |   net/http.(*response).WriteHeader
----------------------------------------------------------+-------------
                                           68.04MB   100% |   github.com/go-siris/siris/core/router.(*Router).ServeHTTP
         0     0%   100%    68.04MB 84.39%                | github.com/go-siris/siris/core/router.(*Router).BuildRouter.func1
                                           68.04MB   100% |   github.com/go-siris/siris/core/router.(*routerHandler).HandleRequest
----------------------------------------------------------+-------------
                                           68.04MB   100% |   net/http.serverHandler.ServeHTTP
         0     0%   100%    68.04MB 84.39%                | github.com/go-siris/siris/core/router.(*Router).ServeHTTP
                                           68.04MB   100% |   github.com/go-siris/siris/core/router.(*Router).BuildRouter.func1
----------------------------------------------------------+-------------
                                           68.04MB   100% |   github.com/go-siris/siris/core/router.(*Router).BuildRouter.func1
         0     0%   100%    68.04MB 84.39%                | github.com/go-siris/siris/core/router.(*routerHandler).HandleRequest
                                           68.04MB   100% |   github.com/go-siris/siris/context.(*context).Do
----------------------------------------------------------+-------------
                                           66.54MB   100% |   github.com/go-siris/siris/context.(*context).Session
         0     0%   100%    66.54MB 82.53%                | github.com/go-siris/siris/sessions.(*Manager).Start
                                           54.03MB 81.21% |   github.com/go-siris/siris/sessions.(*provider).Init
                                               8MB 12.02% |   github.com/go-siris/siris/sessions.Config.Validate.func1
                                            4.50MB  6.76% |   github.com/go-siris/siris/sessions.AddCookie
----------------------------------------------------------+-------------
                                           54.03MB   100% |   github.com/go-siris/siris/sessions.(*Manager).Start
         0     0%   100%    54.03MB 67.02%                | github.com/go-siris/siris/sessions.(*provider).Init
                                           41.53MB 76.86% |   github.com/go-siris/siris/sessions.(*provider).newSession
                                           12.50MB 23.14% |   runtime.mapassign
----------------------------------------------------------+-------------
                                            4.50MB   100% |   github.com/go-siris/siris/sessions.(*Manager).Start
         0     0%   100%     4.50MB  5.58%                | github.com/go-siris/siris/sessions.AddCookie
                                               3MB 66.67% |   net/http.SetCookie
                                            1.50MB 33.33% |   net/http.(*Cookie).String
----------------------------------------------------------+-------------
                                               8MB   100% |   github.com/go-siris/siris/sessions.(*Manager).Start
         0     0%   100%        8MB  9.92%                | github.com/go-siris/siris/sessions.Config.Validate.func1
                                               8MB   100% |   github.com/go-siris/siris/vendor/github.com/satori/go%2euuid.UUID.String
----------------------------------------------------------+-------------
                                               8MB   100% |   github.com/go-siris/siris/sessions.Config.Validate.func1
         0     0%   100%        8MB  9.92%                | github.com/go-siris/siris/vendor/github.com/satori/go%2euuid.UUID.String
                                               8MB   100% |   runtime.slicebytetostring
----------------------------------------------------------+-------------
                                           68.04MB   100% |   github.com/go-siris/siris/context.(*context).Do
         0     0%   100%    68.04MB 84.39%                | main.main.func1
                                           66.54MB 97.79% |   github.com/go-siris/siris/context.(*context).Session
                                            1.50MB  2.21% |   github.com/go-siris/siris/context.(*context).WriteString
----------------------------------------------------------+-------------
                                           71.54MB   100% |   runtime.goexit
         0     0%   100%    71.54MB 88.74%                | net/http.(*conn).serve
                                           68.04MB 95.10% |   net/http.serverHandler.ServeHTTP
                                            2.50MB  3.50% |   net/http.(*conn).readRequest
                                               1MB  1.40% |   net/http.newBufioWriterSize
----------------------------------------------------------+-------------
                                            0.50MB   100% |   runtime.goexit
         0     0%   100%     0.50MB  0.62%                | net/http.(*connReader).backgroundRead
                                            0.50MB   100% |   net.(*conn).Read
----------------------------------------------------------+-------------
                                            1.50MB   100% |   github.com/go-siris/siris/context.(*responseWriter).tryWriteHeader
         0     0%   100%     1.50MB  1.86%                | net/http.(*response).WriteHeader
                                            1.50MB   100% |   net/http.Header.clone
----------------------------------------------------------+-------------
                                               1MB   100% |   net/http.SetCookie
         0     0%   100%        1MB  1.24%                | net/http.Header.Add
                                               1MB   100% |   net/textproto.MIMEHeader.Add
----------------------------------------------------------+-------------
                                            1.50MB   100% |   net/http.(*response).WriteHeader
         0     0%   100%     1.50MB  1.86%                | net/http.Header.clone
                                            1.50MB   100% |   runtime.mapassign
----------------------------------------------------------+-------------
                                               3MB   100% |   github.com/go-siris/siris/sessions.AddCookie
         0     0%   100%        3MB  3.72%                | net/http.SetCookie
                                               2MB 66.66% |   net/http.(*Cookie).String
                                               1MB 33.34% |   net/http.Header.Add
----------------------------------------------------------+-------------
                                           68.04MB   100% |   net/http.(*conn).serve
         0     0%   100%    68.04MB 84.39%                | net/http.serverHandler.ServeHTTP
                                           68.04MB   100% |   github.com/go-siris/siris/core/router.(*Router).ServeHTTP
----------------------------------------------------------+-------------
                                            0.50MB   100% |   net/http.readRequest
         0     0%   100%     0.50MB  0.62%                | net/textproto.(*Reader).ReadLine
                                            0.50MB   100% |   runtime.slicebytetostring
----------------------------------------------------------+-------------
                                            0.50MB   100% |   net/http.readRequest
         0     0%   100%     0.50MB  0.62%                | net/textproto.(*Reader).ReadMIMEHeader
                                            0.50MB   100% |   runtime.mapassign
----------------------------------------------------------+-------------
                                               1MB   100% |   net/http.Header.Add
         0     0%   100%        1MB  1.24%                | net/textproto.MIMEHeader.Add
                                               1MB   100% |   runtime.mapassign
----------------------------------------------------------+-------------
                                            1.53MB   100% |   time.startTimer
         0     0%   100%     1.53MB  1.90%                | runtime.addtimer
                                            1.53MB   100% |   runtime.addtimerLocked
----------------------------------------------------------+-------------
         0     0%   100%    72.04MB 89.36%                | runtime.goexit
                                           71.54MB 99.31% |   net/http.(*conn).serve
                                            0.50MB  0.69% |   net/http.(*connReader).backgroundRead
----------------------------------------------------------+-------------
         0     0%   100%     8.58MB 10.64%                | runtime.mstart
                                            8.58MB   100% |   runtime.systemstack
----------------------------------------------------------+-------------
                                            8.08MB   100% |   runtime.systemstack
         0     0%   100%     8.08MB 10.02%                | runtime.newproc.func1
                                            8.08MB   100% |   runtime.newproc1
----------------------------------------------------------+-------------
                                            8.08MB   100% |   runtime.newproc.func1
         0     0%   100%     8.08MB 10.02%                | runtime.newproc1
                                            7.50MB 92.85% |   runtime.malg
                                            0.58MB  7.15% |   runtime.allgadd
----------------------------------------------------------+-------------
                                               8MB 88.89% |   github.com/go-siris/siris/vendor/github.com/satori/go%2euuid.UUID.String
                                            0.50MB  5.56% |   net/http.(*Cookie).String
                                            0.50MB  5.56% |   net/textproto.(*Reader).ReadLine
         0     0%   100%        9MB 11.16%                | runtime.slicebytetostring
                                               9MB   100% |   runtime.rawstringtmp
----------------------------------------------------------+-------------
                                            1.53MB   100% |   time.AfterFunc
         0     0%   100%     1.53MB  1.90%                | time.startTimer
                                            1.53MB   100% |   runtime.addtimer
----------------------------------------------------------+-------------

That's anything i get and it dont blows after the expire.

Maybe a other like to check...

[Allendar]

Did you try this while running high velocity calls or just a one or a few? I also don't experience the growth while doing fast refreshes in the browser. It purely happens when do many concurrent calls.

Maybe it has something to do with GC within goroutines that makes it never release correctly? I know from Objective-C that within a subthread if you disconnect a strong-typed variable from memory without releasing it, the garbage collector will probably never release it again, as it still thinks its typed strong and thus the responsibility of the programmer. I'm not sure how this works on low-level Go tho 🙈.

[Dexus]

I did run this on my dev server and run wrk with a 600s try. Sessions expire after 15s.

Maybe we get an answer from @kataras or maybe the team of @get-ion have an idea.

Von meinem iPhone gesendet

Am 10.07.2017 um 08:57 schrieb Allendar notifications@github.com:

Did you try this while running high velocity calls or just a one or a few? I also don't experience the growth while doing fast refreshes in the browser. It purely happens when do many concurrent calls.

Maybe it has something to do with GC within goroutines that makes it never release correctly? I know from Objective-C that within a subthread if you disconnect a strong-typed variable from memory without releasing it, the garbage collector will probably never release it again, as it still thinks its typed strong and thus the responsibility of the programmer. I'm not sure how this works on low-level Go tho 🙈.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

[Dexus]

Okay now after multiple reviews and tests it is verified the session manager is crap and has no functional GC. Will fix it in the next days.

[theodesp]

It funny as I was reading the book Release-it recently and there was a chapter about sessions.

Once again, we see that sessions are the Achilles heel of web applications. Want to bring down nearly any dynamic web application? Pick a deep link from the site, and start requesting it, without sending cookies. Don’t even wait for the response; just drop the socket connection as soon as you’ve sent the request.

[Allendar]

Yes it requires a lot of care and love to make sure you don't get speed degradation while developing. I don't mean Early optimization is the root of evil per se, but it's good to run benchmarks every few days of development to see if you didn't make a huge slowdown in the code.

For me; the main rule is that everything that comes from outside Go or lives in a wrapper will slow the code down. That's why for my own projects I only allow MySQL as the only external tooling. With smart buffering I can mostly sooth all the relative slowness that comes from it. MySQL will drag Go apps to cap around a few thousand req/sec. Applying some smart buffers on crucial places will easily crank the performance back up to 38k+ req/sec on my MacBook.

Simply linking Redis will just destroy my whole apps. My biggest app on Redis decreases from ~33k req/s to 340 req/sec. Imagine that in most scripting languages Redis is pro-forma, while for Go it's a ball-on-a-chain.

For this reason I made a custom Session DB wrapper that loads on startup and saves on shutdown of the webserver. I honestly never had server crashes or anything that stopped runtime. Might it ever happen that something crashes and some users have to relog, I gladly take that over absolutely destroying performance. I already have a note of doing intermediate saves every X-period, but it'll probably be a config value per installation.

If you want I can share the simple code and maybe we can make it an example or part of the framework.

[Dexus]

Yeah... but how would work online banking, github and other websites without it... 🤔 Will take care for a secure implementing. But you can look at kataras/go-sessions and look to the commits and then you know how good this guy was... lol sorry don't wanna put my finger into this wound... it's to personal... 🙄

[Allendar]

Well you still have your sessions in memory. And you could do intermediate writes every minute to some shm-disk and every 30/60min. to a normal disk.

Also the Gorilla securecookie + force HTTPS on my projects ensures maximum security. Securecookie lowers my current project's req/sec from ~47k/sec to ~38k/sec. It's the only sacrifice I'm willing to make tho.

[speedwheel]

@allendar can you make an example with the session loader on start-up. I was also looking for a solution for this, since redid is a huge slowdown.

[theodesp]

Maybe a fresh start will help https://www.owasp.org/index.php/Session_Management_Cheat_Sheet https://docs.djangoproject.com/en/1.11/topics/http/sessions/

[Allendar]

Here's a raw dump of my interface SessionDB interface. It's a simple version, but works perfect for me: https://gist.github.com/Allendar/748aef2e4144edc5a5362393bad9b0bd

[Dexus]

Sure you can make with securecookies much, bad on the other side is, that you have a big overhead with each request... headers are bad and cookies with 100kb+ exponential bad!

Edit: @theodesp thanks for the links, was just looking for them and you are to fast for me 🙂

[theodesp]

I thought a Memcached session storage was fast because it has an LRU eviction. Slab allocator and stuff. It works in distributed mode also.

[Allendar]

Nothing can probably beat the 0 allocation style that cmap achieves. The only strategy that needs to be figured out is how often to write away the data and how to efficiently distribute blocks and changes.

The biggest drawback on external tools is that most connect over TCP or socket (fs). That's always a big bump in the road, no matter how much you'd tweak.

[theodesp]

Can we provide controls on how much the cmap grows and key eviction? What about stats?

[Allendar]

You'd probably need something like freecache does. It does have expiration/evacuation/hitrate/etc. stuff. But it's probably not very hard to make some sort of channel that would run non-blocking stats as the sessions get read/written. Size-limitation will probably be something that's a bit more difficult. I only store userID, referrer and some flash data in session tho. I never had issues that I would need to control the size for sessions. Problem is that Go can't count memory on runtime well, and if you would want to you'll need dangerous unsafe calls. It might be better to just assume size based on total entry count to decide what to eject.