metaverse / truss

Truss helps you build go-kit microservices without having to worry about writing or maintaining boilerplate code.
Other
734 stars 143 forks source link

perf: skip walking unnecessary dirs when reading previous generation #302

Closed zaquestion closed 3 years ago

zaquestion commented 3 years ago

Noticed an error like this after running truss with a flattened service path.

truss --svcout . service.proto
FATA[0000] cannot parse input: cannot read previously generated files: cannot fully walk directory /home/zaq/projects/go-mod/src/github.com/<redacted>: cannot read file: /home/zaq/projects/go-mod/src/github.com/<redacted>/.git/refs/remotes/origin/<redacted>: open /home/zaq/projects/go-mod/src/github.com/<redacted>/.git/refs/remotes/origin/<some_branch>: too many open files

NOTE: was using default nofiles limit on linux

$ ulimit -Sn
1024

When using --svcout ., cmd, svc, and handlers folder will be created in the current directory. Which is really nice if you want a flatter service structure. However, this has the unintented consequence of opening significantly more files when truss is building context on it's previous generation. This change cuts down the files read to only those in the handlers dir as these are the only files whose previous generation is leveraged

Ref: gengokit/generator/gen.go:generateResponseFile