dockerfile: cleanup of memory allocations

[tonistiigi]

This adds many updates to reduce memory allocations in Dockerfile frontend when dealing with huge inputs, improving performance and reducing GC pressure for other component.

Most of the issues are in dockerfile parser code and have been there forever (pre-buildkit). There was issue with excessive env reads in variable expansion as well. I also switched the llb.Env() datatype although performance impact of that was not very significant and it should be considered more as a cleanup of unneeded conversions. CSV parsers have also been switched as we used them for parsing small single-line values while stdlib parser is optimized for big documents.

Profile of allocations with giant Dockerfile from https://github.com/moby/buildkit/issues/4948 (5x)

Before:

(pprof) top20
Showing nodes accounting for 2.92GB, 76.90% of 3.79GB total
Dropped 2584 nodes (cum <= 0.02GB)
Showing top 20 nodes out of 198
      flat  flat%   sum%        cum   cum%
    0.65GB 17.18% 17.18%     1.13GB 29.74%  github.com/moby/buildkit/client/llb.EnvList.AddOrReplace (inline)
    0.57GB 14.93% 32.11%     0.57GB 14.93%  github.com/moby/buildkit/frontend/dockerfile/shell.(*wordsStruct).addRawChar (inline)
    0.48GB 12.57% 44.68%     0.48GB 12.57%  github.com/moby/buildkit/client/llb.EnvList.Delete (inline)
    0.24GB  6.22% 50.89%     0.24GB  6.22%  github.com/moby/buildkit/frontend/dockerfile/parser.extractBuilderFlags
    0.15GB  4.00% 54.90%     0.16GB  4.10%  github.com/moby/buildkit/frontend/dockerfile/shell.(*Lex).init
    0.14GB  3.75% 58.64%     0.14GB  3.75%  github.com/moby/buildkit/frontend/dockerfile/shell.BuildEnvs
    0.14GB  3.62% 62.26%     0.14GB  3.62%  bufio.NewReaderSize
    0.11GB  2.95% 65.21%     0.69GB 18.21%  github.com/moby/buildkit/frontend/dockerfile/parser.Parse
    0.08GB  2.16% 67.37%     0.08GB  2.16%  github.com/moby/buildkit/client/llb.EnvList.ToArray
    0.05GB  1.41% 68.79%     0.05GB  1.41%  bytes.growSlice
    0.05GB  1.19% 69.98%     0.05GB  1.25%  go.etcd.io/bbolt.(*DB).allocate
    0.04GB  1.11% 71.08%     1.20GB 31.58%  github.com/moby/buildkit/client/llb.addEnvf.func1.1
    0.04GB  1.03% 72.11%     0.04GB  1.03%  golang.org/x/net/http2/hpack.AppendHuffmanString
    0.03GB  0.82% 72.93%     0.03GB  0.82%  google.golang.org/grpc.nopBufferPool.Get
    0.03GB  0.74% 73.67%     1.15GB 30.25%  github.com/moby/buildkit/client/llb.addEnvf.func1.1.getEnv.func1
    0.03GB  0.71% 74.38%     0.03GB  0.71%  github.com/gogo/protobuf/types.(*Any).Unmarshal
    0.03GB  0.68% 75.06%     0.03GB  0.68%  google.golang.org/protobuf/internal/impl.mergeBytesNoZero
    0.02GB  0.65% 75.71%     0.02GB  0.65%  io.ReadAll
    0.02GB   0.6% 76.30%     0.02GB   0.6%  github.com/moby/buildkit/frontend/dockerfile/parser.parseWords
    0.02GB   0.6% 76.90%     0.02GB   0.6%  encoding/csv.(*Reader).readRecord
(pprof)
(pprof) top20 -cum
Showing nodes accounting for 901.51MB, 23.21% of 3883.95MB total
Dropped 2584 nodes (cum <= 19.42MB)
Showing top 20 nodes out of 198
      flat  flat%   sum%        cum   cum%
         0     0%     0%  2768.01MB 71.27%  golang.org/x/sync/errgroup.(*Group).Go.func1
    7.35MB  0.19%  0.19%  2672.03MB 68.80%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.toDispatchState
         0     0%  0.19%  2661.26MB 68.52%  github.com/moby/buildkit/frontend/dockerfile/builder.Build.func6
         0     0%  0.19%  2634.52MB 67.83%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.Dockerfile2LLB
         0     0%  0.19%  2620.18MB 67.46%  github.com/moby/buildkit/frontend/dockerui.(*Client).Build.func1
    2.77MB 0.071%  0.26%  1834.50MB 47.23%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.dispatch
    4.58MB  0.12%  0.38%  1488.46MB 38.32%  github.com/moby/buildkit/frontend/dockerfile/instructions.parseMount
    4.85MB  0.12%   0.5%  1473.75MB 37.94%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.dispatch.func1
    0.79MB  0.02%  0.52%  1459.55MB 37.58%  github.com/moby/buildkit/frontend/dockerfile/instructions.setMountState
         0     0%  0.52%  1339.45MB 34.49%  github.com/moby/buildkit/frontend/dockerfile/instructions.(*RunCommand).Expand
   42.94MB  1.11%  1.63%  1226.72MB 31.58%  github.com/moby/buildkit/client/llb.addEnvf.func1.1
   28.61MB  0.74%  2.37%  1174.74MB 30.25%  github.com/moby/buildkit/client/llb.addEnvf.func1.1.getEnv.func1
  667.12MB 17.18% 19.54%  1155.17MB 29.74%  github.com/moby/buildkit/client/llb.EnvList.AddOrReplace (inline)
   13.88MB  0.36% 19.90%  1122.60MB 28.90%  github.com/moby/buildkit/client/llb.State.Env
    1.58MB 0.041% 19.94%  1040.14MB 26.78%  github.com/moby/buildkit/client/llb.State.Env.getEnv.func1
    6.94MB  0.18% 20.12%   804.23MB 20.71%  github.com/moby/buildkit/frontend/dockerfile/shell.(*Lex).process
  114.52MB  2.95% 23.07%   707.38MB 18.21%  github.com/moby/buildkit/frontend/dockerfile/parser.Parse
         0     0% 23.07%   671.93MB 17.30%  github.com/moby/buildkit/frontend/dockerfile/shell.(*Lex).ProcessWord
         0     0% 23.07%   642.40MB 16.54%  github.com/moby/buildkit/frontend/dockerfile/shell.(*shellWord).process
    5.57MB  0.14% 23.21%   642.40MB 16.54%  github.com/moby/buildkit/frontend/dockerfile/shell.(*shellWord).processStopOn

After:

(pprof) top20
Showing nodes accounting for 468.47MB, 42.97% of 1090.19MB total
Dropped 2432 nodes (cum <= 5.45MB)
Showing top 20 nodes out of 341
      flat  flat%   sum%        cum   cum%
   46.23MB  4.24%  4.24%    47.94MB  4.40%  go.etcd.io/bbolt.(*DB).allocate
   44.21MB  4.06%  8.30%    44.21MB  4.06%  unicode/utf8.AppendRune (inline)
   40.04MB  3.67% 11.97%    40.04MB  3.67%  golang.org/x/net/http2/hpack.AppendHuffmanString
   30.99MB  2.84% 14.81%    30.99MB  2.84%  google.golang.org/grpc.nopBufferPool.Get
   28.87MB  2.65% 17.46%    28.87MB  2.65%  bytes.growSlice
   26.31MB  2.41% 19.87%    26.31MB  2.41%  google.golang.org/protobuf/internal/impl.mergeBytesNoZero
   25.05MB  2.30% 22.17%    25.06MB  2.30%  io.ReadAll
   21.55MB  1.98% 24.15%    21.55MB  1.98%  github.com/sirupsen/logrus.(*Entry).WithFields
   21.47MB  1.97% 26.12%    34.33MB  3.15%  encoding/json.Marshal
   21.07MB  1.93% 28.05%    21.07MB  1.93%  github.com/gogo/protobuf/types.(*Any).Unmarshal
   20.42MB  1.87% 29.92%    35.05MB  3.22%  encoding/json.(*decodeState).literalStore
   19.30MB  1.77% 31.69%    19.97MB  1.83%  github.com/moby/buildkit/frontend/gateway/pb.(*SolveResponse).Marshal
   19.08MB  1.75% 33.44%   126.33MB 11.59%  github.com/moby/buildkit/solver.(*scheduler).dispatch
   17.50MB  1.61% 35.05%    17.50MB  1.61%  encoding/base64.(*Encoding).EncodeToString
   16.75MB  1.54% 36.58%    16.75MB  1.54%  go.etcd.io/bbolt.(*node).read
   15.53MB  1.42% 38.01%    15.53MB  1.42%  github.com/moby/buildkit/solver/internal/pipe.New
   14.18MB  1.30% 39.31%    14.69MB  1.35%  fmt.Sprintf
   13.63MB  1.25% 40.56%    13.63MB  1.25%  go.etcd.io/bbolt.(*Tx).writeMeta
   13.17MB  1.21% 41.77%    44.24MB  4.06%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.dispatch.func1
   13.12MB  1.20% 42.97%    13.12MB  1.20%  github.com/moby/buildkit/api/services/control.(*BuildHistoryRecord).Marshal
(pprof)
(pprof) top20 -cum
Showing nodes accounting for 29.41MB, 2.70% of 1090.19MB total
Dropped 2432 nodes (cum <= 5.45MB)
Showing top 20 nodes out of 341
      flat  flat%   sum%        cum   cum%
         0     0%     0%   438.28MB 40.20%  golang.org/x/sync/errgroup.(*Group).Go.func1
         0     0%     0%   275.03MB 25.23%  github.com/moby/buildkit/frontend/dockerfile/builder.Build.func6
         0     0%     0%   274.94MB 25.22%  github.com/moby/buildkit/frontend/dockerui.(*Client).Build.func1
         0     0%     0%   236.46MB 21.69%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.Dockerfile2LLB
    7.46MB  0.68%  0.68%   236.24MB 21.67%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.toDispatchState
         0     0%  0.68%   198.40MB 18.20%  google.golang.org/grpc.(*Server).handleStream
         0     0%  0.68%   198.39MB 18.20%  google.golang.org/grpc.(*Server).serveStreams.func1.1
    0.01MB 0.0011%  0.69%   197.76MB 18.14%  google.golang.org/grpc.(*Server).processUnaryRPC
    2.71MB  0.25%  0.93%   150.28MB 13.78%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.dispatch
   19.08MB  1.75%  2.68%   126.33MB 11.59%  github.com/moby/buildkit/solver.(*scheduler).dispatch
         0     0%  2.68%   126.33MB 11.59%  github.com/moby/buildkit/solver.(*scheduler).loop
         0     0%  2.68%   113.19MB 10.38%  go.etcd.io/bbolt.(*DB).Update
         0     0%  2.68%   100.89MB  9.25%  github.com/moby/buildkit/util/grpcerrors.UnaryServerInterceptor
         0     0%  2.68%   100.89MB  9.25%  google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1
         0     0%  2.68%   100.89MB  9.25%  main.unaryInterceptor
         0     0%  2.68%   100.89MB  9.25%  google.golang.org/grpc.getChainUnaryHandler.func1
         0     0%  2.68%   100.75MB  9.24%  sync.(*Once).doSlow
         0     0%  2.68%   100.57MB  9.23%  sync.(*Once).Do (inline)
    0.08MB 0.0072%  2.69%    95.27MB  8.74%  github.com/moby/buildkit/solver.(*edge).unpark
    0.08MB 0.0072%  2.70%    85.74MB  7.86%  github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.toDispatchState.toDispatchState.func2.func3

There are many unrelated updates in here so definitely review per-commit. I can also start to split it to more reasonable smaller PRs. I did it all together as only then the profiler result is reasonable.

[tonistiigi]

Some things that I looked at but did not fix:

• Although #4971 added caching of config requests, there is no cache on converting the config JSON to image object. Unmarshal and new allocation of Image happens for every base image.
• Variable expansions fixed lot of issues, it was missing lot of buffers for reuse. One buffer it did have was the result buffer https://github.com/moby/buildkit/blob/v0.14.1/frontend/dockerfile/shell/lex.go#L181 . This is better than the string concat that was in other places but still new buffer is needed for each expansion. I couldn't make this easily reusable as the function calls itself in certain conditions and currently expects to start with fresh buffer each time. Changes in the function code would be needed to change that semantics.
• It would be nice to have some quick check that there are no expansions possible for an input as this is the case most of the time. But the function also does some word splitting so I didn't look too deeply what would be needed for it.
• In AST parsing there are more places where buffers could be reused instead of reallocating, eg. in flag parsing.
• I updated some case where regexp was used where HasPrefix was enough but there are still cases like https://github.com/moby/buildkit/blob/v0.14.1/frontend/dockerfile/parser/parser.go#L514-L517 that look far for optimal for the use case.