crash in `ASTMemoryAllocator` when using extremely large sources and `--compile`

When using this test case

itBundled("bun/SourceMapLargeFile", {
  target: "bun",
  compile: true,
  files: {
    "/entry.ts": /* js */ `
      // this test is not checking behaviors with any of these libraries but
      // just using them to increase the size of the binary to see if large
      // payloads have issues. accuracy of large sourcemaps is tested elsewhere.
      import * as aws from ${JSON.stringify(require.resolve("aws-cdk-lib"))};
      import * as webpack from ${JSON.stringify(require.resolve("webpack"))};
      import * as mysql2 from ${JSON.stringify(require.resolve("mysql2"))};

      // this file has comments and weird whitespace, intentionally
      // to make it obvious if sourcemaps were generated and mapped properly
      if           (true) code();
      function code() {
        // hello world
                throw   new
          Error("Hello World");
      }

      console.log(aws, webpack, mysql2);
    `,
  },
  sourceMap: "external",
  onAfterBundle(api) {
    rmSync(api.join("entry.ts"), {}); // Hide the source files for errors
  },
  run: {
    exitCode: 1,
    validate({ stderr }) {
      expect(stderr).toStartWith(
        `1 | // this file has comments and weird whitespace, intentionally
2 | // to make it obvious if sourcemaps were generated and mapped properly
3 | if           (true) code();
4 | function code() {
5 |   // hello world
6 |           throw   new
                    ^
error: Hello World`,
      );
      expect(stderr).toInclude("entry.ts:6:19");
    },
  },
});

I get a crash while bundling in ASTMemoryAllocator.

My proposed solution to this is to just delete ASTMemoryAllocator, as it was sort of leftover code from https://github.com/oven-sh/bun/pull/12388. NewStore is a better bump allocator. Looking at the remaining usages of it, I think it could be replaced with two things.

~~add to a way to push and pop reset points onto the threadlocal NewStore that already exists (this is most uses boil down to)~~
an option to configure how much to pre-allocate at initialization (this takes care of bun install's use-case "The default store we use pre-allocates around 16 MB of memory per thread; That adds up in multi-threaded scenarios.")

EDIT: The above solution is not satisfactory as AstMemoryAllocator is used to hold onto an ast for a lifetime longer than what the store allows for.

Opening an issue to look into later instead of just doing a quick fix, since that quick fix is unlikely to be reliable. the better fix is the harder thing.

oven-sh / bun

crash in `ASTMemoryAllocator` when using extremely large sources and `--compile` #13103