Possible memory leak?

[saul-jb]

I have found that there seems to be far more memory allocation in Helia than what is expected when persistent stores are used, the allocation seems to be about equal with the import size so I think it is not freeing up buffer memory somewhere.

I have found 2 cases where this happens - importing local data into Helia & fetching remote data into Helia. The following examples show this first case.

This allocates excess memory:

const helia = await createHelia({
  blockstore: new FsBlockstore('...'),
  datastore: new FsDatastore('...')
})
const ufs = unixfs(helia)

await all(ufs.addAll(globSource('...', '*')))

This does not allocate excess memory:

const ufs = unixfs({ blockstore: new FsBlockstore('...') })

await all(ufs.addAll(globSource('...', '*')))

Is this a mistake on my part, an expected result or a memory leak?

To make it easy to replicate I have made a repo that illustrates these examples (at least on my machine): https://github.com/saul-jb/helia-memory-leak

I have tested it with both node v18.18.2 & v20.7.0.

As far as I can tell from my own debugging so far the problem lies somewhere in Helia's BlockStorage.