search

ipfs / js-ipfs-repo

Implementation of the IPFS Repo spec in JavaScript
https://github.com/ipfs/specs/tree/master/repo
Other
79 stars 50 forks source link

fix: OOM on large DAGs #410

Closed achingbrain closed 2 years ago

achingbrain commented 2 years ago

Storing a set of seen CIDs to short-cut DAG traversal while ensuring we have all the blocks in a DAG in the blockstore can cause OOMs for very large DAGs so replace the unbounded Set with a LRU cache with a configurable maximum size.

BigLep commented 2 years ago

How easy is it to add a regression test?

achingbrain commented 2 years ago

I'm removing an optimisation that doesn't appear to have had a specific test for it - I should really be deleting a test here..

The optimisation is now retained but we use a LRU cache instead of an unbounded Set.

github-actions[bot] commented 2 years ago

:tada: This PR is included in version ipfs-repo-migrations-v13.0.3 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

github-actions[bot] commented 2 years ago

:tada: This PR is included in version ipfs-repo-v15.0.3 :tada:

The release is available on:

Your semantic-release bot :package::rocket: