search

dreamcatcher-tech / dreamcatcher-stack

https://dreamcatcher.land
1 stars 0 forks source link

repo sometimes has too many open files on windows #161

Closed inverted-capital closed 1 year ago

inverted-capital commented 1 year ago

Needs to pipeline requests when doing a large number of walks.

Pipelining walks could also include doing batching with getMany() which will be more performant that individual requests.

inverted-capital commented 1 year ago

rebooting the service stopped this problem. If the problem returns, some limit on the number of open files should be set for safety, and some retry params used.

When the Pulse format transitions to the packed format, then this problem should be less common.

We may need to do some periodic maintenance on the KV store also, to compact the database possibly.