search

tntech-ngin / ndn-hydra

ndn-hydra: A federated data repository with five attributes: resiliency, scalability, usability, efficiency, and security.
https://hydra-repo.io/
Apache License 2.0
2 stars 1 forks source link

Memory exhaustion on the client node. #2

Closed 0rky closed 7 months ago

0rky commented 9 months ago

While trying to publish a file as a client, the file gets stored in the memory(RAM) and then offloaded into the network. If the file is larger than the RAM size, the program gets stuck. Memory thrashing starts to happen, I think.

I am able to recreate the issue:

ndn hydra version:0.3.15 create a 10Gb file and upload publish the file ndn-hydra-client insert -r /hydra -f /10gb_file -p 10gb_file

After a few minutes the program stopped responding. I monitored the client node's RAM utilization. It is almost full (client is using 8Gb of memory). Good news is I am able to fix this issue, by memory mapping the pages to the hard disk. But the same problem is there while fetching a file. If the file size is larger then the RAM size, memory gets exhausted. Not able to solve this problem.

Hope the issue gets resolved. happy to answer any further questions.

tntech-ngin commented 8 months ago

Please pull the updated code and test if the issue still exists.

tntech-ngin commented 7 months ago

This is fixed and merged. PR #14