Is file buffering maximum RAM estimation off?

catalystneuro / dandi_s3_log_parser

S3 log parsing for the DANDI Archive.

BSD 3-Clause "New" or "Revised" License

1 stars 2 forks source link

Is file buffering maximum RAM estimation off? #15

Closed CodyCBakerPhD closed 3 months ago

CodyCBakerPhD commented 3 months ago

File buffering maximum RAM usage seems 3x higher in practice (on Drogon) than intended. I thought I had accounted for the copy/decode mechanisms, but maybe I need to go through those steps again

CodyCBakerPhD commented 3 months ago

I think it is accurate, it's just that the original choice of variable name was misleading

There is more RAM used in total due to overhead that is hard to account for

So changed name in #23