metadb-project / metadb

Metadb extends PostgreSQL with features to support analytics such as streaming data sources, data model transforms, and historical data
Apache License 2.0
8 stars 4 forks source link

Set the default value for "memlimit" in 75% of the current memory #76

Closed nazgaret closed 1 month ago

nazgaret commented 2 months ago

Update the default value for memlimit Update requirements cause the new version with concurrency work better with more CPU/Mem Update docs

sonarcloud[bot] commented 1 month ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

nassibnassar commented 1 month ago

Thanks for these proposed changes. Please clarify why the changes would be an improvement, since memlimit can already be set explicitly.

nazgaret commented 1 month ago

The improvement is achieved by setting the memory limit to 75% of the instance memory instead of setting it to 1 GiB, which may not be enough for parallel reading. In the case where this parameter is not specifically specified @nassibnassar

nassibnassar commented 1 month ago

Thank you for clarifying. There are two issues here: (1) the use of a percentage unit instead of GiB, and (2) that 1 GiB may be too low a default for concurrent consumers.

The percentage unit could make sense if Metadb is the only significant process running; but I think it is often hosted on a shared server with other Metadb instances or other processes. I believe it will be easier to manage if we keep the more explicit unit of GiB. However, I will check if the 1 GiB default value should be increased for concurrent consumers.

Closing.