man-group / ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
http://arcticdb.io
Other
1.23k stars 79 forks source link

Reduce memory usage during conda build #1649

Closed muhammadhamzasajjad closed 1 week ago

muhammadhamzasajjad commented 1 week ago

Add more translation units for binary operators to reduce memory usage.

The build on conda was failing in arcticdb-feedstock as it was using a lot of memory. The feedstock runner only have 6.7GB of RAM. Splitting the binary operators in multiple translation units reduces memory usage and fixes the issue.

What does this implement or fix?

Any other comments?

Checklist

Checklist for code changes... - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?