amzn / pecos

PECOS - Prediction for Enormous and Correlated Spaces
https://libpecos.org/
Apache License 2.0
516 stars 105 forks source link

Add mmap value store for multi-thread fast retrieval of 2D sub-matrix #257

Closed weiliw-amz closed 1 year ago

weiliw-amz commented 1 year ago

Issue #, if available: N/A

Description of changes: Added Numerical (float32) C with Python binding mmap value store (2D matrix).

Accelerates sub-matrix retrieval given rows and cols indices, comparing to for-loop np.array.item(i, j). For value store of 66M x 100, time cost of retrieving (Python binding) random sub-matrix 500x50, comparing to numpy for-loop is 0.09ms (4 threads) vs. 5ms

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.