bancaditalia / black-it

Black-box abm calibration kit by the Bank of Italy
https://bancaditalia.github.io/black-it/
GNU Affero General Public License v3.0
43 stars 2 forks source link

Use tables to append to series_samp array #27

Closed AldoGl closed 1 year ago

AldoGl commented 1 year ago

Proposed changes

Use the .h5 format to store and dynamically update the series_samp array. This PR gives a huge speedup in the update of large arrays during a calibration. Work by @marcofavorito

Types of changes

What types of changes does your code introduce? Put an x in the boxes that apply

Checklist

Put an x in the boxes that apply.

codecov-commenter commented 1 year ago

Codecov Report

Merging #27 (00eee4a) into main (2cba7c8) will increase coverage by 0.11%. The diff coverage is 96.00%.

Additional details and impacted files [![Impacted file tree graph](https://codecov.io/gh/bancaditalia/black-it/pull/27/graphs/tree.svg?width=650&height=150&src=pr&token=FwbdPgBFTD&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)](https://codecov.io/gh/bancaditalia/black-it/pull/27?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) ```diff @@ Coverage Diff @@ ## main #27 +/- ## ========================================== + Coverage 97.60% 97.72% +0.11% ========================================== Files 23 23 Lines 1213 1232 +19 ========================================== + Hits 1184 1204 +20 + Misses 29 28 -1 ``` | [Impacted Files](https://codecov.io/gh/bancaditalia/black-it/pull/27?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [black\_it/calibrator.py](https://codecov.io/gh/bancaditalia/black-it/pull/27/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-YmxhY2tfaXQvY2FsaWJyYXRvci5weQ==) | `97.48% <50.00%> (+0.01%)` | :arrow_up: | | [black\_it/samplers/gaussian\_process.py](https://codecov.io/gh/bancaditalia/black-it/pull/27/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-YmxhY2tfaXQvc2FtcGxlcnMvZ2F1c3NpYW5fcHJvY2Vzcy5weQ==) | `98.80% <100.00%> (+1.21%)` | :arrow_up: | | [black\_it/utils/json\_pandas\_checkpointing.py](https://codecov.io/gh/bancaditalia/black-it/pull/27/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-YmxhY2tfaXQvdXRpbHMvanNvbl9wYW5kYXNfY2hlY2twb2ludGluZy5weQ==) | `100.00% <100.00%> (ø)` | |
AldoGl commented 1 year ago

I will merge this soon as this PR was reviewed already by me and Marco