search

uber / petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.76k stars 281 forks source link

Update CI to use latest versions of pyarrow and numpy. Drop pyarrow 4 test config. #778

Open selitvin opened 1 year ago

selitvin commented 1 year ago

Does not seem to be anything special in pyarrow 4, so dropped it.

codecov[bot] commented 1 year ago

Codecov Report

Base: 86.30% // Head: 86.03% // Decreases project coverage by -0.26% :warning:

Coverage data is based on head (ee8cbb0) compared to base (4e80f2b). Patch has no changes to coverable lines.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #778 +/- ## ========================================== - Coverage 86.30% 86.03% -0.27% ========================================== Files 85 84 -1 Lines 5095 4990 -105 Branches 786 785 -1 ========================================== - Hits 4397 4293 -104 + Misses 559 558 -1 Partials 139 139 ``` | [Impacted Files](https://codecov.io/gh/uber/petastorm/pull/778?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber) | Coverage Δ | | |---|---|---| | [petastorm/etl/\_\_init\_\_.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL2V0bC9fX2luaXRfXy5weQ==) | `66.66% <0.00%> (-9.53%)` | :arrow_down: | | [petastorm/etl/rowgroup\_indexers.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL2V0bC9yb3dncm91cF9pbmRleGVycy5weQ==) | `42.18% <0.00%> (-4.96%)` | :arrow_down: | | [...s/imagenet/tests/test\_generate\_imagenet\_dataset.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-ZXhhbXBsZXMvaW1hZ2VuZXQvdGVzdHMvdGVzdF9nZW5lcmF0ZV9pbWFnZW5ldF9kYXRhc2V0LnB5) | `86.84% <0.00%> (-0.97%)` | :arrow_down: | | [petastorm/workers\_pool/worker\_base.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL3dvcmtlcnNfcG9vbC93b3JrZXJfYmFzZS5weQ==) | `90.00% <0.00%> (-0.91%)` | :arrow_down: | | [petastorm/codecs.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL2NvZGVjcy5weQ==) | `78.74% <0.00%> (-0.81%)` | :arrow_down: | | [petastorm/reader\_impl/shuffling\_buffer.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL3JlYWRlcl9pbXBsL3NodWZmbGluZ19idWZmZXIucHk=) | `93.65% <0.00%> (-0.72%)` | :arrow_down: | | [petastorm/predicates.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL3ByZWRpY2F0ZXMucHk=) | `75.82% <0.00%> (-0.52%)` | :arrow_down: | | [petastorm/ngram.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL25ncmFtLnB5) | `87.23% <0.00%> (-0.40%)` | :arrow_down: | | [examples/mnist/tests/test\_pytorch\_mnist.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-ZXhhbXBsZXMvbW5pc3QvdGVzdHMvdGVzdF9weXRvcmNoX21uaXN0LnB5) | `91.11% <0.00%> (-0.38%)` | :arrow_down: | | [petastorm/workers\_pool/ventilator.py](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber#diff-cGV0YXN0b3JtL3dvcmtlcnNfcG9vbC92ZW50aWxhdG9yLnB5) | `92.95% <0.00%> (-0.38%)` | :arrow_down: | | ... and [22 more](https://codecov.io/gh/uber/petastorm/pull/778/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=uber)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

CLAassistant commented 1 year ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

Yevgeni Litvin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.