search

uber / petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.78k stars 285 forks source link

Added last_row_consumed property to WeightedSamplingReader #554

Closed selitvin closed 4 years ago

selitvin commented 4 years ago

Without this property, make_petastorm_dataset was crashing if WeightedSamplingReader reader was passed to it.

codecov[bot] commented 4 years ago

Codecov Report

Merging #554 into master will decrease coverage by 0.01%. The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #554      +/-   ##
==========================================
- Coverage   86.10%   86.09%   -0.02%     
==========================================
  Files          87       87              
  Lines        4967     4969       +2     
  Branches      791      792       +1     
==========================================
+ Hits         4277     4278       +1     
- Misses        562      563       +1     
  Partials      128      128
Impacted Files Coverage Δ
petastorm/weighted_sampling_reader.py 85.71% <50.00%> (-1.79%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0209a73...4379419. Read the comment docs.