search

asuiu / pyxtension

Pure Python extensions library that includes Scala-like streams, Json with attribute access syntax, and other common use stuff
MIT License
43 stars 1 forks source link

Implement split_in_batches #4

Closed asuiu closed 4 years ago

asuiu commented 6 years ago

Example:

def split_in_batches(self, itr: Iterable[T], batch_max_size: int = BATCH_SIZE) -> Generator[slist[T], None, None]:
        batch = slist()
        for el in itr:
            batch.append(el)
            if batch.size() == batch_max_size:
                yield batch
                batch = slist()
        if batch.size():
            yield batch
asuiu commented 4 years ago

Implemented in stream.batch(). Covered by tests & used in prod code.