Open kevinningthu opened 3 months ago
def get_chunksize(self): return min(25_000, 1 + len(self) // Run().nranks)
Seems like it is set to 25_000 when the dataset is very large.
How to set the parameter chunk_size?
Thanks!
def get_chunksize(self): return min(25_000, 1 + len(self) // Run().nranks)
Seems like it is set to 25_000 when the dataset is very large.
How to set the parameter chunk_size?
Thanks!