:sparkles: Implement len function for XbatcherSlicerIterDataPipe - Githubissues

weiji14 / zen3geo

The 🌏 data science library you've been waiting for~

http://zen3geo.rtfd.io/en/latest/walkthrough.html

GNU Lesser General Public License v3.0

77 stars 3 forks source link

:sparkles: Implement len function for XbatcherSlicerIterDataPipe #75

Closed weiji14 closed 1 year ago

weiji14 commented 1 year ago

Allow the number of chips from XbatcherSlicer datapipes to be calculated directly using len()!

Not previously implemented (see https://github.com/weiji14/zen3geo/pull/22#discussion_r913310227) because it was inefficient in xbatcher, but xbatcher v0.2.0 has implemented lazy batch generation (see https://github.com/xarray-contrib/xbatcher/pull/112) so this should be performant enough now.

TODO:

[x] Implement __len__ in XbatcherSlicer
[x] Update tutorials that previously used list comprehension, to instead call len(dp_xbatcher) directly.

References:

https://pytorch.org/data/0.5/tutorial.html#length