search

allenai / ir_datasets

Provides a common interface to many IR ranking datasets.
https://ir-datasets.com/
Apache License 2.0
316 stars 42 forks source link

what is a "`streamer`" #232

Open MangoIV opened 1 year ago

MangoIV commented 1 year ago

It is not documented anywhere, what a streamer as used in classes like "RelativePath" is. It would be immensely helpful to have at least one tiny docstring explaining the use of this function.

seanmacavaney commented 1 year ago

Good call -- I think that #103 (and corresponding WIP branch) should help with this, e.g., by adding base classes and such.

Basically, a streamer is an object that provides a stream() method that returns an iostream that can be opened with a context manager, read in sequence, and closed by the context manager.