search

mila-iqia / fuel

A data pipeline framework for machine learning
MIT License
867 stars 268 forks source link

Make get_data optional in AbstractDataStream. #368

Closed dwf closed 8 years ago

dwf commented 8 years ago

I've noticed that it's often a lot more expedient to stub out get_data if all you care is epoch iterators, so we might as well make it an optional implement.

rizar commented 8 years ago

I understand you right, your point is that very often the only method of a data stream that is actually called is get_epoch_iterator. Thus, it makes sense to make all other methods optional.

If my understanding is correct, I approve this change. I guess you still have to rebase.

dwf commented 8 years ago

@rizar I made everything but get_epoch_iterator optional. Waiting on a Theano fix to Theano/Theano#5088 to un-break our tests.