fastai / fastai_dev

fast.ai early development experiments
Apache License 2.0
638 stars 350 forks source link

Can't setup batch transforms from TfmdDL #259

Closed slawekslex closed 4 years ago

slawekslex commented 4 years ago

The Transform.setup() is called before the TfmdDL super constructor here As the result the transform wont have access to the data. See for example:

class foo(Transform):
    def setup(self, items):
        n = len(items)
test_ds = DataSource(L(range(10)))
test_dl = TfmdDL(test_ds,bs=2,after_batch=[foo])
sgugger commented 4 years ago

What is your use case? I think they should be passed self.dataset for the setup but want to be sure that's what you need.

slawekslex commented 4 years ago

Getting the dataset sounds good.

sgugger commented 4 years ago

Just fixed this, the setup gets passed the full DataLoader in the end. Pipelines are setup in the logical order: after_item, before_batch and after_batch.