fastai / course-v3

The 3rd edition of course.fast.ai
https://course.fast.ai/
Apache License 2.0
4.9k stars 3.55k forks source link

TypeError: intercept_args() got an unexpected keyword argument 'dl_tfms' #396

Open magic-lantern opened 5 years ago

magic-lantern commented 5 years ago

When running this code

class Seq2SeqDataBunch(TextDataBunch):
    "Create a `TextDataBunch` suitable for training an RNN classifier."
    @classmethod
    def create(cls, train_ds, valid_ds, test_ds=None, path:PathOrStr='.', bs:int=32, val_bs:int=None, pad_idx=1,
               pad_first=False, device:torch.device=None, no_check:bool=False, backwards:bool=False, **dl_kwargs) -> DataBunch:
        "Function that transform the `datasets` in a `DataBunch` for classification. Passes `**dl_kwargs` on to `DataLoader()`"
        datasets = cls._init_ds(train_ds, valid_ds, test_ds)
        val_bs = ifnone(val_bs, bs)
        collate_fn = partial(seq2seq_collate, pad_idx=pad_idx, pad_first=pad_first, backwards=backwards)
        train_sampler = SortishSampler(datasets[0].x, key=lambda t: len(datasets[0][t][0].data), bs=bs//2)
        train_dl = DataLoader(datasets[0], batch_size=bs, sampler=train_sampler, drop_last=True, **dl_kwargs)
        dataloaders = [train_dl]
        for ds in datasets[1:]:
            lengths = [len(t) for t in ds.x.items]
            sampler = SortSampler(ds.x, key=lengths.__getitem__)
            dataloaders.append(DataLoader(ds, batch_size=val_bs, sampler=sampler, **dl_kwargs))
        return cls(*dataloaders, path=path, device=device, collate_fn=collate_fn, no_check=no_check)

in the translation/sequence to sequence notebooks (https://github.com/fastai/course-v3/blob/master/nbs/dl2/translation.ipynb and https://github.com/fastai/course-v3/blob/master/nbs/dl2/translation_transformer.ipynb) the following error occurs:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-07789a0b3d53> in <module>
----> 1 data = src.databunch()

~/anaconda3/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in databunch(self, path, bs, val_bs, num_workers, dl_tfms, device, collate_fn, no_check, **kwargs)
    545         path = Path(ifnone(path, self.path))
    546         data = self.x._bunch.create(self.train, self.valid, test_ds=self.test, path=path, bs=bs, val_bs=val_bs,
--> 547                                     num_workers=num_workers, dl_tfms=dl_tfms, device=device, collate_fn=collate_fn, no_check=no_check, **kwargs)
    548         if getattr(self, 'normalize', False):#In case a normalization was serialized
    549             norm = self.normalize

<ipython-input-9-96e2696c16f2> in create(cls, train_ds, valid_ds, test_ds, path, bs, val_bs, pad_idx, pad_first, device, no_check, backwards, **dl_kwargs)
      9         collate_fn = partial(seq2seq_collate, pad_idx=pad_idx, pad_first=pad_first, backwards=backwards)
     10         train_sampler = SortishSampler(datasets[0].x, key=lambda t: len(datasets[0][t][0].data), bs=bs//2)
---> 11         train_dl = DataLoader(datasets[0], batch_size=bs, sampler=train_sampler, drop_last=True, **dl_kwargs)
     12         dataloaders = [train_dl]
     13         for ds in datasets[1:]:

TypeError: intercept_args() got an unexpected keyword argument 'dl_tfms'

It appears that at some point in the past, the DataBunch classes had a dl_tfms argument added, but the DataLoader does not accept that argument.

I'm not sure if a change is warranted in the based fastai library DataLoader, or just that client code should be updated.

magic-lantern commented 5 years ago

If client code should be updated, I'm happy to do a pull request to update these notebooks. It seems that just dropping the dl_tfms argument before calling the DataLoader works:

cp_args = dict(dl_kwargs)
del cp_args['dl_tfms']
train_dl = DataLoader(datasets[0], batch_size=bs, sampler=train_sampler, drop_last=True, **cp_args)
StephennFernandes commented 3 years ago

ive been using fastai 1.0.61, get the exact same error , do you know how to get around it ??

magic-lantern commented 3 years ago

@StephennFernandes - It's been several months since I reported this issue, but not sure if anything has changed in v1 of the library. Looks like according to https://github.com/fastai/fastai1/releases the most current release is 1.0.63.

You may have success with my previous comment about dropping the dl_tfms argument.

However, my recommendation would be to switch to v2 of the the fastai library if you can.