fastai / course-v3

The 3rd edition of course.fast.ai
https://course.fast.ai/
Apache License 2.0
4.91k stars 3.57k forks source link

It's not possible to apply those transforms to your dataset: #511

Open guixermo opened 4 years ago

guixermo commented 4 years ago

getting error while trying to transform the images to 224 size

`--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) /usr/local/lib/python3.6/dist-packages/fastai/data_block.py in _check_kwargs(ds, tfms, kwargs) 593 x = ds[0] --> 594 try: x.apply_tfms(tfms, kwargs) 595 except Exception as e:

11 frames /usr/local/lib/python3.6/dist-packages/fastai/vision/image.py in apply_tfms(self, tfms, do_resolve, xtra, size, resize_method, mult, padding_mode, mode, remove_out) 122 x = tfm(x, size=_get_crop_target(size,mult=mult), padding_mode=padding_mode) --> 123 else: x = tfm(x) 124 return x.refresh()

/usr/local/lib/python3.6/dist-packages/fastai/vision/image.py in call(self, x, *args, *kwargs) 523 "Randomly execute our tfm on x." --> 524 return self.tfm(x, args, {self.resolved, **kwargs}) if self.do_run else x 525

/usr/local/lib/python3.6/dist-packages/fastai/vision/image.py in call(self, p, is_random, use_on_y, *args, *kwargs) 469 "Calc now if args passed; else create a transform called prob p if random." --> 470 if args: return self.calc(args, **kwargs) 471 else: return RandTransform(self, kwargs=kwargs, is_random=is_random, use_on_y=use_on_y, p=p)

/usr/local/lib/python3.6/dist-packages/fastai/vision/image.py in calc(self, x, *args, kwargs) 474 "Apply to image x, wrapping it if necessary." --> 475 if self._wrap: return getattr(x, self._wrap)(self.func, *args, *kwargs) 476 else: return self.func(x, args, kwargs)

/usr/local/lib/python3.6/dist-packages/fastai/vision/image.py in affine(self, func, *args, *kwargs) 182 m = tensor(func(args, **kwargs)).to(self.device) --> 183 self.affine_mat = self.affine_mat @ m 184 return self

RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #3 'mat2' in call to _th_addmm_out

During handling of the above exception, another exception occurred:

Exception Traceback (most recent call last)

in () ----> 1 data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs).normalize(imagenet_stats) 2 /usr/local/lib/python3.6/dist-packages/fastai/vision/data.py in from_name_re(cls, path, fnames, pat, valid_pct, **kwargs) 156 assert res,f'Failed to find "{pat}" in "{fn}"' 157 return res.group(1) --> 158 return cls.from_name_func(path, fnames, _get_label, valid_pct=valid_pct, **kwargs) 159 160 @staticmethod /usr/local/lib/python3.6/dist-packages/fastai/vision/data.py in from_name_func(cls, path, fnames, label_func, valid_pct, seed, **kwargs) 145 "Create from list of `fnames` in `path` with `label_func`." 146 src = ImageList(fnames, path=path).split_by_rand_pct(valid_pct, seed) --> 147 return cls.create_from_ll(src.label_from_func(label_func), **kwargs) 148 149 @classmethod /usr/local/lib/python3.6/dist-packages/fastai/vision/data.py in create_from_ll(cls, lls, bs, val_bs, ds_tfms, num_workers, dl_tfms, device, test, collate_fn, size, no_check, resize_method, mult, padding_mode, mode, tfm_y) 95 "Create an `ImageDataBunch` from `LabelLists` `lls` with potential `ds_tfms`." 96 lls = lls.transform(tfms=ds_tfms, size=size, resize_method=resize_method, mult=mult, padding_mode=padding_mode, ---> 97 mode=mode, tfm_y=tfm_y) 98 if test is not None: lls.add_test_folder(test) 99 return lls.databunch(bs=bs, val_bs=val_bs, dl_tfms=dl_tfms, num_workers=num_workers, collate_fn=collate_fn, /usr/local/lib/python3.6/dist-packages/fastai/data_block.py in transform(self, tfms, **kwargs) 503 if not tfms: tfms=(None,None) 504 assert is_listy(tfms) and len(tfms) == 2, "Please pass a list of two lists of transforms (train and valid)." --> 505 self.train.transform(tfms[0], **kwargs) 506 self.valid.transform(tfms[1], **kwargs) 507 if self.test: self.test.transform(tfms[1], **kwargs) /usr/local/lib/python3.6/dist-packages/fastai/data_block.py in transform(self, tfms, tfm_y, **kwargs) 722 def transform(self, tfms:TfmList, tfm_y:bool=None, **kwargs): 723 "Set the `tfms` and `tfm_y` value to be applied to the inputs and targets." --> 724 _check_kwargs(self.x, tfms, **kwargs) 725 if tfm_y is None: tfm_y = self.tfm_y 726 tfms_y = None if tfms is None else list(filter(lambda t: getattr(t, 'use_on_y', True), listify(tfms))) /usr/local/lib/python3.6/dist-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs) 594 try: x.apply_tfms(tfms, **kwargs) 595 except Exception as e: --> 596 raise Exception(f"It's not possible to apply those transforms to your dataset:\n {e}") 597 598 class LabelList(Dataset): Exception: It's not possible to apply those transforms to your dataset: Expected object of scalar type Float but got scalar type Double for argument #3 'mat2' in call to _th_addmm_out`
ghost commented 4 years ago

I had the same issue when I run it on local. Then I fixed it by running it on Sagemaker. The differences I found:

  1. pytorch version was 1.5 on local, but 1.4 on cloud. You can check version:

    import torch
    print(torch.__version__)
  2. I did not use GPU on local, but "by default it will provision a SageMaker notebook instance of type ml.p2.xlarge which has the Nvidia K80 GPU and 50 GB of EBS disk space".

You can check if your pytorch is using GPU:

torch.cuda.is_available()

I will suggest you try to run the lesson on cloud since the environment is set up for you.

guixermo commented 4 years ago

I had the same issue when I run it on local. Then I fixed it by running it on Sagemaker. The differences I found:

  1. pytorch version was 1.5 on local, but 1.4 on cloud. You can check version:
import torch
print(torch.__version__)
  1. I did not use GPU on local, but "by default it will provision a SageMaker notebook instance of type ml.p2.xlarge which has the Nvidia K80 GPU and 50 GB of EBS disk space".

You can check if your pytorch is using GPU:

torch.cuda.is_available()

I will suggest you try to run the lesson on cloud since the environment is set up for you.

Thank you for your advice. I'm working on colab and the pytorch version was 1.5. I downgraded to 1.4.0 as weell as torchvision to 0.5.0. Now it works.

You can look your version with: import torch print(torch.version)

I downgraded with: !pip install torch==1.4.0 Then restart and installed torchvision with !pip install torchvision==0.5.0

wenzhenghe7 commented 4 years ago

I had the same issue when I run it on local. Then I fixed it by running it on Sagemaker. The differences I found:

  1. pytorch version was 1.5 on local, but 1.4 on cloud. You can check version:
import torch
print(torch.__version__)
  1. I did not use GPU on local, but "by default it will provision a SageMaker notebook instance of type ml.p2.xlarge which has the Nvidia K80 GPU and 50 GB of EBS disk space".

You can check if your pytorch is using GPU:

torch.cuda.is_available()

I will suggest you try to run the lesson on cloud since the environment is set up for you.

Thank you for your advice. I'm working on colab and the pytorch version was 1.5. I downgraded to 1.4.0 as weell as torchvision to 0.5.0. Now it works.

You can look your version with: import torch print(torch.version)

I downgraded with: pip install torch==1.4.0 Then restart and installed torchvision with pip install torchvision==0.5.0

This works for me too. Thanks!

someshwarrc commented 4 years ago

I received the error with pyTorch on my local system and solved the issue by using (float) with the train data after looking it up on StackOverFlow. Is it a requirement in the latest pyTorch version ?

guixermo commented 4 years ago

I received the error with pyTorch on my local system and solved the issue by using (float) with the train data after looking it up on StackOverFlow. Is it a requirement in the latest pyTorch version ?

How did you solve it? I tried using float but it didnt work for me so i created this issue.

someshwarrc commented 4 years ago

With only torch on colab to train a model I used type conversion while passing the feature data tensor to the model. I couldn't solve the issue with fast.ai with the latest version as well. So I think this is a requirement in the latest version

guixermo commented 4 years ago

Great, thanks. I've chaged from colab to local and with 1.5.0 it only gives me a warning but it works so maybe is just colab.

s-bernard commented 4 years ago

@guixermo @someshwarrc Can you give us some code? How would you modify the course notebook?

guixermo commented 4 years ago

@guixermo @someshwarrc Can you give us some code? How would you modify the course notebook?

You dont have to modify it, you just need to downgrade the pytorch and torchvision versions with the comands i wrote above.

s-bernard commented 4 years ago

Well that’s not what I call a solution.

Thanks for your reply.