RuntimeError: unsupported operation: some elements of the input tensor and the written-to tensor refer to a single memory location. Please clone() the tensor before performing the operation.

[vivekvkashyap]

When using Dataset Condensation with Differentiable Siamese Augmentation network for training cifar 10 with ipc =1. I am getting an error.

Evaluation
model_train = ConvNet, model_eval = ConvNet, iteration = 0
DSA augmentation strategy: 
 color_crop_cutout_flip_scale_rotate
DSA augmentation parameters: 
 {'aug_mode': 'S', 'prob_flip': 0.5, 'ratio_scale': 1.2, 'ratio_rotate': 15.0, 'ratio_crop_pad': 0.125, 'ratio_cutout': 0.5, 'brightness': 1.0, 'saturation': 2.0, 'contrast': 0.5}
/home/ece/anaconda3/envs/distillation/lib/python3.9/site-packages/torch/nn/functional.py:4255: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn(
/home/ece/anaconda3/envs/distillation/lib/python3.9/site-packages/torch/nn/functional.py:4193: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn(
/home/ece/anaconda3/envs/distillation/lib/python3.9/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1646756402876/work/aten/src/ATen/native/TensorShape.cpp:2228.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
[2023-02-06 17:12:26] Evaluate_00: epoch = 1000 train time = 3 s train loss = 0.000725 train acc = 1.0000, test acc = 0.1447
[2023-02-06 17:12:31] Evaluate_01: epoch = 1000 train time = 3 s train loss = 0.003311 train acc = 1.0000, test acc = 0.1494
[2023-02-06 17:12:35] Evaluate_02: epoch = 1000 train time = 3 s train loss = 0.001446 train acc = 1.0000, test acc = 0.1440
[2023-02-06 17:12:40] Evaluate_03: epoch = 1000 train time = 3 s train loss = 0.000057 train acc = 1.0000, test acc = 0.1455
[2023-02-06 17:12:44] Evaluate_04: epoch = 1000 train time = 3 s train loss = 0.000042 train acc = 1.0000, test acc = 0.1444
Evaluate 5 random ConvNet, mean = 0.1456 std = 0.0020
-------------------------
Traceback (most recent call last):
  File "/mnt/8ec88946-510d-4eb2-925c-16cccd63436b/shubham/Piyush/ebm-distillation/baseline/DatasetCondensation/main_DM.py", line 215, in <module>
    main()
  File "/mnt/8ec88946-510d-4eb2-925c-16cccd63436b/shubham/Piyush/ebm-distillation/baseline/DatasetCondensation/main_DM.py", line 158, in main
    img_syn = DiffAugment(img_syn, args.dsa_strategy, seed=seed, param=args.dsa_param)
  File "/mnt/8ec88946-510d-4eb2-925c-16cccd63436b/shubham/Piyush/ebm-distillation/baseline/DatasetCondensation/utils.py", line 516, in DiffAugment
    x = f(x, param)
  File "/mnt/8ec88946-510d-4eb2-925c-16cccd63436b/shubham/Piyush/ebm-distillation/baseline/DatasetCondensation/utils.py", line 561, in rand_flip
    randf[:] = randf[0]
RuntimeError: unsupported operation: some elements of the input tensor and the written-to tensor refer to a single memory location. Please clone() the tensor before performing the operation.

[PatrickZH]

I guess it is a general problem that may be caused by the changing of some pytorch library version. Maybe you can specify these pytorch libraries listed in requirements.txt or fix it by searching some general tricks by Google.

[vivekvkashyap]

I was able to rectify the error using .clone() for the all the augmentation tricks.Thanks for the help.