search

cuiaiyu / dressing-in-order

(ICCV'21) Official code of "Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing" by Aiyu Cui, Daniel McKee and Svetlana Lazebnik
https://cuiaiyu.github.io/dressing-in-order
Other
513 stars 127 forks source link

got NoneType during running the demo.ipynb #29

Closed Arvohy closed 2 years ago

Arvohy commented 2 years ago

Hi Aiyu Cui, When I was running the pose transfer part of demo.ipynb file, I found something wrong with the pose tensor. (I finished the training process and no error occured.)

Traceback (most recent call last):
  File "demo.py", line 158, in <module>
    plot_img(pimg, gimgs, oimgs, gen_img, pose)
  File "demo.py", line 86, in plot_img
    if pose != None:
TypeError: ne() received an invalid combination of arguments - got (NoneType), but expected one of:
 * (Tensor other)
      didn't match because some of the arguments have invalid types: (NoneType)
 * (Number other)
      didn't match because some of the arguments have invalid types: (NoneType)

Then I printed the 'pose', and found it was mostly invalid.

tensor([[[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         ...,
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]],

        [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         ...,
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]],

        [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         ...,
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]],

        ...,

        [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 9.8091e-45,
          1.4013e-45, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 2.6625e-44,
          2.8026e-45, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 7.0065e-44,
          8.4078e-45, 1.4013e-45],
         ...,
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]],

        [[4.8051e-42, 3.6002e-41, 2.6235e-40,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.4000e-41, 1.0490e-40, 7.6443e-40,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [3.9678e-41, 2.9728e-40, 2.1663e-39,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         ...,
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]],

        [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 5.6037e-37,
          8.5935e-38, 1.2818e-38],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 1.6328e-36,
          2.5040e-37, 3.7347e-38],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 4.6272e-36,
          7.0960e-37, 1.0584e-37],
         ...,
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]]], device='cuda:0')

Sorry for distrubing you, and look forward to your suggestion.

cuiaiyu commented 2 years ago

Please check if the pose data prepared correctly first.

Arvohy commented 2 years ago

Hi, I checked the dataroot which was the same as you described in README.

dataroot

Then I printed the keypoints and pose in the function def _load_kpt(self, name): ( in the deepfashion_datasets.py file).

keypoints_y    [41, 87, 85, 146, 205, 90, 156, 216, 210, -1, ...
keypoints_x    [103, 87, 45, 26, 36, 129, 137, 132, 61, -1, -...
Name: fashionMENSweatersid0000085901_1front.jpg, dtype: object
[[ 41 103]
 [ 87  87]
 [ 85  45]
 [146  26]
 [205  36]
 [ 90 129]
 [156 137]
 [216 132]
 [210  61]
 [ -1  -1]
 [ -1  -1]
 [215 107]
 [ -1  -1]
 [ -1  -1]
 [ 32  94]
 [ 32 109]
 [ 34  75]
 [ -1  -1]]
[[[0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   1.2529010e-41 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   9.9231549e-41 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   7.6442933e-40 0.0000000e+00]
  ...
  [2.0150111e-40 0.0000000e+00 0.0000000e+00 ... 2.1813527e-32
   0.0000000e+00 0.0000000e+00]
  [2.8430945e-41 0.0000000e+00 0.0000000e+00 ... 3.5363172e-33
   0.0000000e+00 0.0000000e+00]
  [3.9012149e-42 0.0000000e+00 0.0000000e+00 ... 5.5758720e-34
   0.0000000e+00 0.0000000e+00]]

 [[0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   3.1771640e-41 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   2.5164518e-40 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   1.9385297e-39 0.0000000e+00]
  ...
  [6.2066732e-40 0.0000000e+00 0.0000000e+00 ... 5.2327930e-32
   0.0000000e+00 0.0000000e+00]
  [8.7572746e-41 0.0000000e+00 0.0000000e+00 ... 8.4831842e-33
   0.0000000e+00 0.0000000e+00]
  [1.2017536e-41 0.0000000e+00 0.0000000e+00 ... 1.3375822e-33
   0.0000000e+00 0.0000000e+00]]

 [[0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   7.8363413e-41 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   6.2066732e-40 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   4.7812766e-39 0.0000000e+00]
  ...
  [1.8594166e-39 0.0000000e+00 0.0000000e+00 ... 1.2208928e-31
   0.0000000e+00 0.0000000e+00]
  [2.6235110e-40 0.0000000e+00 0.0000000e+00 ... 1.9792599e-32
   0.0000000e+00 0.0000000e+00]
  [3.6002160e-41 0.0000000e+00 0.0000000e+00 ... 3.1207891e-33
   0.0000000e+00 0.0000000e+00]]

 ...

 [[0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  ...
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]]

 [[0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  ...
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]]

 [[0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  ...
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]
  [0.0000000e+00 0.0000000e+00 0.0000000e+00 ... 0.0000000e+00
   0.0000000e+00 0.0000000e+00]]]

Is there something wrong with pose calculation function? Or the data I put isn't correct.

cuiaiyu commented 2 years ago

so it is a heatmap in 18x256x176? what makes it "most invalid"?

Arvohy commented 2 years ago

Yeah, it's a heatmap in 18×256×176. Then I commented out the line if pose != None:, and it worked. （ I still don't know why it got Nonetype. ) Thanks for your help and patience.🌹