Type error trigered by generate_scenes.py

[fuyb1992]

Your ideas about using transformers in this project is great, and I'm trying reproducing your project, but I got some obstacles. Atfter traning, I run the following code:

python generate_scenes.py ../config/bedrooms_config.yaml /data/3D-generate /data/3D-FUTURE-pickle/threed_future_model_bedroom.pkl ../demo/floor_plan_texture_images --weight_file /data/weights/8A9ENPCMK/model_00100

and get erros:

Running code on cuda:0
Loaded 2354 3D-FUTURE models
Applying no_filtering filtering
Loaded 162 scenes with 21 object types:
Loading weight file from /data/weights/8A9ENPCMK/model_00100
0 / 10: Using the 147 floor plan of scene MasterBedroom-109561
Traceback (most recent call last):
  File "generate_scenes.py", line 266, in <module>
    main(sys.argv[1:])
  File "generate_scenes.py", line 192, in main
    bbox_params = network.generate_boxes(room_mask=room_mask)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/codes/ATISS/scene_synthesis/networks/autoregressive_transformer.py", line 227, in generate_boxes
    box = self.autoregressive_decode(boxes, room_mask=room_mask)
  File "/codes/ATISS/scene_synthesis/networks/autoregressive_transformer.py", line 202, in autoregressive_decode
    F = self._encode(boxes, room_mask)
  File "/codes/ATISS/scene_synthesis/networks/autoregressive_transformer.py", line 165, in _encode
    start_symbol_f = self.start_symbol_features(B, room_mask)
  File "/codes/ATISS/scene_synthesis/networks/autoregressive_transformer.py", line 93, in start_symbol_features
    room_layout_f = self.fc_room_f(self.feature_extractor(room_mask))
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/codes/ATISS/scene_synthesis/networks/feature_extractors.py", line 24, in forward
    return self._feature_extractor(X)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torchvision/models/resnet.py", line 220, in forward
    return self._forward_impl(x)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torchvision/models/resnet.py", line 203, in _forward_impl
    x = self.conv1(x)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 419, in forward
    return self._conv_forward(input, self.weight)
  File "/usr/local/anaconda3/envs/atiss/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 415, in _conv_forward
    return F.conv2d(input, weight, self.bias, self.stride,
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

envs: cuda 10

[fuyb1992]

Problem solved by change codes: generate_scenes.py

room_mask = room_mask.to(device) # new add
bbox_params = network.generate_boxes(room_mask=room_mask, device=device) # change
boxes = dataset.post_process(bbox_params)

autoregressive_transformer.py

  def generate_boxes(self, room_mask, max_boxes=32, device="cpu"):
      boxes = self.start_symbol(device)
      for i in range(max_boxes):
          box = self.autoregressive_decode(boxes, room_mask=room_mask)

          for k in box.keys():
              boxes[k] = torch.cat([boxes[k], box[k]], dim=1)

          # Check if we have the end symbol
          if box["class_labels"][0, 0, -1] == 1:
              break

      return {
          "class_labels": boxes["class_labels"].to('cpu'),  # change
          "translations": boxes["translations"].to('cpu'),  # change
          "sizes": boxes["sizes"].to('cpu'),  # change
          "angles": boxes["angles"].to('cpu')  # change
      }

[paschalidoud]

Hi @fuyb1992,

I was only running the 'generate_boxes.py script on CPU, so thank you very much for pointing out this issue when running our code in GPU. Based on your suggestion, I just committed a small fix that should resolve this issue.

Best, Despoina