could not reproduce the claimed result

[AnarchistKnight]

As pretrained weights are not provided so far, I downloaded 3D-FRONT and trained the model following your instructions on both bedroom and livingroom dataset. I changed the feature_extractor in the config file from pointnet_simple to resnet18 as below. Besides, I only changed some paths and dirs.

#feature_extractor:
#    name: "pointnet_simple"
#    feat_units: [4, 64, 64, 512, 64]

feature_extractor:
    name: "resnet18"
    feature_size: 64
    freeze_bn: true
    input_channels: 1

The loss for livingroom finally converges at around 0.039. Though I adjusted the learning rate to 0.00001, there is no substantial reduce in the loss even after hundreds of epochs. So is the loss for bedroom, which finally oscillated about 0.028.

I rendered the test sample and they did not look good quality. Can you explain this or figure out if something I did wrong? Some of the test results are shown below.

[SiyiHu]

Your modification to the config file is the correct way to switch from PointNet to ResNet-18 floor plan feature extractor. We have not tried training rate that low. We also did not do much parameter tuning for models using ResNet-18, since this is not our key contribution. Based on experience training DiffuScene models (which uses ResNet-18) and ours, I would guess you need at least 60-65k epochs (if you keep all other parameters in the config file). You can also try 100k epochs. The validation loss overfit might be pretty big, but the final model should be able to generate reasonable layouts.

[AnarchistKnight]

The loss went down to around 0.011 for bedroom scenes after 18k epochs. The evaluation results on testset are very close to that claimed in paper. While, some pieces of furniture are not correctly oriented. Do you recomment me train more epochs or revise the loss function?

[SiyiHu]

I have not run evaluations around 18k epochs. For bedroom, I am pretty sure there are non-trivial improvements from 30k to 50k. If furniture is consistently off by 180 deg (which does happen to some of our runs), you can try removing "cosin_angle" in the config file.

[AnarchistKnight]

The loss for bedroom scenes went down to around 0.0088 after 45k epochs. The orientation problem remains, and I will try removing cosine angle then. For livingrooms, I trained 100k epochs and the performance on testset looks good.