Performance on Semantic3D and S3DIS

[zhulf0804]

Semantic3D

I submitted Semantic3D results and got the following results:

• semantic-8: 0.690
• reduced-8: 0.758

S3DIS

I got the following results and much lower than the paper results(70.0 mIoU):

• eval accuracy: 0.848214404296875
• mean IOU:0.5890085112691167
• Mean IoU = 58.9%

  --------------------------------------------------------------------------------------
  58.90 | 90.67 96.72 77.22  0.00 21.03 53.60 35.11 72.84 84.13 54.55 67.89 63.17 48.77
  --------------------------------------------------------------------------------------

  Best m_IoU is: 59.315

[yoxu515]

Hi, lifa, I am training it from scratch on S3DIS and got approximately the same results as yours. Specifically, 30 epochs now (still training), best mIoU 58.388, best eval acc 84.5.

Could you tell me how many epochs/iterations you trained?

Another thing needed to be aware is that the 70 mIoU in the paper is obtained by 6 fold cross validation. I guess you only evaluated one model on Area 5.

[zhulf0804]

Hi, lifa, I am training it from scratch on S3DIS and got approximately the same results as yours. Specifically, 30 epochs now (still training), best mIoU 58.388, best eval acc 84.5.

Could you tell me how many epochs/iterations you trained?

Another thing needed to be aware is that the 70 mIoU in the paper is obtained by 6 fold cross validation. I guess you only evaluated one model on Area 5.

Hi, yoxu, I trained for 100 epoches as the default setting.

Thank you for your suggestions, i'll verify the method to evaluate on S3DIS dataset.

[meidachen]

Hi @zhulf0804 , I got a similar miou on area 5 as yours, could you please kindly let me know if you are getting the same result (70%) miou for 6 fold cross-validation using the same setup as you did when you got 60% on area 5?

[hhhhsmx]

嗨，利法，我正在S3DIS上从头开始训练它，并得到了与你大致相同的结果。具体来说，现在有30个时代（仍在训练），最佳mIoU 58.388，最佳评估为84.5。 你能告诉我你训练了多少个纪元/迭代吗？ 需要注意的另一件事是，论文中的70 mIoU是通过6倍交叉验证获得的。我猜你只在5区评估了一个模型。

嗨，yoxu，我训练了100个epoches作为默认设置。

感谢您的建议，我将验证在S3DIS数据集上评估的方法。

hello。How do I make sure the training takes place on the GPU? My model training is very slow, taking about 3.4 hours per epoch。thank