yinbow
commented
3 months ago If your data scale can not be enlarged, maybe you can try: (1) A higher dropout rate helps prevent overfitting by ensuring the network doesn’t rely too much on any individual node. You can change it in the config file https://github.com/VCIP-RGBD/DFormer/blob/935944de7756ffaac8726d5bfd43a87af42dd6f1/local_configs/NYUDepthv2/DFormer_Large.py#L25 (2) Reduce Model Complexity: You may choose a small model, i.e., DFormer-B instead of -L. (3) You mentioned monitoring and early stopping: Our framework can monitor the mIoU at the validation set and save the best one. It has the same effect as the early stopping The current framework does not support the following (4)-(6). Maybe you need to implement them. (4) More Data Augmentation: You can refer to https://github.com/VCIP-RGBD/RGBD-Pretrain/blob/main/data/auto_augment.py for more data augmentation for the RGBD data. (5) Cross-Validation: Use cross-validation to ensure that the model’s performance is consistent across different subsets of the data. (6) Ensemble Methods: Combine predictions from multiple models to reduce variance and improve generalization. But this will sacrifice the model's efficiency.
johanna27045
While setting up your neural network to on it with own data, I was wondering if you have any tactics to prevent overfitting used in your code. Is there some kind of monitoring and early stopping?