Open zzyy410 opened 7 months ago
This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.
You can try fixing random seeds (eg. numpy.random.seed random.seed and torch.manual_seed
)to obtain consistent results.
thanks!
This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.
You can try fixing random seeds (eg.
numpy.random.seed random.seed and torch.manual_seed
)to obtain consistent results.
thanks!
This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position.
You can try fixing random seeds (eg.
numpy.random.seed random.seed and torch.manual_seed
)to obtain consistent results.
Hi again sorry to bother you, I've tried setting seed before training starts but still can't get consistent results for each target detection, where should I set the random seed? Is it easy to tell which file? It would be more appreciated if you can tell the exact location. Thank you for your time! Here's where I've tried to set random seeds:
class DetSolver(BaseSolver): def fit(self, ): print("Start training") self.train()
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
...... def val(self, ): self.eval()
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# dist.set_seed(42)
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
....... def train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module, data_loader: Iterable, optimizer: torch.optim.Optimizer, device: torch.device, epoch: int, max_norm: float = 0, **kwargs):
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# dist.set_seed(42)
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
....... def evaluate(model: torch.nn.Module, criterion: torch.nn.Module, postprocessors, data_loader, base_ds, device, output_dir):
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
# dist.set_seed(42)
np.random.seed(42)
random.seed(42)
torch.manual_seed(42)
....... if name == 'main': parser = argparse.ArgumentParser()
parser.add_argument('--config', '-c', type=str, default="/home/lw/rtdetr_pytorch/configs/rtdetr/rtdetr_r18vd_6x_sonar.yml")
parser.add_argument('--resume', '-r', type=str, default=False)
parser.add_argument('--tuning', '-t', type=str, default=False)
parser.add_argument('--test-only', action='store_true', default=False,)
parser.add_argument('--amp', action='store_true', default=False,)
args = parser.parse_args() np.random.seed(42) random.seed(42) torch.manual_seed(42) main(args)
You can add it into first line of main
function
def set_seed_and_config():
np.random.seed(42)
torch.manual_seed(42)
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = False
This phenomenon is normal. During the training process of the model, there are many random factors that can lead to the final convergence position. You can try fixing random seeds (eg.
numpy.random.seed random.seed and torch.manual_seed
)to obtain consistent results.Hi again sorry to bother you, I've tried setting seed before training starts but still can't get consistent results for each target detection, where should I set the random seed? Is it easy to tell which file? It would be more appreciated if you can tell the exact location. Thank you for your time! Here's where I've tried to set random seeds:
class DetSolver(BaseSolver): def fit(self, ): print("Start training") self.train() # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False np.random.seed(42) random.seed(42) torch.manual_seed(42) ...... def val(self, ): self.eval() # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False # dist.set_seed(42) np.random.seed(42) random.seed(42) torch.manual_seed(42) ....... def train_one_epoch(model: torch.nn.Module, criterion: torch.nn.Module, data_loader: Iterable, optimizer: torch.optim.Optimizer, device: torch.device, epoch: int, max_norm: float = 0, kwargs): # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False # dist.set_seed(42) np.random.seed(42) random.seed(42) torch.manual_seed(42) ....... def evaluate(model: torch.nn.Module, criterion: torch.nn.Module, postprocessors, data_loader, base_ds, device, output_dir): # add myself 可复现 torch.backends.cudnn.deterministic = True torch.backends.cudnn.benchmark = False # dist.set_seed(42) np.random.seed(42) random.seed(42) torch.manual_seed(42) ....... if name == 'main**': parser = argparse.ArgumentParser() # parser.add_argument('--config', '-c', type=str, ) parser.add_argument('--config', '-c', type=str, default="/home/lw/rtdetr_pytorch/configs/rtdetr/rtdetr_r18vd_6x_sonar.yml") parser.add_argument('--resume', '-r', type=str, default=False) parser.add_argument('--tuning', '-t', type=str, default=False) parser.add_argument('--test-only', action='store_true', default=False,) parser.add_argument('--amp', action='store_true', default=False,) args = parser.parse_args() np.random.seed(42) random.seed(42) torch.manual_seed(42) main(args)
Hello, now I have the same problem as you, how do you solve it?
helllo.Why are the mAP results inconsistent every time I train under the same conditions?code is rtdetr_pytorch? What should I fix, and could you provide some advice,please