Evaluation

[BurstLink666]

Sorry to disturbing you again, but I got a question when doing evaluation.

During training process, I saved best-3 model according to the mae and rmse, and I used the one with rmse = 16.2453 and mae = 11.7919 to do the evaluation in the same dataset, I mean use the same dataset as the training. I load the model as using following code:

model = get_model( backbone=args.model, input_size=args.input_size, reduction=args.reduction, bins=bins, anchor_points=anchor_points, prompt_type=args.prompt_type, num_vpt=args.num_vpt, vpt_drop=args.vpt_drop, deep_vpt=not args.shallow_vpt ) state_dict = torch.load(args.weight_path, map_location=device) state_dict = state_dict if "best" in os.path.basename(args.weight_path) else state_dict["model_state_dict"] model.load_state_dict(state_dict, strict=True) model = model.to(device)

And run the evaluation.py to test it, but got mae = 605.7568 and rmse = 606.5157.

It seems that the weight is loaded incorrectly. Is there any problem in my code?

I using following args:

"--model", "clip_vit_b_16", "--weight_path", "./checkpoints/rgbt/clip_vit_b_16_word_448_8_4_fine_1.0_dmcount/best_rmse_0.pth", "--anchor_points", "average", "--dataset", "mydataset", "--truncation", "4", "--device", "cuda:0"

[Yiming-M]

Can you make sure that the model is trained with 224x224 and evaluated using the sliding window method with the same window size (i.e., 224x224)?

[BurstLink666]

In sliding_window_predict function, you input 5 arguments, but the function only takes 4 arguments. The arguments should be model, image, window_size and stride, but you input argument named "strategy".

[BurstLink666]

And, I only input model, image, window_size and stride, with training image size and window size both equal to 448, the test result still keep the same.

[Yiming-M]

Hi,

The issue in test_nwpu.py has been fixed. Training and testing models on private datasets are not guaranteed to work properly. Thank you for your understanding.