JiuqingDong
commented
1 year ago Oh, I already solved this issue and forgot to put the solution here.
Step 1: Save the full model for best_metric. if curr_acc >= best_metric: model_state = self.model.state_dict() out_path = os.path.join(self.cfg.OUTPUT_DIR, "best_model.pth") torch.save(model_state, out_path)
Step 2: Train(Tuning) the model and get the checkpoint file. Besides,
Use 'model.load_state_dict(torch.load(cfg.MODEL.WEIGHT_PATH))' to load the checkpoint.
Step 3: Create a shell file like this:
python tune_fgvc.py \ --train-type "prompt" \ --config-file configs/prompt/cub.yaml \ MODEL.TYPE "vit" \ DATA.BATCH_SIZE "128" \ MODEL.PROMPT.DEEP "True" \ MODEL.PROMPT.DROPOUT "0.1" \ MODEL.PROMPT.NUM_TOKENS "10" \ DATA.FEATURE "sup_vitb16_imagenet21k" \ MODEL.MODEL_ROOT "models/" \ OUTPUT_DIR "./prompt_tuning/CUB/" \ MODEL.WEIGHT_PATH "/Path/val_CUB_model.pth" \ SOLVER.TOTAL_EPOCH '0' \
By the way, It is a naive way to do evaluation. I want to know how to use logits.pth to do the evaluation, because in your code, you just save the {"targets": total_targets, "joint_logits": joint_logits} in checkpoint file.
Hi, could you tell me how can I evaluate the model when I finished the tuning?