Closed alfieroddan closed 8 months ago
Thanks for your interest in our work :-)
If you only want to get the assert working, you need to change it to
torch.isclose(x.abs().sum(dim=1), torch.tensor(1.0, device=x.device, dtype=x.dtype), atol=0.1)
That the features of the batch have dtype float16 is expected and usually what you want.
btw: I recently wrote a SinglePredictor
class which can be used to create predictions outside of our usual producer/consumer architecture. Using that, the basic prediction pipeline may look like:
config = Config(args.config_path)
spec = DataSpecification.from_config(config)
paths = spec.paths("val")
dataloader = DatasetImageBatch.batched_iteration(paths, config)
predictor = SinglePredictor(model="image", run_folder="2023-02-08_14-48-02_organ_transplantation_0.8")
for batch in dataloader:
logits = predictor.predict_batch(batch)["class"]
logits.shape # 5, 19, 480, 640
Cool worked like a charm! I'll use the SinglePredictor as my entry point from now on. Thanks.
Just a question about model loading, Why does HTCModel.find_pretrained_run(model, run_folder, path)
load the pretrained weights from the backbone when using a model checkpoint?
(env) tay@tay:~/Code/ms-seg$ PATH_Tivita_HeiPorSPECTRAL=/media/tay/4TB/Datasets/HeiPorSPECTRAL PATH_HTC_RESULTS=./results/ python test.py
[INFO][htc.no_duplicates] Found pretrained run in the local results dir at HTCModel.py:477
/home/tay/Code/ms-seg/results/training/image/2023-12-13_09-24-20_htc-config
[INFO][htc.no_duplicates] Found pretrained run in the local hub dir at HTCModel.py:484
/home/tay/.cache/torch/hub/htc_checkpoints/image/2023-02-08_14-48-02_organ_transplantation_0.8
[INFO][htc] Successfully loaded the pretrained model (2 keys were skipped: HTCModel.py:312
['model.architecture.segmentation_head.0.weight', 'model.architecture.segmentation_head.0.bias']).
It seems odd that it skips keys when loading a model checkpoint, also no logging info on whether a pretrained run has been loaded.
I am not entirely sure I understand the question but maybe this clarification helps:
HTCModel.find_pretrained_run(model, run_folder, path)
does not load a checkpoint, it only searches for the specified training run.
With my code from above (using the SinglePredictor
) there should be no skipped weights as it directly loads the lightning class and then you can use the model as-is (i.e. based on the checkpoint) to make predictions. So, this is a good entrypoint for inference.
If you use the model classes directly, e.g. ModelImage.pretrained_model()
, then you are getting a model instance which you can use in your own lightning class for further training. The weights for the segmentation head are skipped because those weights depend on the number of classes you want to use (there is also a n_classes
parameter). Hence, they are initialized randomly so that you can kick off your training with your class setting. So, this is a good entrypoint for training.
All understood! Thank you again for your help.
I look forward to experimenting more with your package. A great piece of work :)
:question: Question
Just a quick question, trying to use a pretrained model to run over a validation set using a custom script (using htc).
I get the following error:
The batch["features"] type is float-16 and the
torch.tensor
is float, so obviouslytorch.isclose
raises an error.Am I creating the data loaders wrong?
Any help would be much appreciated. Starting to get familiar with HTC, thanks for the hard work creating this package :).
Description
htc_data.py
custom script
htc config.json