tgjantos
commented
6 months ago Hi Daniele,
you are right. Currently PoET can only be trained in a two step manner, i.e. the transformer part is trained on top of an already pre-trained object detector backbone. Regarding the missing labels for training the YOLO network, please check out this issue.
You can probably load the .pth model and extract the backbone model weights and load them into YOLO for training. However, the pre-trained model that I provide is already fully trained for the YCB-V dataset and hence you might not see any learning progress. Unfortunately, the pre-trained model for YOLO, that I used as a starting point for training on the YCB-V dataset, is not publicly available anymore from the original author. Due to the size I can not upload it, however if you provide me with an email address or write me an email, I can send you the file (which would be an .pt file).
Best, Thomas
DanieleBertagnoli
Hi I was trying to train the YOLO model (https://github.com/aau-cns/yolov4) using YCB-V dataset (in order to do some tests for the poet model). Can you link me the YCB version you used for training that YOLO model? I noticed that in
datasets.pyyou look for labels and images files that are not present in the standard YCB dataset. Since my final goal is to train both YOLO and poet on a custom dataset, I'm interested into understanding how to train both, since as I read from other issues the backbone is currently not trainable directly in poet.Moreover, is there a way to use the pre-trained .pth model you provided? I see in yolov4/train.py that I can load weights of the model but the file extension is .pt and not .pth (sorry if this sounds as a silly question but I'm quite new to deep learning)
Best regards, Daniele