Closed ClaireTun closed 6 years ago
Yes to both of your first two questions.
I had also the loss nan problem and this was the most frustrating of all. It turend out to be coused from bad data. If you are using the dataset from Cornell than do a search for a 'nan' or 'NAN' string in the *cpos.txt
file. Tell me if you find something!
Hi,
at least in folder 01 of the dataset you will find some 'NaN' values for the rectangles. Do a grep -ri --include="*cpos.txt" "nan" .
to find if there are others and just delete the rectangle that they fall in, so the four corresponding lines.
I wrote to the Cornell Lab about the bad data. I hope they will fix this!
@tnikolla Hey i am using the imagenet model i.e., --model_path=./models/imagenet/m2/m2.ckpt, which is giving the following error -> "Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint."
And when i use --model_path=./models/grasp/m4/m4.ckpt the loss is nan. I remove all the bad files with nan rectangle from dataset, still it shows loss as NAN.
Can you please help me solve this. Also if you have a trained model can you mail me on : goyal.pragya001@gmail.com
This is really urgent, if you could help.
Hi, I am wondering whether the grasp model is the one pretrained on the imagenet? Can I use it to train the trainset? I tried , but the loss is nan?