Evaluation ERROR: Couldn't read: 006116.txt of ground truth.

[YuqiShen]

Hi,

Thank you for sharing the code.

I wonder if anyone meets the similar problem while running the run_evaluation.py. I could run the evaluation and got the right results two days ago and this suddenly doesn't work today and outputs error like following:

101000 ERROR: Couldn't read: 006116.txt of ground truth. Please write me an email! An error occured while processing your results. 120000 ERROR: Couldn't read: 006116.txt of ground truth. Please write me an email! An error occured while processing your results.

I am quiet confused since I don't change anything and using the same checkpoint file as before. I find a similar error here (https://github.com/kujason/avod/issues/93#issue-346395681), and this solution works for me. But I just don't understand why I get this error since nothing is changed. Does Anyone have idea on this?

Thank you for your time and help in advance!

[666zz666]

Hello，I still can't resolve this error.Could you give me more details?Please.

[YuqiShen]

I do not solve it directly with run_evaluation.py, but use the solution in https://github.com/kujason/avod/issues/93#issuecomment-409814425.

As it shows,

First run the file save_kitti_predictions.py to create labels in the following directories: For 2d predictions: avod/avod/data/outputs/.../predictions/kitti_predictions_2d/val/0.1/.../data For 3d predictions: avod/avod/data/outputs/.../predictions/kitti_predictions_3d/val/0.1/.../data

Then evaluate the checkpoint run the following (adjust the paths to the label and the result folder): ./evaluate_object_3d_offline /KITTI/training/label_2 /avod/avod/data/outputs/.../predictions/kitti_predictions_2d/val/0.1/...

And you will get the evaluation result for one checkpoint. And I also can not figure out where is the error in the evaluation or other codes.

[666zz666]

So should I run run_evaluation.py before this？

[YuqiShen]

Yes, run the evaluation before these steps.

[666zz666]

I run the following (adjust the paths to the label and the result folder): ./evaluate_object_3d_offline /KITTI/training/label_2 /avod/avod/data/outputs/.../predictions/kitti_predictions_2d/val/0.1/...

But I got the same error: ERROR: Couldn't read: 004290.txt of ground truth. Please write me an email! An error occured while processing your results.

So, do you have this?

[YuqiShen]

@666zz666, sorry, I don't have this error with this solution.

Maybe you could just write an email. I also wrote one but don't get the reply yet.

[666zz666]

LOL. Thank you very much for your time.

[yxy1995123]

I think I found the solution.In file run_eval.sh and run_eval_05_iou.sh../evaluate_object_3d_offline_05_iou ~/Kitti/object/training/label_2/ $2/$3 | tee -a ./$4_results_05iou$2.txt. change the label path according to your computer

[asvath]

Any updates on this? I'm facing the same issue.

[DGCHAO]

I think I found the solution.In file run_eval.sh and run_eval_05_iou.sh../evaluate_object_3d_offline_05_iou ~/Kitti/object/training/label_2/ $2/$3 | tee -a ./$4_results_05iou$2.txt. change the label path according to your computer

Thank you,you are right.I solution my problem.

[EchoMaria123]

I think I found the solution.In file run_eval.sh and run_eval_05_iou.sh../evaluate_object_3d_offline_05_iou ~/Kitti/object/training/label_2/ $2/$3 | tee -a ./$4_results_05iou$2.txt. change the label path according to your computer

Thank you,you are right.I solution my problem.

Could elaborate on that ? I am still confused

[patelrahil224]

In my case restarting the terminal worked! Not sure what the problem is.

When you see a source code working in the background there shouldn't be any problem as long as even the blank file lies in that directory. The error should only be generated when there is no file which is definitely not the case here

for (int32_t i=0; i<indices.size(); i++) {
    // file name
    char file_name[256];
    sprintf(file_name,"%06d.txt",indices.at(i));

    // read ground truth and result poses
    bool gt_success,det_success;
    vector<tGroundtruth> gt   = loadGroundtruth(gt_dir + "/" + file_name,gt_success);
    vector<tDetection>   det  = loadDetections(result_dir + "/data/" + file_name,
            compute_aos, eval_image, eval_ground, eval_3d, det_success);
    groundtruth.push_back(gt);
    detections.push_back(det);

    // check for errors
    if (!gt_success) {
      mail->msg("ERROR: Couldn't read: %s of ground truth. Please write me an email!", file_name);
      return false;
    }
    if (!det_success) {
      mail->msg("ERROR: Couldn't read: %s", file_name);
      return false;
    }

[kujason]

Please set up the dataset as described in the readme, as the run_evaluation.py assumes the labels are at ~/Kitti/object/training/label_2. You can specify a different label_2 location if you run evaluate_object_3d directly from the command line.

[Cli98]

I run this error now. And find way to solve it.

1. If you have only 1 kept, use evaluate_object_3d
2. Make sure the path to label is correct. If you can not load det, you get only filename. If you cannot load gt, then author asks you to send him an email :) Actually case 2 is pretty straightforward to debug.

[faziii0]

hi do we need to do evalution on testset or trainval set to submit to kitti as groudtruth label_2 are just 7480 and testset has 7517. please help