Evaluation

[deepmeng]

Hi, Nguyen. I evaluated this work by using the 'kitti eval' tool. As far as I know, this SFA3D has the same detection range as your work of https://github.com/maudzung/Complex-YOLOv4-Pytorch and https://github.com/maudzung/YOLO3D-YOLOv4-PyTorch/. However, this work has a better performance than the other two work. Please tell me if my conclusion about these three methods' performance is right?

[iliiliiliili]

@deepmeng, could you share evaluation results here? As far as I know, there are no available KITTI evaluation results for this method. Thanks

[maudzung]

Hi @deepmeng

Yeb, this work has a better performance compared to the YOLO-based methods in both accuracy and speed. Please contribute the full evaluation tool if you make it. Thanks.

[bilgecanpullinen]

Hi @deepmeng and @maudzung I couldn't figure out how I can evaluate (write the prediction results as txt files). Could you share your evaluation file. Thanks

[deepmeng]

@bilgecanpullinen OK. The code in my hand is chaotic. I will fine-tune and share the code.

[deepmeng]

Hi @iliiliiliili and @bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo (https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results.

[bilgecanpullinen]

Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one.

deepmeng @.***>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı:

Hi @iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maudzung/SFA3D/issues/16#issuecomment-809865098, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ .

[iliiliiliili]

Hi @iliiliiliili and @bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo (https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results.

Thank you, @deepmeng. Could you add evaluation results on KITTI validation subset with your code to README? It would be very helpful 🙂

[deepmeng]

@bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'.

Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.***>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … Hi @iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ .

[bilgecanpullinen]

Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving

ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh

For detailed information on this command, run: evaluate.py evaluate --help

which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions.

deepmeng @.***>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı:

@bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'.

Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.***>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m2157938398064785683> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) https://github.com/maudzung/SFA3D/issues/16#issuecomment-809865098>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maudzung/SFA3D/issues/16#issuecomment-810055000, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ .

[deepmeng]

Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng @.>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … @bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m2157938398064785683> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ .

I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea.

[bilgecanpullinen]

Thank you again. I am having another problem and sorry for asking it again but I have tried many things. g++ -o evaluate_object_3d_offline evaluate_object_3d_offline.cpp -O3 evaluate_object_3d_offline.cpp:12:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory

include <boost/numeric/ublas/matrix.hpp>

      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

compilation terminated.

This is my error while I try to compile. Do you remember having an issue like this.

deepmeng @.***>, 30 Mar 2021 Sal, 15:07 tarihinde şunu yazdı:

Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng @.

>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … <#m6312827631054034898> @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m2157938398064785683> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 https://github.com/maudzung/SFA3D/issues/16 (comment) <#16 (comment) https://github.com/maudzung/SFA3D/issues/16#issuecomment-809865098>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) https://github.com/maudzung/SFA3D/issues/16#issuecomment-810055000>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ .

I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maudzung/SFA3D/issues/16#issuecomment-810166725, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJEVPQ3FCGGZ36P453TGG5H3ANCNFSM4TOLZ7LQ .

[deepmeng]

Thank you again. I am having another problem and sorry for asking it again but I have tried many things. g++ -o evaluate_object_3d_offline evaluate_object_3d_offline.cpp -O3 evaluate_object_3d_offline.cpp:12:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory #include <boost/numeric/ublas/matrix.hpp> ^~~~~~~~ compilation terminated. This is my error while I try to compile. Do you remember having an issue like this. deepmeng @.**>, 30 Mar 2021 Sal, 15:07 tarihinde şunu yazdı: … Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng @. >, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … <#m6312827631054034898> @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.*>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m2157938398064785683> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 <#16> (comment) <#16 (comment) <#16 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ . I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJEVPQ3FCGGZ36P453TGG5H3ANCNFSM4TOLZ7LQ .

Well. I'm not sure your experimental environment. You must make sure you have installed the Boost library. Maybe you can try 'sudo apt-get install libboost-all-dev'.

[bilgecanpullinen]

Okay, I don't have the sudo right for that environment but I will manage it from here. Thank you :)

deepmeng @.***>, 30 Mar 2021 Sal, 16:09 tarihinde şunu yazdı:

Thank you again. I am having another problem and sorry for asking it again but I have tried many things. g++ -o evaluate_object_3d_offline evaluate_object_3d_offline.cpp -O3 evaluate_object_3d_offline.cpp:12:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory

include <boost/numeric/ublas/matrix.hpp> ^~~~~~~~

compilation terminated. This is my error while I try to compile. Do you remember having an issue like this. deepmeng @.**>, 30 Mar 2021 Sal, 15:07 tarihinde şunu yazdı: … <#m-3983213664647106577> Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng @. >, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … <#m6312827631054034898> @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.*>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m2157938398064785683> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 https://github.com/maudzung/SFA3D/issues/16 <#16 https://github.com/maudzung/SFA3D/issues/16> (comment) <#16 https://github.com/maudzung/SFA3D/issues/16 (comment) <#16 (comment) https://github.com/maudzung/SFA3D/issues/16#issuecomment-809865098>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 https://github.com/maudzung/SFA3D/issues/16 (comment) <#16 (comment) https://github.com/maudzung/SFA3D/issues/16#issuecomment-810055000>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ . I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) https://github.com/maudzung/SFA3D/issues/16#issuecomment-810166725>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJEVPQ3FCGGZ36P453TGG5H3ANCNFSM4TOLZ7LQ .

Well. I'm not sure your experimental environment. You must make sure you have installed the Boost library. Maybe you can try 'sudo apt-get install libboost-all-dev'.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maudzung/SFA3D/issues/16#issuecomment-810222300, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GP7ACZNWV4GCXGGQBLTGHEQPANCNFSM4TOLZ7LQ .