I'm using GTAVision without any changes and your post-processing from https://github.com/umautobots/gta-postprocessing. There, I only changed the numbers 1920 to 1280 and 1080 to 720.
I collected some data for tests in postgreSQL and I have a problem with the coordinate transformation. You can find sample images attached in zip.
1.bmp: Box around the car is good, but the ones on the left shouldn't be there, since there is nothing within 150 meters.
2.bmp: Box around car is still good, but the other boxes are also there
3.bmp: Now, this problem exists in many coordinate transformation methods within GTA. With yours, I tried 3 different methods and always got this problem. As soon as the camera passes a point (in this case the back corners of the car) the corners will be projected wrong and appear in front of the car. As soon as I pass the last visible corner of the car (car is completely out of image), the projection will stop.
4.bmp: Shows an extreme case, since there are much more cars around.
Did you also encounter this problem? I'm not sure, but I think, that clipping is needed, so that points behind the car and within the frustum won't be projected. I tried this with scripted cams and also with gameplay camera with same error.
Can you also tell a bit about the npz files and the results.pkl file, which are created by the post-processing? How can I view them, are these images with the pixel colors and the 2D bounding boxes?
I'm using GTAVision without any changes and your post-processing from https://github.com/umautobots/gta-postprocessing. There, I only changed the numbers 1920 to 1280 and 1080 to 720. I collected some data for tests in postgreSQL and I have a problem with the coordinate transformation. You can find sample images attached in zip. 1.bmp: Box around the car is good, but the ones on the left shouldn't be there, since there is nothing within 150 meters. 2.bmp: Box around car is still good, but the other boxes are also there 3.bmp: Now, this problem exists in many coordinate transformation methods within GTA. With yours, I tried 3 different methods and always got this problem. As soon as the camera passes a point (in this case the back corners of the car) the corners will be projected wrong and appear in front of the car. As soon as I pass the last visible corner of the car (car is completely out of image), the projection will stop. 4.bmp: Shows an extreme case, since there are much more cars around.
Did you also encounter this problem? I'm not sure, but I think, that clipping is needed, so that points behind the car and within the frustum won't be projected. I tried this with scripted cams and also with gameplay camera with same error.
images.zip
Can you also tell a bit about the npz files and the results.pkl file, which are created by the post-processing? How can I view them, are these images with the pixel colors and the 2D bounding boxes?
Thanks.