Triangulation not working?

[imanolooo]

Hi,

I'm quite new using Regard3D and I'm really happy with how it usually works. However, I think there is an issue in the triangulation process for some configurations of the parameters.

In some configurations, I'm able to get a final model, but in others it doesn't work. The matching process works well and I can see the matches found. But, although the triangulation step seems to work (cameras, number of points and residual errors are shown), in the rendering widget nothing is shown. If I continue with the process, the densification step does the same: it seems to be working but it shows nothing in the rendering widget. And, finally the surface generation step crashes.

Anybody can help me to understand what is going on? and the most important part, how can I solve it?

Best,

[AlansCodeLog]

I had a similar problem with the latest version, don't know if it's the exact same issue. For me nothing showed, not even the circular widget after toggling it on/off. I thought it was because it was an existing project I was working over, maybe something had changed when opening it with the new version. I closed it and opened it again, and everything was showing again.

You can also try opening the generated files manually from the project folder to check if anything was actually generated.

[imanolooo]

Thanks for answering! I appreciate that.

However, my problem is not like that. I can see the circular widget, but nothing (points, cameras,...) in the widget and although I restart the application, it doesn't show anything...

I'm trying to open the generated files manually, but what is the folder for the triangulation process?

Best!

Imanol Muñoz i Pandiella

2018-03-07 22:03 GMT+01:00 Alan North notifications@github.com:

I had a similar problem with the latest version, don't know if it's the exact same issue. For me nothing showed, not even the circular widget after toggling it on/off. I thought it was because it was an existing project I was working over, maybe something had changed when opening it with the new version. I closed it and opened it again, and everything was showing again.

You can also try opening the generated files manually from the project folder to check if anything was actually generated.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rhiestan/Regard3D/issues/22#issuecomment-371283849, or mute the thread https://github.com/notifications/unsubscribe-auth/AQgu2uhgoR9hBiZWFCo5qfIIQh1wta_Nks5tcEs2gaJpZM4Sgl00 .

[AlansCodeLog]

My problem ended up being that I had multiple versions installed. A clean installed fixed it. Might want to give that a try as well.

As for where the files are, you have to follow the path in the tree, so say you're in the first picture set, second match, first triangulation, it would be pictureset_0 > matching_1 > triangulation_0 > out. In there except for the initialPair.ply and the resections which are just parts, any of the other ply files should contain the triangulation cloud. And I believe, given the name, FinalColorized.ply is the one it displays.

[imanolooo]

I only have one version installed, but maybe I can remove it and install it again.

I have checked the FinalColorized.ply files for all the configurations I get no output and, although there are several vertices (>100K) their coordinates are very similar... I copy a small set of one of them. ply format ascii 1.0 element vertex 133837 property double x property double y property double z property uchar red property uchar green property uchar blue end_header -0.0227859833672198 -0.4839432099329821 0.6594028206784737 149 165 162 -0.0227859804635297 -0.4839432100327379 0.6594028210925097 47 43 34 -0.0227859781239427 -0.4839432102799627 0.6594028217236084 77 59 75 -0.0227859772984980 -0.4839432102694252 0.6594028217165572 131 122 115 -0.0227859737984274 -0.4839432100466320 0.6594028211402164 129 120 121 -0.0227859817672886 -0.4839432099997641 0.6594028209974772 159 147 147 -0.0227859818019589 -0.4839432099866099 0.6594028210002485 102 89 96 -0.0227859798012039 -0.4839432101866900 0.6594028215427582 143 121 123 -0.0227859839352303 -0.4839432098788152 0.6594028206349383 88 82 84 -0.0227859847312013 -0.4839432098120254 0.6594028204906776 119 104 97 -0.0227859793990184 -0.4839432101929090 0.6594028216983789 72 59 69 -0.0227859838279961 -0.4839432098408357 0.6594028206615141 78 67 65 -0.0227859812187726 -0.4839432099452727 0.6594028210586976 172 145 152 -0.0227859773152495 -0.4839432101947857 0.6594028217202774 33 21 33 -0.0227859770564436 -0.4839432102120950 0.6594028217804012 153 144 147 -0.0227859798105549 -0.4839432101317216 0.6594028215707196 171 159 161 -0.0227859780438358 -0.4839432101722013 0.6594028217433805 79 67 69 -0.0227859854731800 -0.4839432098212183 0.6594028205630842 40 25 28 -0.0227859794866671 -0.4839432101416389 0.6594028216580249 155 146 149 -0.0227859855610087 -0.4839432098143324 0.6594028205590675 26 21 17 -0.0227859816276990 -0.4839432098936611 0.6594028210013133 179 167 171 -0.0227859775325117 -0.4839432101579713 0.6594028217289859 185 181 180 -0.0227859848888510 -0.4839432097911741 0.6594028205925035 144 143 151 -0.0227859847240429 -0.4839432097842638 0.6594028205835791 29 25 14 -0.0227859809339097 -0.4839432098952898 0.6594028210682252 86 67 69 -0.0227859785367443 -0.4839432101641423 0.6594028217755860 158 142 143 -0.0227859828819765 -0.4839432098902967 0.6594028209796896 163 162 168 -0.0227859813802659 -0.4839432098935484 0.6594028210676167 28 26 29 -0.0227859774952293 -0.4839432101386390 0.6594028217312744 102 100 105 -0.0227859773496489 -0.4839432101377337 0.6594028217227874 153 133 145 -0.0227859883717707 -0.4839432098357744 0.6594028204179041 133 118 123 -0.0227859846265072 -0.4839432097552447 0.6594028205852842 129 114 111 -0.0227859781854453 -0.4839432101454984 0.6594028217641057 84 72 74 -0.0227859768430916 -0.4839432101463377 0.6594028217641218 148 132 142 -0.0227859753200954 -0.4839432098493999 0.6594028210384125 132 138 136 -0.0227859736926449 -0.4839432099337211 0.6594028212047249 140 124 125 -0.0227859839171364 -0.4839432097683712 0.6594028206640676 121 128 134 -0.0227859820586830 -0.4839432098724000 0.6594028210309464 180 174 174 -0.0227859774502554 -0.4839432101200037 0.6594028217360841 167 157 156 -0.0227859742556165 -0.4839432098967918 0.6594028211645416 116 107 100 -0.0227859842829706 -0.4839432097454592 0.6594028206183171 157 131 130 -0.0227859839910373 -0.4839432097600632 0.6594028206625041 70 60 59 -0.0227859811394248 -0.4839432098712289 0.6594028210820858 137 126 124 -0.0227859806702884 -0.4839432098670705 0.6594028210922290 169 159 160 -0.0227859780939155 -0.4839432101278639 0.6594028217671850 90 70 81 -0.0227859772198965 -0.4839432101414094 0.6594028218064851 82 71 79 -0.0227859766872558 -0.4839432101271109 0.6594028217667817 79 64 67 -0.0227859837765203 -0.4839432097592701 0.6594028207025097 134 126 124 -0.0227859832917022 -0.4839432097804704 0.6594028207706556 54 42 62 -0.0227859828319601 -0.4839432098560383 0.6594028209860523 109 105 106 -0.0227859842365333 -0.4839432097466537 0.6594028206537650 159 148 152 -0.0227859824183228 -0.4839432098447822 0.6594028210013276 89 80 85 -0.0227859781581273 -0.4839432101112594 0.6594028217790990 173 156 162 -0.0227859813197719 -0.4839432098544083 0.6594028210757987 138 126 126 -0.0227859777533561 -0.4839432101156539 0.6594028217965251 95 77 91 -0.0227859814930330 -0.4839432098201771 0.6594028210025860 96 84 86 -0.0227859812420734 -0.4839432098622765 0.6594028211140901 71 59 61 -0.0227859839106680 -0.4839432097336571 0.6594028206831830 65 63 68 -0.0227859834909315 -0.4839432097505431 0.6594028207607331 82 62 61 -0.0227859833301973 -0.4839432097530688 0.6594028207673945 158 162 174 -0.0227859847732025 -0.4839432097203762 0.6594028206174911 140 123 131 -0.0227859750102378 -0.4839432098140352 0.6594028210953526 123 116 108 -0.0227859747608861 -0.4839432098256054 0.6594028211353981 134 125 118 -0.0227859856491683 -0.4839432097215058 0.6594028205801939 166 146 145 -0.0227859836186477 -0.4839432097299973 0.6594028207470056 94 75 77 -0.0227859873147719 -0.4839432097521605 0.6594028205206600 148 115 126 -0.0227859824067833 -0.4839432098040988 0.6594028210145154 144 138 140 -0.0227859872047720 -0.4839432097355341 0.6594028205112008 157 141 142 -0.0227859831784653 -0.4839432097941166 0.6594028209625004 107 89 87 -0.0227859810413367 -0.4839432098105207 0.6594028211142260 101 81 92 ... Maybe the problem is that they are to similar, or what could be happening?

Imanol Muñoz i Pandiella

2018-03-14 21:47 GMT+01:00 Alan North notifications@github.com:

My problem ended up being that I had multiple versions installed. A clean installed fixed it. Might want to give that a try as well.

As for where the files are, you have to follow the path in the tree, so say you're in the first picture set, second match, first triangulation, it would be pictureset_0 > matching_1 > triangulation_0 > out. In there except for the initialPair.ply and the resections which are just parts, any of the other ply files should contain the triangulation cloud. And I believe, given the name, FinalColorized.ply is the one it displays.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rhiestan/Regard3D/issues/22#issuecomment-373169163, or mute the thread https://github.com/notifications/unsubscribe-auth/AQgu2nOvqIY8fx7u9n5-PiGvvyavOGDRks5teYHxgaJpZM4Sgl00 .

[AlansCodeLog]

Sorry, don't know what it could be, I don't really know anything much about how the code works, just trying to help since I have used the program a lot.

If you're using your own photos I would try with the kermit test picture set instead. Also you said certain parameters were resulting in this, what where they? Maybe it has something to do with those.

It is possible to get no matches without an error, but if in the lower left it says it matched most of the cameras and yet isn't showing them, that feels like a bug.

[imanolooo]

Thanks for your help. It also looks like a bug to me.

I'm using my set of images and it works depending on the configuration. For example, the configuration that I changed is the threshold and dist ratio. I was testing different configurations to get the best final result.

But it seams there is some bug on it, because if I try to generate a final surface with the results where I don't see anything, Regard3D crashes... After seeing the pointcloud files, could be for the low distances... but i don't know. I don't either know the details of the code.

It seams that the matching and triangulation is working:

Cameras: 139/166
Number of points: 133400
Residual errors: 7.03e-08/5.6/0.267/0.157

Don't think so?

[AlansCodeLog]

Yeah, very strange, seems like a bug. I would try it if possible on a different machine just to be sure. And if you can replicate it consistently with any of the photo sets available online that would also probably be potentially useful information for the developer.

Do you mean the keypoint sensitivity and matching ratio? I'm in the process of doing a post/video about the best settings, and I've found the matching ratio is the most important. If your computer can handle ultra, then after the incremental triangulation usually gets 100% of the cameras. Keypoint sensitivity can get a few more keypoints, but usually increasing the matching ratio is more than enough.

Also, 166 seems like a lot of pictures to me. I'm not sure what you're scanning, but for most singular objects you don't need that many, ~80 is more than enough (4 heights x 20 angles).

[imanolooo]

I don't think it could be generated with other photo sets available online, because maybe the problem is that the algorithm is not able to reconstruct using my images. I have opened the output files of the triangulation process when it fails in Meshlab and the results are really weird, like if it is reconstructing the same points all the time: Here an EXAMPLE:

Yes I mean keypoint sensitivity and matching ratio, I used to increase both of them.

The number of images maybe is too high? I'm reconstructing cultural heritage monuments and I need the highest possible quality. For this I took so many pictures. Do you think that with less pictures it would work better?

Thanks a lot!

[pmoulon]

@imanolooo I thin you hit a problem in the bundle adjustment step. It seems related to a scale effect that give stair like point cloud. You can also try to open your point cloud in CloudCompare. This last software is less sensible to floating point precision.

If you want you can try your dataset with OpenMVG (the underlying reconstruction pipeline) so this way we could debug and understand what's going on.

[AlansCodeLog]

Maybe, but when I've had problems with picture sets (for example, I've found some surfaces/textures can confuse the global reconstruction engine), at least I could see a few points and the cameras. That's what really puzzles me. But I'm way out of my depth at this point. Try pmoulon's suggestions.

As for the number of pictures, I don't think it's the problem, the number just surprised me. Of course, it can depend on the detail/scale of what you're scanning and the type of surface, but if many of them are almost the same view, it can't hurt to try with fewer.

[imanolooo]

I have oppened the same ply with cloud compare and the result is similar: You can see a capture HERE.

I can try the dataset with openmvg, but what do you want I publish here?

Best,

[pmoulon]

I think the most important for you is to have a successful reconstruction, whatever you are using.

[imanolooo]

Well, my goal is to have the most precise reconstruction possible. I was using Autodesk Remake before and the results were really good. I would like to get similar results with opensource code.

For that I'm testing with different configurations to find the best and for that reason I would like to solve my problem with some configurations, to find the best one.

[pmoulon]

Feel free to ask me help for processing your dataset with OpenMVG ;-)

[smpel]

Could anybody please explain the maths behind the 4 residuals for the below results? : Triangulation method: Global (L1 rotation averaging, SoftL1 triangulation averaging), intrinsic camera (pinhole) parameters refined Cameras: 56/85 (green points) Number of points: 150134 Residuals: 1.09e-08/4.45/0.219/0.147 Time: 05m:58.831sec

[pmoulon]

As you can see in the code it display the min, max, mean, median of the triangulation errors: https://github.com/rhiestan/Regard3D/blob/2822275259e9ca140b77b89c1edeccf72bc45f07/src/threads/R3DTriangulationThread.cpp#L661

[pmoulon]

So it displays how good your 3D point and camera fit your observations data (the 2d points detected in the images).

[smpel]

Thank you very much, Pierre!

Roman Hiestand did a great job with Regard3D and you have a deep knowledge of it, but please do not expect me to read the code. :-)

It will be a loss of time for me because I will not understand it. :-)

I did an architectural survey of an old monastery to check the results and I came to this Poisson surface I attach bellow in the picture.

Now I am wondering if there is a way to get rid of these unnecessary triangles and leave only the wall and the building behind it with the texture.

I will appreciate any advice. I thought to prepare a quick and dirty code in Lisp to read the .ply file and transfer to AutoCAD the triangles in 3dfaces colourized with the average RGB values of the triangle corners. Is there any better way?

I think also it would be very useful if the program could report the average scale of the model. In photogrammetry, the scale is the ratio of the focal length by the distance of the object from the camera. I apologize if you already calculate it and I can not find it.

Congratulations to all contributors in this effort and Happy new year!

[rhiestan]

In Regard3D in the surface dialog, use the "Trim threshold" slider to adjust the generation of triangles. You will need a bit of experimentation to find a good value for your scene.

[smpel]

Thanks, Roman and congratulations on your excellent application. Finally, I trimmed the obsolete triangles manually after I made a program in Lazarus to convert the PLY files to Coloured DXF and then with AutoCAD using Autolisp routines to delete the 3DFACES filtering the selections with max triangle perimeter, colour, max angles and max sides. Of course, it needed to do much manual labour to get rid of all the unnecessary triangles. I will follow your advice to experiment on the "Trim threshold" slider. I attach the result! I am at your disposal if I can help somehow with data or with calibration techniques to improve accuracy.

[Aerial-Scouts]

I'm trying to figure out why triangulation took 3 hours to complete with matches of plenty from 668 photos... I ran triangulation and it found 0 cameras created 0 points. Any thoughts?

This is the object I'm reconstructing:

[Aerial-Scouts]

Sooooo, I opened up the camera models database to see if my camera was in there, and lo and behold it is not. I'm hoping that I do not have to rerun the matches when I update the EXIF data of the photo batch with my copyright information- I believe I just need to update the camera database with a user-defined model to get the triangulation to work correctly.

[Aerial-Scouts]

I just found that the online camera database is actually newer than the database that was installed with the MacOS installation (I downloaded and installed 3 days ago). The camera database likely needs to be updated to include the camera models from DJI- the online version had 2 cameras listed, and the local copy for me had zero.

Neither had my camera model for the Mavic 2 Zoom, released several years ago with the same sensor on 7 models of DJI's most popular consumer drones.

[Aerial-Scouts]

I re-ran the matching step, with the camera information in the user-defined database. This time, when I ran triangulation, I received 0/668 cameras calibrated with 0 3D points. I changed to the Old SFM option and ran it again. That time I received an error message upon initialization. I'm on the 3rd triangulation now, opting to not use GPS information to help initialize, and am hopeful since it is not taking only a few minutes to throw back an error or zero message. Ten minutes in- I'll report back once this step completes.

[rhiestan]

I suggest to start with only a few images, for example 10, and experiment with the parameters to obtain a good triangulation. Only after that you can try to use more and more images. My usual test cases have less than 20 images. The software does not have a limitation, but it will become very slow and use huge amounts of RAM. Expect a 4 times slow down when you have twice the amount of images.

[Aerial-Scouts]

Download full resolution images Available until Jul 4, 2020 https://www.icloud.com/attachment/?u=https%3A%2F%2Fcvws.icloud-content.com%2FB%2FAdbKT8oE5DqFaomeeaBH52HgObkWAYq6evOsHO4egingp1GCJjlSYSSC%2F%24%7Bf%7D%3Fo%3DAon21qh8kQjwh09Y04QbDAm_QrofTcC2q748xPN2IJDo%26v%3D1%26x%3D3%26a%3DCAogDplLT9d-4DhOY-s4B-Ksa5rtpELeO11u_Pzg82_hWSUSeRCN_-iJqC4YjY_k3bEuIgEAKggByAD_V9FxslIE4Dm5FloEUmEkgmonwi9Nvl1T-c9vPVbOhwIH26smPUmurqsw8STPfqzCpn29CxjCzp8CcidPbcZe1W6LG4w7YYwmfiw8tEKiE0dvtiKzQA6gwNR93-qe01HFj5U%26e%3D1593897977%26fl%3D%26r%3D81259D57-A4E7-4705-A4C5-85113714F838-1%26k%3D%24%7Buk%7D%26ckc%3Dcom.apple.largeattachment%26ckz%3D084B9487-4F2C-41E3-B84D-F6B60FFD4DD2%26p%3D45%26s%3D_kFu6LtpmmhzhTwfuO4xaOx6HIc&uk=1URFN_cyPbGlgS8KdRonTg&f=Images.zip&sz=24915076Thanks! I’ll let this one continue in hopes it actually completes the process. The last one crashed sometime after 401 Robust Resection of camera indexes had processed. It’s gotten back up to 395, so, fingers crossed.

June 3 @ 7:40 PM - 290 indexed June 3 @ 10:28 PM - 338 indexed - 3.2 photos/hour June 4 @ 9:31 PM - 374 indexed - 3.2 photos/hour June 4 @ 4:03 PM - 395 indexed - 3.2 photos/hour

Its running steady at about 3.2 photos/hour. At that rate, I have another 91.5 hours to go. I have that many photos because its a 150 foot statue that’s being modeled. There wasn’t much I could do with the lighting unfortunately.

I’m running this on macOS Catalina with the 3 Ghz 6-Core Intel Core i5 & 64GB 2667 Mhz DDR4 RAM. The hard drive is also a SSD 1TB. I also went in to the terminal and increased the priority of Regard3D to -19 so most other processes that run in the background do not take priority.

I’m especially liking your program for this because it does not rely on CUDA. That may be more efficient for others, but NVIDIA no longer supports Apple machines, so I’m really glad you took the time and effort to make a product that doesn’t rely on a specific hardware manufacturer.

On Jun 4, 2020, at 1:36 AM, Roman Hiestand <notifications@github.com mailto:notifications@github.com> wrote:

I suggest to start with only a few images, for example 10, and experiment with the parameters to obtain a good triangulation. Only after that you can try to use more and more images. My usual test cases have less than 20 images. The software does not have a limitation, but it will become very slow and use huge amounts of RAM. Expect a 4 times slow down when you have twice the amount of images.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rhiestan/Regard3D/issues/22#issuecomment-638634477, or unsubscribe https://github.com/notifications/unsubscribe-auth/APYK5BVOFLRHMKJSHK5ZHZTRU46HHANCNFSM4EUCLU2A.