Closed sweeneychris closed 7 years ago
Hi! First of all thanks, I'm glad you're able to use the code.
The depth map cutting is still kinda sketchy and I'm trying to figure out a more serious way to do this. The original idea, including the surface power, is already presented in Semerjian 2014. But in general there are two reasons why I want to do this:
What happens after L127 is broadly the following:
Does that help?
Thanks so much for a detailed insight into the depth cutting. I figured it was something similar to what you are saying, but it is good to have additional insight and confirmation. Thanks again!
Do you have any suggestions for outputting the depth maps from smvs as images? I have tried to read in the smvs-cut.mvei and smvs-S1.mvei files but I get weird output that looks incomplete and much lower resolution.
Input image:
SMVS-S1:
SMVS-cut:
This looks kind of reasonable to me, see my mail. Posting it here for reference: You’re reconstructing mostly specular materials and there’s not a lot of texture on the van. This is basically the worst case scenario for MVS as it is not really robust for this type of data. Other reasons could be missing neighbors or inaccuracies in the initial SfM.
The reason for why it looks so blocky is that the optimization works on larger patches and not on single pixels. You can optimize on smaller patches using the -o
option.
Note that the algorithm in general is tuned for high quality multi-view reconstruction and not simply generating the most complete depth map for each image. If there is a hole it is because we would rather have no geometry than incorrect geometry. This is usually different from something you would do for regular stereo.
Closing. Feel free to reopen if further questions arise.
Thanks for a seriously AWESOME library! It worked for me right out of the box and has given me really great results on most scenes I've run it on.
I'm interested in learning more about the depth map cutting (which is WIP I guess): https://github.com/flanggut/smvs/blob/master/lib/mesh_generator.cc#L24
It seems that the idea here is that you compute a consistency score by projecting points out from the depth map, computing the normals, then compare those normals to the values in your normal map. That all makes sense to me. However, I am getting a bit lost in the lines 127-141 https://github.com/flanggut/smvs/blob/master/lib/mesh_generator.cc#L127
These lines seem to be filtering the data somehow based on... the depth consistency? Could you clarify what is going on in those if statements and why, for example, you want to compare surface_power_j_j > 0.5 * surface_power (where does the 0.5 come from?!), etc.
Thanks and again, awesome work!