search

6o6o / PWP3D

PWP3D, Real-time segmentation and tracking of 3D objects.
5 stars 0 forks source link

Posteriors never change #1

Open wetoo-cando opened 1 month ago

wetoo-cando commented 1 month ago

@6o6o With your shuttle example, the posterior images never change across the 100 iterations. Shouldn't the posteriors be re-evaluated at every iteration based on the contour projection with the current pose estimate? It seems the initial mask fixes the posteriors once and for all, but this cannot be correct?

6o6o commented 1 month ago

The posterior is evaluated once during the initialization with the object mask. You can update it by calling the related function on every subsequent image but the imperfections in pose alignment will eventually lead to its degradation.

The color segmentation approach provided in this implementation is more of a demonstration and is not reliable in real life scenarios as pixel intensities vary greatly during changes in lighting conditions.

wetoo-cando commented 1 month ago

Dear @6o6o thank you very much for your insights.

wetoo-cando commented 1 month ago

How did you get the results with the power drill in this video: https://www.youtube.com/watch?v=O7tr2cwzhsQ

Here you updated the posterior every iteration, right?

6o6o commented 4 days ago

In the demo app used to render the video example the posterior remains static. It is computed at initialization by segmenting the first input image using the ground truth alpha mask to create background and foreground histograms. The initial region statistics are used to process all the images.

The concept of online adaptation of the histograms is mentioned in the paper. The key idea is to update the histogram after each iteration of pose estimation, using the current estimated pose to render a new mask, and then use this mask to update the histogram. It's implemented in the library using the UpdateVarBinHistogram method overload but is not invoked at every iteration.

I include the described logic in the app but leave it intentionally disabled by default. To enable this functionality uncomment the lines below in PerseusLibTest.cpp where the mergeAlpha parameters control the rate of adaptation and the method UpdateVarBinHistogram is called in the main loop. In my experiments however it caused the deterioration of the histograms over time. The paper suggests checking for convergence before updating the histograms, but it's not implemented. Performing this check could potentially improve the results, but I have not incorporated it.

https://github.com/6o6o/PWP3D/blob/d2c3d29af4bd48a7b3e2e172370279ba0668d7c4/Application/PerseusLibTest.cpp#L157-L158 https://github.com/6o6o/PWP3D/blob/d2c3d29af4bd48a7b3e2e172370279ba0668d7c4/Application/PerseusLibTest.cpp#L182-L183