search

moberweger / deep-prior

Fast and accurate 3D hand pose estimation from single depth images
GNU General Public License v3.0
63 stars 23 forks source link

Confusion in the use of poseNet in real time #23

Open AlmitraB opened 6 months ago

AlmitraB commented 6 months ago

Hi. I am trying to reproduce your model in Tensorflow and Keras, however, I am confused about the use of poseNet. In the first phase, you train poseNet model type = 0, in real-time you load the trained model, but you call it with type = 11, which has not been previously trained, and ask it to predict. I do not understand why we try to predict with a model that has not been trained, nor what was the purpose of training the previous one. I think maybe it is a kind of transfer learning, but I am not sure. Please help me to understand.

Looking forward to your answers

moberweger commented 6 months ago

Thanks for using DeepPrior! The type=0 model is trained to output the PCA subspace, whereas the type=11 model has the PCA components added to the network as an additional layer. Note: PCA projection can be implemented as a Dense/FC layer. Hope that helps.

On Mon, Feb 19, 2024 at 9:15 PM AlmitraB @.***> wrote:

Hi. I am trying to reproduce your model in Tensorflow and Keras, however, I am confused about the use of poseNet. In the first phase, you train poseNet model type = 0, in real-time you load the trained model, but you call it with type = 11, which has not been previously trained, and ask it to predict. I do not understand why we try to predict with a model that has not been trained, nor what was the purpose of training the previous one. I think maybe it is a kind of transfer learning, but I am not sure. Please help me to understand.

Looking forward to your answers

— Reply to this email directly, view it on GitHub https://github.com/moberweger/deep-prior/issues/23, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQSCXF2YCXMVEY34SY7HRTYUOXGFAVCNFSM6AAAAABDQAKKOKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2DGMBYGM4TANI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

AlmitraB commented 5 months ago

Hi, Thank you very much for your reply. In my eagerness to understand the code I have re-read the article completely to contrast it with your code, and more questions have arisen. This one specifically is related to the graphs that are drawn and saved after the training of the Pose Estimation network. It strikes me that the X-axis of the cost graph has values from 0 to 9000, I don't understand what kind of values it corresponds to. Similarly, in the error graph, I would like to know what the X-axis refers.

moberweger commented 5 months ago

The cost graph that you are referring to, plots the training loss (cost) over the batches. Thus, when training for 100 epochs, the network is trained using ~90 batches each 128 samples, leading to 9000 steps on the x-axis. Hope that helps you to understand the code!

On Tue, Mar 26, 2024 at 7:47 PM AlmitraB @.***> wrote:

Hi, Thank you very much for your reply. In my eagerness to understand the code I have re-read the article completely to contrast it with your code, and more questions have arisen. This one specifically is related to the graphs that are drawn and saved after the training of the Pose Estimation network. It strikes me that the X-axis of the cost graph has values from 0 to 9000, I don't understand what kind of values it corresponds to. Similarly, in the error graph, I would like to know what the X-axis refers.

— Reply to this email directly, view it on GitHub https://github.com/moberweger/deep-prior/issues/23#issuecomment-2021223453, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQSCXGCNRD4ITRB3D4LVYTY2GYCRAVCNFSM6AAAAABDQAKKOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGIZDGNBVGM . You are receiving this because you commented.Message ID: @.***>

AlmitraB commented 5 months ago

Thank you very much for your help, I get the cost graph. Could you please explain me the validation error graph ? I think in this case it is along the epochs, but the number of epochs is 100 and the horizontal axis goes up to 50. What exactly is being plotted here ?

moberweger commented 5 months ago

The validation error is the mean Euclidean error over all joints in [mm].

On Thu, Apr 11, 2024 at 9:13 PM AlmitraB @.***> wrote:

Thank you very much for your help, I get the cost graph. Could you please explain me the validation error graph ? I think in this case it is along the epochs, but the number of epochs is 100 and the horizontal axis goes up to 50. What exactly is being plotted here ?

— Reply to this email directly, view it on GitHub https://github.com/moberweger/deep-prior/issues/23#issuecomment-2050346540, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQSCXHV4EUYAEP37PHXKHTY43OEDAVCNFSM6AAAAABDQAKKOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJQGM2DMNJUGA . You are receiving this because you commented.Message ID: @.***>