Code for Full Framework Demo

[fraserprice]

I am interested in qualitative testing of your framework by demoing on my own data. I essentially want to combine Detectron, PoseNet and RootNet, and push my own image/video data through to obtain final absolute camera-centered coordinates of multiple persons’ keypoints (+ visualizations, although I see code exists for this so should be OK).

Do you have the code to demo the full pipeline? It seems no code exists for testing of your full framework, which I am sure others will also be interested in.

[mks0601]

Hi, Thanks for good suggestion. Actually, except for DetectNet, I think all pipeline is available in current released code. The DetectNet is essentially the Detectron, so you can just use that for your purpose. Which component do you think it should be provided further?

[fraserprice]

@mks0601 Thank you for your quick response!

I understand that your solution contains 3 distinct models (and that they are all available, which is great!), but it might be useful to provide a demo script to easily go from image --> 3D keypoints in one step + in a single repo. Would just save a lot of time for those who want to try your code (which seems awesome!).

Also on an unrelated point, I see that the RootNet requires a number of parameters to calculate k. I am slightly unclear as to how alpha_x and alpha_y are calculated, and cannot seem to find code to do so; could you clarify exactly what camera/other parameters are required to calculate these values?

[mks0601]

I think that is a good suggestion, but it is hard to put three different models in a one repo and provide a single demo code. I think it could take some time because currently, I'm kind of busy :( Anyway, thanks for good suggestion. I'll release some single demo code when I have time.

About the question, alpha_x and alpha_y are focal lengths in pixel unit, which are provided from dataset. At the 73th line of data/datasets.py, you can see f[0] and f[1], which are essentially alpha_x and alpha_y, respectively.

[fraserprice]

I think that is a good suggestion, but it is hard to put three different models in a one repo and provide a single demo code. I think it could take some time because currently, I'm kind of busy :( Anyway, thanks for good suggestion. I'll release some single demo code when I have time.

Completely understandable, just whenever you have the time could be a good addition :)

About the question, alpha_x and alpha_y are focal lengths in pixel unit, which are provided from dataset. At the 73th line of data/datasets.py, you can see f[0] and f[1], which are essentially alpha_x and alpha_y, respectively.

OK, so essentially unless we have focal length parameters we cannot use the RootNet. I am interested in demoing this code on images outside of given datasets (i.e. random images where focal length of camera not available), so assume this is not possible?

[mks0601]

As you can see in Section 5.2 of my paper, its prediction result is not dependent on a specific camera. You can just assume f[0] = 1500, f[1] = 1500, then the estimated absolute depth is a distance from a camera whose focal lengths are (1500,1500) and an object. I pre-computed absolute depth in a COCO dataset, which can be downloaded in README. I set focal lengths as (1500,1500) when calculating those distances.

[fraserprice]

As you can see in Section 5.2 of my paper, its prediction result is not dependent on a specific camera. You can just assume f[0] = 1500, f[1] = 1500, then the estimated absolute depth is a distance from a camera whose focal lengths are (1500,1500) and an object. I pre-computed absolute depth in a COCO dataset, which can be downloaded in README. I set focal lengths as (1500,1500) when calculating those distances.

Ah sorry I missed that!! My bad for just skimming 😬

Thanks once again for your help

[mks0601]

No problem! :smile:

[363325971]

@fraserprice so have you repreduce the result on your own data?

[abhigoku10]

@fraserprice hi so did u test it on ur own data ? how r the results is it possible to share the pipeline and resutls for reference THanks in advance