search

Simple-Robotics / cosypose

Code for "CosyPose: Consistent multi-view multi-object 6D pose estimation", ECCV 2020.
MIT License
69 stars 15 forks source link

A question of custom usage of CosyPose for robotics #15

Open shoryamalani opened 1 year ago

shoryamalani commented 1 year ago

Hey, I was working with a team hoping to make some pose estimations for two very simple objects. When we realized we needed to find the orientation of these objects, we found this and how cosypose is quite time efficient. Unfortunately, if this is an option for us, we would have to run this on an orange pi (http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-5.html) which is powerful but has limitations.

For starters, as far as I am aware, an orange pi does not support CUDA. So I guess for my first question is, can this be run on decently powerful but possibly without GPU acceleration with a live feed of the camera? We only need the Pose of the image and care quite little for the visualization of the box around the object afterward.

I am happy to update this repository with more CPU support if that is possible, however, I am not sure where I would even start going about making changes like this and was wondering where you would start.

I also was wondering if the detection of the object is significantly slower than the following detections as it may make this difficult to use with a fast-moving robot.

So in summary: Can this run on a lower-power silicon without CUDA while only having to get the float values for 6d pose, without visualization, for multiple objects, at around 5 fps (whatever resolution)? Lastly, how accurate would the transform from the pose to the camera be, or really how accurate would the pose be for an object that is about 12 inches by 6 inches by 6 inches at a distance of 4-6 feet?

I absolutely love python and am not afraid to get my hands dirty if I know where to start.

Thank you in advance

Toefinder commented 11 months ago

Hi @shoryamalani, I am looking into a similar application case for my personal project, where the computer only has CPU. Since the time you posted this issue till now, have you found out whether it's possible yet?

shoryamalani commented 11 months ago

It is realistically too slow to be used real-time with just a CPU. I ended up finding different solutions that could give me different bits of information that I could then collate to get to the same outcome.

Toefinder commented 11 months ago

By "different solutions", do you mean something else other than CosyPose, or some hacks on CosyPose to make it run on your hardware?

shoryamalani commented 11 months ago

I meant more of moving to having multiple cameras do two d object detection with depth and inferring some information based on the fact that there are only two stable orientations of the object I was looking at. Was it as good, no, but it did work in real time.