Question about realsense branch

gnoopy commented 5 years ago

I've noticed that there's realsense branch todya. I have two questions on the realsense camera relevant matter.

Firstly, are there any prerequisites to use this version for people tracking? As Kinect V2 version requires libfreenect install, I guess there might be something I have to do before installing the docker image of realsense branch to my environment. Could you share what I should do?

Lastly, Does open_ptrack_v2 support multiple realsense cameras on one sensor node which is a PC equipped with one GPU? aren't there any person who experienced this?

It would be appreciated if you share any experience related to realsense camera on Open Ptrack V2.

itabr commented 5 years ago

Regarding the prerequisites for the realsense image, it is required to have the bellow installed on the host:

Nvidia driver: version 410.78
CUDA: version 10.0
nvidia-docker 2

Other dependencies such as Intel® RealSense™ SDK 2.0 are included in the image.

Regarding the open_ptrack_v2 support for multiple realsense cameras. Currently, it is alpha testing on the dev branch and expected to be included in the next release. You are welcome to checkout the dev branch inside the container and recompile open_ptrack_v2.

gnoopy commented 5 years ago

Thanks for your kind reply. It is quite exciting to hear it from you. I will try to install realsense image , checkout dev branch in the container and recompile it.

gnoopy commented 5 years ago

Dear Samir @itabr I tried to launching single camera tracking by executing following command lines one by one on the container of single_camera_tracking from the dev branch 1) roslaunch tracking single_camera_tracking_node.launch enable_pose:=false enable_people_tracking:=true enable_object:=false 2) roslaunch detection_and_tracking_zed.launch

Since the command 1) shows the message that it cannot find Kinect 2, I tried similar approach of ZED camera. I think there might be more tips to make it work. Could you share more information to run realsense branch or dev branch on the container?

For your reference, I planned to run single camera version to check the working status of realsense camera first and then to test multi camera on one PC node.

msacco22 commented 5 years ago

I can relate with @gnoopy. I am checked out on the realsense branch, have the prerequisites for the realsense image but not managing to launch the rviz software with the intel realsense camera (d435). The steps I am taking are:

Cloning the repo git clone https://github.com/OpenPTrack/open_ptrack_docker_config.git cd open_ptrack_docker_config
Running these commands with setup_host modified to install Nvidia driver: version 410 chmod +x setup_host ./setup_host
Run the following and obtain the showed output sudo docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi

Run the container (checked out in realsense branch)

cd ~/open_ptrack_docker_config/single_camera_tracking chmod +x run_single_camera ./run_single_camera

Trying to follow instructions from here and trying to launch other .launch files (none worked so far with the intel camera) https://github.com/OpenPTrack/open_ptrack_v2/wiki/Person-Single-Camera

itabr commented 5 years ago

@gnoopy thanks. The realsense support in alpha testing is now under the dev branch. here are the instructions:

git clone -b dev https://github.com/OpenPTrack/open_ptrack_docker_config.git
cd open_ptrack_docker_config/single_camera_tracking/
./run_single_camera

Inside the container it is possible to use the following to start the single camera tracking with realsense

roslaunch tracking single_camera_tracking_node.launch enable_pose:=false enable_people_tracking:=true enable_object:=false sensor_type:=realsense

sensor_type by default is set to kinect.

dimaw24 commented 5 years ago

Thank you very much!

after I have spent about a day trying to get openptrack V2 up and running on a nvidia jetson tx2, I've finally made it. I'm using a realsense d435 camera, but the results are pretty bad. I'am the only person in the room, but after launching the single_camera_node I get lots of different tracks (up to 40 after about 2 minutes of me walking randomly in front of the camera). I've played with the parameters for for a while now, but it doesn't seem to affect the results much.

What parameters did you use?

Edit: I have only version 9.0 of cuda installed. Can this be the reason for the detection not working properly?

msacco22 commented 5 years ago

Thankyou @itabr!

I've managed to get the realsense camera/drivers configured on the machine and started the tracking launch command as instructed above. Ground plane is being detected manually successfully. Afterwards, tracking IDs and track logging information can be seen in the command shell but no visualisations on Rviz are taking place. Trying to find the 'simpleImage' addon on Rviz and I it's not there (only 'Image' add-on is present with no signal being detected).

1) Could it be that there are different versions of Rviz visualisation? 2) Do I have to perform any calibration if I'm just using 1 realsense D435 camera? 3) Can both detection/tracking and visualisation happen on the same node of which the camera is connected to?

msacco22 commented 5 years ago

Update - managed to get the Rviz and D435 working properly.

The problem was a wrong Rviz configuration being loaded automatically!

dimaw24 commented 5 years ago

Update - managed to get the Rviz and D435 working properly.

The problem was a wrong Rviz configuration being loaded automatically!

did you change any configurations and at what height is your camera mounted?

msacco22 commented 5 years ago

@dimaw24 I changed the configuration from the drop-down of Rviz to handle single-camera setup. Camera was mounted at 2m from the ground. Will be building the solution from scratch again as I only tested the solution on docker.

nathansomavarapu commented 5 years ago

@dimaw24 I was wondering if/how you fixed the issue of generating false tracks with the realsense camera if you wouldn't mind sharing?

msacco22 commented 5 years ago

@dimaw24 are there any guidelines similar to what you provided above with regards to launching a multi-camera set-up using realsense cameras?

siehlema commented 5 years ago

I also want to use the Realsense Camera (D435 for skeleton tracking) and I got it working, but the results are - like stated in some comments above - pretty bad. So I also want to know, am I doing something wrong or is the Realsense Camera just not working good with OPT?

I found out, that the skeleton tracking gets better, if you move further away from the camera (>4m)

jburkeucla commented 5 years ago

The issues you are seeing are consistent with our early tests, too. We are doing more evaluation later this summer and will update this thread if we see any better results.

From: Martin Siehler notifications@github.com Reply-To: OpenPTrack/open_ptrack_v2 reply@reply.github.com Date: Tuesday, June 18, 2019 at 8:37 AM To: OpenPTrack/open_ptrack_v2 open_ptrack_v2@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [OpenPTrack/open_ptrack_v2] Question about realsense branch (#51)

I also want to use the Realsense Camera (for skeleton tracking) and I got it working, but the results are - like stated in some comments above - pretty bad. So I also want to know, am I doing something wrong or is the Realsense Camera just not working good with OPT?

I found out, that the skeleton tracking gets better, if you move further away from the camera (>4m)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenPTrack/open_ptrack_v2/issues/51?email_source=notifications&email_token=AADFGRZMPFEBYPTUPG4SB6TP3D6K5A5CNFSM4GSY3ERKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX7BN6Q#issuecomment-503191290, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AADFGR7H23AED33OUSLYCPDP3D6K5ANCNFSM4GSY3ERA.

msacco22 commented 5 years ago

Hi,

Just checking if there were any updates on this.

I have been experimenting and debugging open_ptrack with D435. I believe that if no post-processing is done - the "wavy" point-cloud generated from the camera poses great threat to "adequate" performance.

Interested about any other find outs.

jburkeucla commented 4 years ago

We're not sure of the source of the "waviness" but it seems intrinsic to the imager. We'll leave this open but have no solution yet.

OpenPTrack / open_ptrack_v2

Question about realsense branch #51