search

OpenPTrack / open_ptrack

Original open source people tracking library (now deprecated with the release of OPT v2 "Gnocchi").
http://openptrack.org/
BSD 3-Clause "New" or "Revised" License
319 stars 109 forks source link

Calibrating and Tracking Issues #137

Closed tonigialanze closed 7 years ago

tonigialanze commented 7 years ago

At the moment I am testing calibration and tracking in a 6x10m garage and am using 6 Kinect v2 sensors all connected to Jetson TK1s except the master. I managed to calibrate but I when I came to the tracking part there where a lot of split markers for just 1 user.

1) If the checkerboard does not have a white border does this affect the result of tracking or is it just more difficult for the sensors to read it?

2) I am thinking it is also an overkill using 6 cameras in that area and maybe too much sensors will confuse each other?

3) *stupid question > If i reduce the number of rows and columns by one, I guess this messes up the calibration right? I tried this since I do not have white border (I only saw this info inside this forum, maybe it should be updated for others to see, unless i missed it)

This project will be done on a wide staircase and I definitely need some tips. First thought was to adjust the Kinects v2 parallel to the stairs and finally put the checkerboard at the angle of the stairs.

Your help will be much appreciated. Thanks

jburkeucla commented 7 years ago

Toni,

I don’t think the border is critical but will let others reply. The count of squares is important. The layout should follow what’s shown here: https://github.com/OpenPTrack/open_ptrack/wiki/Camera-Network-Calibration And the number of rows/columns MUST match what’s in the camera_network YAML: https://github.com/OpenPTrack/open_ptrack/blob/2341a51459fb962e56d24c819c09bf2278828e50/opt_calibration/conf/camera_network.yaml.example It also must be on a rigid piece of material, so it can’t deform when moved.

--

Some causes of splitting:

Some amount of this can be fixed by following the calibration refinement step. https://github.com/OpenPTrack/open_ptrack/wiki/Calibration-Refinement

--

Overall: Six sensors is probably about right for your space. It’s a little tricky to get larger TK1 networks to work, and there is some impact from environmental factors. We’d be happy to chat on Skype to try to give you some pointers that are time-consuming to write out.

From: tonigialanze notifications@github.com Reply-To: OpenPTrack/open_ptrack reply@reply.github.com Date: Thursday, June 22, 2017 at 3:04 PM To: OpenPTrack/open_ptrack open_ptrack@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [OpenPTrack/open_ptrack] Some questions about Calibrating and Tracking (#137)

At the moment I am testing calibration and tracking in a 6x10m garage and am using 6 Kinect v2 sensors all connected to Jetson TK1s except the master. I managed to calibrate but I when I came to the tracking part there where a lot of split markers for just 1 user.

  1. If the checkerboard does not have a white border does this affect the result of tracking or is it just more difficult for the sensors to read it?

  2. I am thinking it is also an overkill using 6 cameras in that area and maybe too much sensors will confuse each other?

  3. *stupid question > If i reduce the number of rows and columns by one, I guess this messes up the calibration right? I tried this since I do not have white border (I only saw this info inside this forum, maybe it should be updated for others to see, unless i missed it)

This project will be done on a wide staircase and I definitely need some tips. First thought was to adjust the Kinects v2 parallel to the stairs and finally put the checkerboard at the angle of the stairs.

Your help will be much appreciated. Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenPTrack/open_ptrack/issues/137, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAZTR2-OCbAk3iX5DnwVx7Dtl_LH5XRIks5sGuT1gaJpZM4OC53c.

tonigialanze commented 7 years ago

I went ahead and the checkerboard so it has a white background.

One thing I forgot to mention was that;

When I start calibrating and put the checkerboard in front of the first Kinect v2 (I start with the Master), it's always detecting another sensor first as it is in its FOV too. Does this make calibration worse?

Also how much images is good enough for each images to get?

Since you offered it would be amazing if we could chat on skype if it's needed :) how can I add you please?

tonigialanze commented 7 years ago

Been trying to Calibrate and am having these issues:

Calibration Refinement images (totally messed up!)

refinementimage

refinement2

Calibration Refinement looks like is makinf things worse. Also the tracks have missing parts, like some areas are not being tracked AND the dots of the tracks are very far apart from each other, even the master, since I am thinking the framerate of the TK1 is causing this.

I am also getting these errors when launching refinement:

refinmenterror1

Instead of someone moving around with the checkerboard and hold it still, I am keeping it steady with a tripod stand (like this there won't be any movement). Each time I move out of the image. Does it messes up the calibration the small amount of time I move out of the way and leave the checkerboard in place? Here's an image:

rviz

This is the Calibration Status:

calibration status

I am guessing this doesn't have to mean its good since Tracking is all over the place.

Sometimes in Tracking Node Launch I am getting these errors:

Track 201 set to NORMAL
Track 202 set to NORMAL
Track 202 DELETED
[ WARN] [1498226677.891301674]: [/KinectMaster] still waiting for detection messages...
[ WARN] [1498226677.891355372]: [/KinectNode1] still waiting for detection messages...
[ WARN] [1498226677.891381901]: [/KinectNode2] still waiting for detection messages...
[ WARN] [1498226677.891396824]: [/KinectNode3] still waiting for detection messages...
[ WARN] [1498226677.891410307]: [/KinectNode4] still waiting for detection messages...
[ WARN] [1498226677.891423447]: [/KinectNode5] still waiting for detection messages...
Track 176 DELETED
[ INFO] [1498226680.895944576]: Created 203
[ INFO] [1498226681.426008235]: Created 204

Also when doing ntpq -p I am having these results:

213.251.52.234  193.62.22.74     2 u   17   64    1  156.494   38.270   0.000
 orac.technovisi 131.188.3.221    2 u   27   64    1   80.521    2.706   0.000
 one             192.168.8.100    4 u   10   16    1    0.793   10.137   0.000
 two             192.168.8.101    5 u   12   16    1    0.530   12.426   0.000
 three           192.168.8.101    5 u   11   16    1    0.550   12.556   0.000
 four            192.168.8.101    5 u    9   16    1    0.336   11.734   0.000
 five            192.168.8.101    5 u    6   16    1    0.408   12.633   0.000

and I don't know if its ok or not, since some are sowing 192.168.8.100 (master) and others 192.168.8.101. Sometimes those IPs are variate more.

Could it also be the Kinects v2 are too high above ground level (around 2.75m)?

Does the global world fixed frame stay the same after I try to redo the calibration? And how can I re-calibrate it if so?

geeez, where to start?

tonigialanze commented 7 years ago

I have now switched to two cameras so that I can manage better.

When launching the calibration refinement I am getting this error:


... logging to /home/late_interactive/.ros/log/ddeafc88-582a-11e7-b317-0862660dea0c/roslaunch-lateinteractive-X555LD-25731.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.8.100:35496/

SUMMARY
========

PARAMETERS
 * /opt_calibration_refinement/acceleration_variance: 100
 * /opt_calibration_refinement/calibration_refinement: True
 * /opt_calibration_refinement/calibration_refinement_iterations: 4
 * /opt_calibration_refinement/checkerboard/cell_height: 0.13
 * /opt_calibration_refinement/checkerboard/cell_width: 0.13
 * /opt_calibration_refinement/checkerboard/cols: 6
 * /opt_calibration_refinement/checkerboard/rows: 5
 * /opt_calibration_refinement/debug_active: True
 * /opt_calibration_refinement/detection_debug: True
 * /opt_calibration_refinement/detection_history_size: 500
 * /opt_calibration_refinement/detections_to_validate: 3
 * /opt_calibration_refinement/detector_likelihood: True
 * /opt_calibration_refinement/detector_weight: -0.25
 * /opt_calibration_refinement/extrinsic_calibration: True
 * /opt_calibration_refinement/gate_distance_probability: 0.99
 * /opt_calibration_refinement/history_pointcloud: True
 * /opt_calibration_refinement/history_size: 500
 * /opt_calibration_refinement/icp_max_correspondence_distance: 2.0
 * /opt_calibration_refinement/markers: True
 * /opt_calibration_refinement/max_detection_delay: 2.0
 * /opt_calibration_refinement/min_confidence_initialization: 4.0
 * /opt_calibration_refinement/motion_weight: 0.25
 * /opt_calibration_refinement/network: [{'pc': 'Master',...
 * /opt_calibration_refinement/orientation/vertical: False
 * /opt_calibration_refinement/position_variance_weight: 30
 * /opt_calibration_refinement/rate: 30
 * /opt_calibration_refinement/save_detection_clouds: False
 * /opt_calibration_refinement/sec_before_fake: 2.4
 * /opt_calibration_refinement/sec_before_old: 8
 * /opt_calibration_refinement/sec_remain_new: 1.2
 * /opt_calibration_refinement/time_bin_size: 0.25
 * /opt_calibration_refinement/tracking_results: True
 * /opt_calibration_refinement/velocity_in_motion_term: True
 * /opt_calibration_refinement/voxel_size: 0.06
 * /opt_calibration_refinement/world_frame_id: /world
 * /rosdistro: indigo
 * /rosversion: 1.11.21

NODES
  /
    KinectMaster_broadcaster (tf/static_transform_publisher)
    KinectMaster_broadcaster_2 (tf/static_transform_publisher)
    KinectNode5_broadcaster (tf/static_transform_publisher)
    KinectNode5_broadcaster_2 (tf/static_transform_publisher)
    opt_calibration_refinement (opt_calibration/opt_calibration_refinement)
    rviz (rviz/rviz)

ROS_MASTER_URI=http://192.168.8.100:11311/

core service [/rosout] found
process[KinectNode5_broadcaster-1]: started with pid [25749]
process[KinectNode5_broadcaster_2-2]: started with pid [25750]
process[KinectMaster_broadcaster-3]: started with pid [25752]
process[KinectMaster_broadcaster_2-4]: started with pid [25766]
process[opt_calibration_refinement-5]: started with pid [25783]
process[rviz-6]: started with pid [25789]
[ERROR] [1498238590.613817252]: transform exception: "world" passed to lookupTransform argument target_frame does not exist. 
[ERROR] [1498238590.630185680]: transform exception: "world" passed to lookupTransform argument target_frame does not exist. 
[ERROR] [1498238590.663584712]: transform exception: Lookup would require extrapolation at time 1498238590.685549388, but only time 1498238590.722524411 is in the buffer, when looking up transform from frame [KinectNode5_ir_optical_frame] to frame [world]

"world" passed to lookupTransform argument target_frame does not exist ??

Unfortunately sometimes only one of the two Kinects stop working. One of the errors when one stops working:

[TransferPool::submit] failed to submit transfer
[DepthPacketStreamParser::handleNewData] not all subsequences received 1022
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer
[TransferPool::submit] failed to submit transfer

I also noticed that during calibration refinement the tracking seems to be working better than when I actually launch the tracking.

jburkeucla commented 7 years ago

Closing this - answers in Wiki or via email.