Method to reconcile world coordinates after each calibration or setup

[damonseeley]

A similar tracking system has a feature that we suggest could be useful: Once cameras are registered to each other two or more physical camera locations could be entered into a config file, relative to a logical origin. Then one could "Adjust World Coordinates" to reconcile new calibrations against a known space and tracking data would be consistent even after re-calibrations.

[nanodust]

maybe we can just pub a new ROS topic that is just a translation of the tracking coordinates

/tracking/app_worldspace

or something

and it's a mere linear interpolation of a rect in a new config file

[redfood]

Its not linear interpolation because there at 3 interdependent axes. (In the touch code I'm using UV mapping and ignoring height).

Regardless, its not the interpolation that is the problem, it is that every time the cameras are re calibrated the coordinate system changes so you need some way to reconcile the camera's coordinate system and the world/installation/interaction/room coordinate system.

Are the cameras' coordinates exposed? Those are what is needed.

[mmunaro]

From the wiki: https://github.com/OpenPTrack/open_ptrack/wiki/Multisensor-Calibration-and-Tracking-in-a-Distributed-Network

"The origin of the global (/world) reference frame is set at the top-left square intersection, as illustrated in the figure below. The /world reference frame saved for tracking will have the x-axis pointing towards down and the y-axis pointing towards right. Note that the top of the checkerboard can be univocally determined only if the checkerboard has an odd number of intersections in one direction (e.g., rows) and an even number in the other direction (e.g. cols). The placement of the origin can be checked in Rviz when calibrating the network."

If the checkerboard is always placed at the same position, the world reference frame should no change. If it changes, it could be that you are using a checkerboard with a non-ideal check number (see above).

The poses of the cameras are written in 'opt_calibration/launch/opt_calibration_results.launch'.

If needed, I think that changing the reference frame should be something easy to do at the application level.

[nanodust]

Indeed - this is as simple as a piece of tape on the floor, so successive calibrations use the same checkerboard position and direction. we have done this when we can.

what gets difficult is when we re-calibrate after changing the camera network.yaml - that is, removed a camera, or changed positioning- it's not always possible to use the same place on the floor for 0,0 as it's not always visible (enough) for the final ground plane calibration step.

This forces the upstream application to re-calibrate as well.

[mmunaro]

In such a scenario, the simplest option to me seems to measure the displacement between the two pieces of tape (1st calibration and 2nd calibration) along the x and y axes. Or, at every calibration, you can measure the (x and y) displacement of the current piece of tape from a reference point in the room.

[redfood]

Using the checkerboard as the origin, and a piece of tape (or pieces) of tape on the floor is the "simplest option" for the system's programmers not for the users.

If you desire the software to be adopted and used, try to think about what is easiest for your users. The goal should be make registration and/or re-registration on quick and fool proof as possible for the person doing the registration.

I've used numerous tracking systems for many installations. Because of constraints, it is not always possible to place a checkerboard in a well controlled, well oriented, and well measured position. For many reasons (both aesthetic and practical) it may also not be possible to mark the space.

Even if you can place the checkerboard exactly where you want, the orientation of the checkerboard determines the coordinate system's axes. This means that a small change in rotation will create a large change in the coordinate system for all but the smallest installation spaces. Placement of the checkerboard shouldn't require surveying equipment.

There should be a reasonably easy way to re-calibrate the system and have the coordinate system remain unchanged. One way to do this (as suggested by Damon) is to use the known positions of the cameras as reference points. There maybe other solutions to this problem too and I'm happy to help brainstorm if people are interested. However, I know from experience, the answer to this problem isn't to carefully place the checkerboard in the "correct" location.

[mmunaro]

Ok, then you assume that at least one camera is kept fixed during re-calibrations. Then, wouldn't be sufficient to use the procedure we describe here? https://github.com/OpenPTrack/open_ptrack/wiki/Multisensor-Calibration-and-Tracking-in-a-Distributed-Network#31-perform-a-new-calibration-maintaining-the-previously-estimated-floor

[jburkeucla]

No. While this function is useful, the point is that the tracking system may completely change setup, but the place being tracked will not. The ability to use arbitrary, user-defined reference points in the world space is the ultimate goal. Camera locations might be interim but may not be predictable.

From: Matteo Munaro notifications@github.com<mailto:notifications@github.com> Reply-To: OpenPTrack/open_ptrack reply@reply.github.com<mailto:reply@reply.github.com> Date: Mon, 13 Oct 2014 14:05:47 -0700 To: OpenPTrack/open_ptrack open_ptrack@noreply.github.com<mailto:open_ptrack@noreply.github.com> Subject: Re: [open_ptrack] Method to reconcile world coordinates after each calibration or setup (#10)

Ok, then you assume that at least one camera is kept fixed during re-calibrations. Then, wouldn't be sufficient to use the procedure we describe here? https://github.com/OpenPTrack/open_ptrack/wiki/Multisensor-Calibration-and-Tracking-in-a-Distributed-Network#31-perform-a-new-calibration-maintaining-the-previously-estimated-floor

— Reply to this email directly or view it on GitHubhttps://github.com/OpenPTrack/open_ptrack/issues/10#issuecomment-58954735.

[redfood]

1st I'm not wedded to using the cameras as reference points. I just think using a single checkerboard as the reference point isn't tenable because it requires too much precision on the part of the person doing the installation. And this is what was being offered as a solution.

Are you thinking of a use case where the camera's themselves are mobile? (That is something something I haven't been considering).

Assuming you are talking about stationary cameras, I do think using the cameras as reference points is particularly convent. No matter what you will have to do some measurements to re-establish your coordinate system.

I do feel that the locations of the camera's are particularly easy to establish since they are likely to be mounted and stationary but, again, I'm not wedding to this as a particular solution. I would be fine if you could put the checkerboard down if a few different known locations hand have that do the trick too. Or, I'd be happy if you could walk over to 0,0. Stand still for 15 seconds. Then way to 0,30 and stand still again. Then walk to 30,30 and stand still. Or if you could bring up a GUI and click on the ground plane in a known locations. As long its the long term solution isn't to "simply" place a single checkerboard in the exact same place every time you calibrate.

[damonseeley]

Hi All,

My suggestion for using camera locations is because these are the best derived but fixed points in a static OPT calibration. It's easy to measure to the nodal point of a camera in most circumstances. Any other points would require correlation of the real world point with observed tracking data, correct? This is what we're doing now with Flowers and it's error-prone.

BTW what I proposed can be thought of as a second, utility coordinate space. If the cameras are moving or some other condition makes this UCS unusable one could perhaps subscribe to the native OPT UCS instead.

I agree on the difficulty of using the checkerboard. I can imagine many scenarios where a recalibration is required but a specific point on the floor is obscured (eg: theater scenery is now installed at that point, which takes 4 hours to move).

Thanks, Damon

On Mon, Oct 13, 2014 at 2:29 PM, Eitan Mendelowitz <notifications@github.com

wrote:

1st I'm not wedded to using the cameras as reference points. I just think using a single checkerboard as the reference point isn't tenable because it requires too much precision on the part of the person doing the installation. And this is what was being offered as a solution.

Are you thinking of a use case where the camera's themselves are mobile? (That is something something I haven't been considering).

Assuming you are talking about stationary cameras, I do think using the cameras as reference points is particularly convent. No matter what you will have to do some measurements to re-establish your coordinate system.

I do feel that the locations of the camera's are particularly easy to establish since they are likely to be mounted and stationary but, again, I'm not wedding to this as a particular solution. I would be fine if you could put the checkerboard down if a few different known locations hand have that do the trick too. Or, I'd be happy if you could walk over to 0,0. Stand still for 15 seconds. Then way to 0,30 and stand still again. Then walk to 30,30 and stand still. Or if you could bring up a GUI and click on the ground plane in a known locations. As long its the long term solution isn't to "simply" place a single checkerboard in the exact same place every time you calibrate.

— Reply to this email directly or view it on GitHub https://github.com/OpenPTrack/open_ptrack/issues/10#issuecomment-58957724 .

[jburkeucla]

Using camera position isn't a bad idea and is a function that we could/should provide.

But, in portable deployments – not an uncommon scenario for us - the boundaries/conditions of the space will vary a lot, but not the "playing area". So it's easier to have reference points in the active area than depend on replicable camera positions for it. A simple variation of this challenge is happening now in our Indiana University deployment.

Another way to look at it is that camera positions should be irrelevant to the application coordinate space – they are used to generate tracks within a world... Using them as a geometric reference unnecessarily conflates system setup conditions with the application world space. This unnecessarily couples the deployment with the app in my mind.

I am not ready yet to think about moving cameras :) though that's potentially supported by the system.

From: Eitan Mendelowitz notifications@github.com<mailto:notifications@github.com> Reply-To: OpenPTrack/open_ptrack reply@reply.github.com<mailto:reply@reply.github.com> Date: Mon, 13 Oct 2014 14:29:43 -0700 To: OpenPTrack/open_ptrack open_ptrack@noreply.github.com<mailto:open_ptrack@noreply.github.com> Cc: Jeff Burke jburke@remap.ucla.edu<mailto:jburke@remap.ucla.edu> Subject: Re: [open_ptrack] Method to reconcile world coordinates after each calibration or setup (#10)

1st I'm not wedded to using the cameras as reference points. I just think using a single checkerboard as the reference point isn't tenable because it requires too much precision on the part of the person doing the installation. And this is what was being offered as a solution.

Are you thinking of a use case where the camera's themselves are mobile? (That is something something I haven't been considering).

Assuming you are talking about stationary cameras, I do think using the cameras as reference points is particularly convent. No matter what you will have to do some measurements to re-establish your coordinate system.

I do feel that the locations of the camera's are particularly easy to establish since they are likely to be mounted and stationary but, again, I'm not wedding to this as a particular solution. I would be fine if you could put the checkerboard down if a few different known locations hand have that do the trick too. Or, I'd be happy if you could walk over to 0,0. Stand still for 15 seconds. Then way to 0,30 and stand still again. Then walk to 30,30 and stand still. Or if you could bring up a GUI and click on the ground plane in a known locations. As long its the long term solution isn't to "simply" place a single checkerboard in the exact same place every time you calibrate.

— Reply to this email directly or view it on GitHubhttps://github.com/OpenPTrack/open_ptrack/issues/10#issuecomment-58957724.

[bassofil]

If I didn't get wrong, you're proposing to set one camera as the reference frame (i.e. the world) of the whole camera system. As for the calibration procedure, actually, this is possible. In fact the procedure estimates the poses of all the cameras with respect to the first one that "have seen" the checkerboard, only in a second phase they are converted to the last checkerboard's reference frame. The last checkerboard is used as the reference frame because of the need of estimating the floor for the detection phase. To save all the camera poses with respect to a selected camera, we need to define a novel way of saving the floor coordinates such that the detector node can read them and therefore modify the detector.

[mmunaro]

This enhancement has been provided with commit b023a89b218d96b2378c896c1ed51b990b6f4ae1. Now two points on the images can be selected and coordinates provided in order to define a new reference frame. While creating files for calibration with roslaunch opt_calibration calibration_initializer.launch a new file that launches the GUI for defining the new reference frame is created and can be launched with roslaunch opt_calibration opt_define_reference_frame.launch

Then, instructions are provided via command line.

Instructions on this procedure should be added to the Wiki.

Closing this issue.