I have a doubt about the hand detection algorithm.

[lxz1104]

file: src/util/handdetector.py function: def detect(self, size=(250, 250, 250), doHandSize=True) -> com, size line: 606 The main code snippet is as follows:

 M = cv2.moments(contours[c])
cx = int(numpy.rint(M['m10']/M['m00']))
cy = int(numpy.rint(M['m01']/M['m00']))

# crop
xstart = int(max(cx-100, 0))
xend = int(min(cx+100, self.dpt.shape[1]-1))
ystart = int(max(cy-100, 0))
yend = int(min(cy+100, self.dpt.shape[0]-1))

cropped = self.dpt[ystart:yend, xstart:xend].copy()
cropped[cropped < i*dz + self.minDepth] = 0.
cropped[cropped > (i+1)*dz + self.minDepth] = 0.
com = self.calculateCoM(cropped)
if numpy.allclose(com, 0.):
    com[2] = cropped[cropped.shape[0]//2, cropped.shape[1]//2]
com[0] += xstart
com[1] += ystart

I donn't know why com[0] += xstart and com[1] += ystart

[moberweger]

The center of mass (com) is first initialized on the full image (cx,cy), and then refined on a crop around this location. This only takes a small patch around the initial (cx, cy), thus the offset (xstart, xend) is added afterwards. Optionally, a CNN is used to predict this offset (xstart, ystart).

[lxz1104]

Thanks!