Computer Vision

[shiffman]

• [X] avg point tracking
• [X] motion detection / frame differencing
• [X] blob detection algorithm
• [X] blob detection clamping improvement
• [ ] blobs with ids (persistence)
• [ ] using a library for blobs (opencv)
• [ ] opencv polygon approximation for shape detection
• [ ] blobs in the browser
• [ ] OpenTSPS and CCV
• [ ] opencv face detection
• [ ] face detection with persistence (ids)
• [ ] save faces to file
• [ ] more CV in the browser? (reference for JS: http://kylemcdonald.github.io/cv-examples/)

Great reference by @golanlevin: http://www.flong.com/texts/essays/essay_cvad/

[proggreg]

Will you be doing blob detection with the kinect?

[shiffman]

@proggreg yes, I plan to. I think this should be a part of #24

[Pixelinator]

Could you scan a given image to find a Color Palette of the most used colors in this image?

[majman]

yes please! Would love to see video exploring aspects of CV in p5 along the lines of face recognition (face swapping?), object tracking (snapchat filters), scene detection.

good stuff here: https://trackingjs.com/

[meiamsome]

Reminder from stream: Track cups as ears and replace with cartoon ears

[MatiasConTilde]

@proggreg this is my try on it for the Kinect 2:

import org.openkinect.processing.*;

Kinect2 kinect;

ArrayList <Blob> blobs;

int threshold = 1200;
PImage display;

void setup() {
  size(512, 424, P2D);

  blobs = new ArrayList();

  kinect = new Kinect2(this);
  kinect.initDepth();
  kinect.initDevice();

  display = createImage(kinect.depthWidth, kinect.depthHeight, RGB);
}

void draw() {
  background(0);

  int[] depth = kinect.getRawDepth();
  PImage bkg = kinect.getDepthImage();

  blobs.clear();

  display.loadPixels();
  for (int x = 0; x < kinect.depthWidth; x++) {
    for (int y = 0; y < kinect.depthHeight; y++) {
      int index =  x + y*kinect.depthWidth;
      int rawDepth = depth[index];
      if (rawDepth > 0 && rawDepth < threshold) {
        boolean found = false;
        for (Blob b : blobs) {
          if (b.isNear(x, y, 32)) {
            b.add(x, y);
            found = true;
          }
        }
        if (!found) blobs.add(new Blob(x, y));
        display.set(x, y, color(170, 0, 0));
      } else display.set(x, y, bkg.pixels[index]);
    }
  } 
  display.updatePixels();

  image(display, 0, 0);
  textSize(32);
  text(blobs.size(), 20, 20+32);
  for (Blob b : blobs) b.display();
}

class Blob {
  PVector min, max;

  Blob(float x, float y) {
    min = new PVector(x, y);
    max = new PVector(x, y);
  }

  void add(float x, float y) {
    min.set(min(min.x, x), min(min.y, y));
    max.set(max(max.x, x), max(max.y, y));
  }

  //Very important! This is not my algorithm, it's a modified version of this: http://openprocessing.org/sketch/20795
  boolean isNear(float circleX, float circleY, float radius) {
    float rectangleX = min.x;
    float rectangleY = min.y;
    float rectangleWidth = max.x - rectangleX;
    float rectangleHeight = max.y - rectangleY;
    float circleDistanceX = abs(circleX - rectangleX - rectangleWidth/2);
    float circleDistanceY = abs(circleY - rectangleY - rectangleHeight/2);

    if (circleDistanceX > rectangleWidth/2 + radius) return false;
    if (circleDistanceY > rectangleHeight/2 + radius) return false;
    if (circleDistanceX <= rectangleWidth/2) return true;
    if (circleDistanceY <= rectangleHeight/2) return true;

    float cornerDistanceSq = sq(circleDistanceX - rectangleWidth/2) + sq(circleDistanceY - rectangleHeight/2);

    return (cornerDistanceSq <= sq(radius));
  }

  float size() {
    return min.dist(max);
  }

  void display() {
    rectMode(CORNERS);
    noFill();
    stroke(255);
    rect(min.x, min.y, max.x, max.y);
  }
}

[spaanse]

can you do a live droste effect that replaces a colored square

[spaanse]

Blobs with distance to the nearest pixel is possible, you just forgot to add the first pixel, so it had nothing to compare possible new pixels to, the arraylist size wasn't too big, it was too small: 0 items

[shiffman]

@spaanse aaaaaack, hah, thank you. Will revisit this next time!

[spaanse]

no problem

[Pixelinator]

Skeletal Tracking might be a real cool thing to do (maybe a hard one too)

[abb00sh]

Image display; ---> PImage display;

[lfkopp]

I would love to see the license plate recognition.

[jeykech]

thanx for suggesting my suggestion with these suggestions :) 👍