Output image has circular pixel border

[hojinkangdotcom]

Hello again, as I mentioned in an early issue ticket, I'm using your OpenFrameworks library to analyze the Pixels provided by the MosaicCORE Starter Kit. Strangely the output image of the thermal camera has a circular pixel border.

Do you know how to get a standard rectangular image output?

Any help is greatly appreciated.

Best Hojin

[ryohajika]

It's very cool that you got access to the MosaicCORE kit! I'm not sure about the phenomena, since this addon relies on a C++ library to access the old generation of seek thermal cameras by maartenvds. The problem could be arisen from 1) the library or 2) drawing in OF or 3) frame resolution difference from the seek thermal compact camera and the one you use, since I see some weird gradient with dots in the image.

How do you visualize the frame data? I may be able to check what's happening by seeing the section of the code of drawing. One thing you may want to try is, retrieving a raw pixel data as CV::Mat (frame data format in OpenCV) with a method getRawCVFrame(cv::Mat& dst).

Let me know how it goes!

[hojinkangdotcom]

Thank you so much for the quick response! Just to give you the context: We are using the seek thermal camera for a Breath-controlled Light Interaction. We use the thermal camera to detect breath, OF/openCV for pixel analysis and the artnet library to output a LED-light.

This is the code:

ofApp.cpp

/*
 Project Title: 663.044.400
 Description: Breath-controlled Light Interaction using thermal images
 © Julian Hespenheide & Hojin Kang 2022
 hej@julian-h.de info@hojinkang.com
 https://julian-h.de https://hojinkang.com
 */

#include "ofApp.h"
#include "ofxArtnet.h"

//--------------------------------------------------------------
void ofApp::setup(){
    ofSetVerticalSync(true);
    ofSetFrameRate(30);

#ifdef CREATE_FLATFIELD
    cam.setCreateFlatfield(300, 80, FLATFIELD_DATA_PATH);
    cam.setup(OFX_SEEK_THERMAL_CAM_COMPACT);
#else
//    cam.setup(OFX_SEEK_THERMAL_CAM_COMPACT, FLATFIELD_DATA_PATH);
//    cam.setup(OFX_SEEK_THERMAL_CAM_COMPACT);
    cam.setup(OFX_SEEK_THERMAL_CAM_PRO);
//    cam.setup(OFX_SEEK_THERMAL_CAM_PRO, FLATFIELD_DATA_PATH);
#endif
    cam.setVerbose(false);

    img.allocate(THERMAL_WIDTH, THERMAL_HEIGHT, OF_IMAGE_COLOR);
    rawImg.allocate(THERMAL_WIDTH, THERMAL_HEIGHT, OF_IMAGE_GRAYSCALE);

    int width = 170;
    int height = 1;
    int internalformat = GL_RGB;
    sendData.allocate(width, height, internalformat);
    artnet.setup(targetIp);
}

//--------------------------------------------------------------
void ofApp::update(){
    if(cam.isInitialized()){
        if(cam.isFrameNew()){
            img.setFromPixels(cam.getVisualizePixels());
            rawImg.setFromPixels(cam.getRawPixels());
        }
    }
}

//--------------------------------------------------------------
void ofApp::draw(){
    img.draw(0, 0, img.getWidth(), img.getHeight());
    //rawImg.draw(10, 10+img.getHeight()*2, rawImg.getWidth()*2, rawImg.getHeight()*2);

    // analysieren wir unser kamera bild
    int result = analyzeImage(img);

    sendData.begin();
      ofClear(0);
      ofSetColor(result, 0, 0);
      ofDrawRectangle(0, 0, sendData.getWidth(), 1);
    sendData.end();

    // Convert the frame buffer to ofPixels
    ofPixels data;
    sendData.readToPixels(data);

    // Send the pixel data over Artnet
    artnet.sendArtnet(data);

    // report for onscreen debugging
    stringstream reportStr;
    reportStr << "Atemskulptur / FPS " << ofGetFrameRate() << endl
              << "Detected pixels: " << result << endl
              << "Input:" << img.getWidth() << "x" << img.getHeight() << endl
              << "r:" << rPixels << " / g: " << gPixels << " / b: " << bPixels << endl
              << "center pixel --> r:" << (int)centerPixel.r << " / g: " << (int)centerPixel.g << " / b: " << (int)centerPixel.b;
    ofDrawBitmapString(reportStr.str(), 350, 30);
}

//--------------------------------------------------------------
void ofApp::keyPressed(int key){
    switch (key) {
        case 'f':
            ofToggleFullscreen();
            break;

        case 's':
            ofSaveScreen(ofToString( ofGetSystemTimeMillis()) + "_thermalimage.jpg");
            break;

        default:
            break;
    }
}

//--------------------------------------------------------------
int ofApp::analyzeImage(ofImage img){

    ofPixels ourPixels = img.getPixels();
    ofColor color = 0;
    float brightness = 0;
    int counter = 0;
    rPixels = 0;
    gPixels = 0;
    bPixels = 0;
    centerPixel = ourPixels.getColor(img.getWidth()/2*img.getHeight()/2);
    for(int i = 0; i<img.getWidth()*img.getHeight(); i++){
        color = ourPixels.getColor(i);
        //ofLog() << (int)color.g;
        // ist noch weird
        // braucht eine bessere methode um zwischen farbkanälen zu unterscheiden
        //if(color.getHue() >= 75 && color.getHue() <= 95) counter++;
        if((int)color.g < 40) counter++;
        int treshold = 125;
        if((int)color.r >= treshold && (int)color.g < treshold && (int)color.b < treshold) rPixels++;
        if((int)color.r < treshold && (int)color.g >= treshold && (int)color.b < treshold) gPixels++;
        if((int)color.r < treshold && (int)color.g < treshold && (int)color.b >= treshold) bPixels++;

    }

    total = total - readings[readIndex];
    readings[readIndex] = counter;
    total = total + readings[readIndex];
    readIndex = readIndex + 1;
    if (readIndex >= numReadings) {
      // ...wrap around to the beginning:
      readIndex = 0;
    }
    average = total / numReadings;
    //println(average);
    if(counter < minPixel) minPixel = counter;
    if(counter >= maxPixel) maxPixel = counter;

    //result =
    //ofLog() << average;
    //ofLog() << "-------";
    // = ;
    //ofLog() << color;
    return ofMap(average, minPixel, maxPixel, 255, 0);
 }

//--------------------------------------------------------------
void ofApp::keyReleased(int key){

}

//--------------------------------------------------------------
void ofApp::mouseMoved(int x, int y ){

}

//--------------------------------------------------------------
void ofApp::mouseDragged(int x, int y, int button){

}

//--------------------------------------------------------------
void ofApp::mousePressed(int x, int y, int button){

}

//--------------------------------------------------------------
void ofApp::mouseReleased(int x, int y, int button){

}

//--------------------------------------------------------------
void ofApp::mouseEntered(int x, int y){

}

//--------------------------------------------------------------
void ofApp::mouseExited(int x, int y){

}

//--------------------------------------------------------------
void ofApp::windowResized(int w, int h){

}

//--------------------------------------------------------------
void ofApp::gotMessage(ofMessage msg){

}

//--------------------------------------------------------------
void ofApp::dragEvent(ofDragInfo dragInfo){

}

ofApp.h

/*
 Project Title: 663.044.400
 Description: Breath-controlled Light Interaction using thermal images
 © Julian Hespenheide & Hojin Kang 2022
 hej@julian-h.de info@hojinkang.com
 https://julian-h.de https://hojinkang.com
 */

#pragma once

#include "ofMain.h"
#include "ofxSeekThermal.h"
#include "opencv2/opencv.hpp"
#include "ofxArtnet.h"

//#define CREATE_FLATFIELD 1
#define FLATFIELD_DATA_PATH "flatfield.png"  // this data will be bundled to .app file

class ofApp : public ofBaseApp{

    public:
        void setup();
        void update();
        void draw();

        void keyPressed(int key);
        void keyReleased(int key);
        void mouseMoved(int x, int y );
        void mouseDragged(int x, int y, int button);
        void mousePressed(int x, int y, int button);
        void mouseReleased(int x, int y, int button);
        void mouseEntered(int x, int y);
        void mouseExited(int x, int y);
        void windowResized(int w, int h);
        void dragEvent(ofDragInfo dragInfo);
        void gotMessage(ofMessage msg);

        int analyzeImage(ofImage img);
        //void sendData();

        ofxSeekThermalGrabber cam;
        ofImage img;
        ofImage rawImg;

        ofFbo sendData;
        ofxArtnetSender artnet;

        string targetIp = "2.12.22.108"; // DBP Pixel-router
        int minPixel = 9999999;
        int maxPixel = 0;
        int numReadings = 20;

        int readings[20];
        int readIndex = 0;
        int total = 0;
        int average = 0;
        int counter = 0;

        int rPixels = 0;
        int gPixels = 0;
        int bPixels = 0;

       ofColor centerPixel;

};

Can't wait to hear from you!

[ryohajika]

Hi @hojinkangdotcom , Sorry for slow response (again)! I'm wondering if the resolution setting and data fetching method is alright for the mosaic core camera. A discussion thread here on the libseek-thermal C library, where someone tried to use the same library we use here a mosaic core camera and it did work at least. It seems like you are using SeekThermal Pro setting so it should be working though. I don't have access to Mosaic Core camera kit so I cannot test at my side, but can you check how many frames are incoming from the camera each second? The processing may takes some time so that the camera frame grabbing time get affected. Alternatively, you may want to try download the base library itself and try out the bundled sample commandline application to see if the result has any pixel glitch. Otherwise, the mosaic core has an official C++ SDK it seems, so we maybe able to take a look into that.

[ryohajika]

I will close this for now since I don't have get any update. We can reopen this if we need any extra work.