search

jishipp / javacv

Automatically exported from code.google.com/p/javacv
GNU General Public License v2.0
0 stars 0 forks source link

readme

=JavaCV=

==Introduction== JavaCV first provides wrappers to commonly used libraries by researchers in the field of computer vision: [http://opencv.org/ OpenCV], [http://ffmpeg.org/ FFmpeg], [http://damien.douxchamps.net/ieee1394/libdc1394/ libdc1394], [http://www.ptgrey.com/products/pgrflycapture/ PGR FlyCapture], [http://openkinect.org/ OpenKinect], [http://muonics.net/school/spring05/videoInput/ videoInput], and [http://studierstube.icg.tugraz.at/handheld_ar/artoolkitplus.php ARToolKitPlus]. The classes found under the com.googlecode.javacv.cpp package namespace expose their complete APIs. Moreover, utility classes make their functionality easier to use on the Java platform, including Android.

JavaCV also comes with hardware accelerated full-screen image display (CanvasFrame and GLCanvasFrame), easy-to-use methods to execute code in parallel on multiple cores (Parallel), user-friendly geometric and color calibration of cameras and projectors (GeometricCalibrator, ProCamGeometricCalibrator, ProCamColorCalibrator), detection and matching of feature points (ObjectFinder), a set of classes that implement direct image alignment of projector-camera systems (mainly GNImageAligner, ProjectiveTransformer, ProjectiveColorTransformer, ProCamTransformer, and ReflectanceInitializer), a blob analysis package (Blobs), as well as miscellaneous functionality in the JavaCV class. Some of these classes also have an OpenCL and OpenGL counterpart, their names ending with CL or starting with GL, i.e.: JavaCVCL, GLCanvasFrame, etc.

To learn how to use the API, since documentation currently lacks, please refer to the [#Quick_Start_for_OpenCV_and_FFmpeg] section below as well as the [http://code.google.com/p/javacv/source/browse/samples/ sample programs], including two for Android (FacePreview.java and RecordActivity.java), also found in the samples directory. You may also find it useful to refer to the source code of [http://code.google.com/p/javacv/source/browse?repo=procamcalib ProCamCalib] and [http://code.google.com/p/javacv/source/browse?repo=procamtracker ProCamTracker] as well as [http://code.google.com/p/javacv/source/browse?repo=examples Examples ported from OpenCV2 Cookbook] and the associated [http://code.google.com/p/javacv/wiki/OpenCV2_Cookbook_Examples Wiki pages].

Please keep me informed of any updates or fixes you make to the code so that I may integrate them into the next release. Thank you! And feel free to ask questions on [http://groups.google.com/group/javacv the mailing list] if you encounter any problems with the software! I am sure it is far from perfect...

==Required Software== To use JavaCV, you will need to download and install the following software:

And please make sure your Java and OpenCV have the same bitness: 32-bit and 64-bit modules do not mix under any circumstances. Further, although not always required, some functionality of JavaCV also relies on:

Finally, because we are dealing with native code, bugs can easily crash the virtual machine. Luckily, Java provides some tools to help us debug under those circumstances:

==Build Instructions== To rebuild the source code, please note that the project files were created for:

Once installed, simply call the usual mvn install command for both JavaCPP and JavaCV. By default, all the dependencies listed above are NOT required, except for OpenCV and a C++ compiler for JavaCPP, whose command line options can be passed via the javacpp.options Maven property, such as [http://code.google.com/p/javacpp/#Instructions_for_Android those required for Android]. Please refer to the comments inside the pom.xml file for further details.

==Quick Start for OpenCV and FFmpeg== Simply put all the JAR files of JavaCV (javacpp.jar, javacv.jar, and javacv-*.jar), of OpenCV (opencv-*.jar), and of FFmpeg (ffmpeg-*.jar) somewhere in your CLASSPATH. Here are some more specific instructions for common cases:

NetBeans (Java SE 6 or 7):

In the Projects window, right-click the Libraries node of your project, and select "Add JAR/Folder...".

Locate the JAR files, select them, and click OK.

Eclipse (Java SE 6 or 7):

Navigate to Project > Properties > Java Build Path > Libraries and click "Add External JARs...".

Locate the JAR files, select them, and click OK.

Eclipse (Android 2.2 or newer):

Follow the instructions on this page: http://developer.android.com/training/basics/firstapp/

Go to File > New > Folder, select your project as parent folder, type "libs/armeabi" as Folder name, and click Finish.

Copy javacpp.jar and javacv.jar into the newly created "libs" folder.

Extract all the *.so files from javacv-android-arm.jar, opencv-2.4.8-android-arm.jar, and ffmpeg-2.1.1-android-arm.jar directly into the newly created "libs/armeabi" folder, without creating any of the subdirectories found in the JAR files.

Navigate to Project > Properties > Java Build Path > Libraries and click "Add JARs...".

Select both javacpp.jar and javacv.jar from the newly created "libs" folder.

Or point your build file to the Maven repository http://maven2.javacv.googlecode.com/git/, and make sure that the library files of OpenCV and FFmpeg (*.so, *.dylib, or *.dll) can be found either in their default installation directories or in the system library PATH, which under Windows includes the current working directory. (For answers to problems frequently encountered with OpenCV on the Windows platform, please refer to [http://code.google.com/p/javacv/wiki/Windows7AndOpenCV Common issues with OpenCV under Windows 7].)

After that, the wrapper classes for OpenCV and FFmpeg can automatically access all of their C/C++ APIs:

The class definitions are basically ports to Java of the original header files in C, plus the missing functionality exposed only by the C++ API of OpenCV, and I deliberately decided to keep as much of the original syntax as possible. For example, here is a method that tries to load an image file, smooth it, and save it back to disk:

{{{ import static com.googlecode.javacv.cpp.opencv_core.; import static com.googlecode.javacv.cpp.opencv_imgproc.; import static com.googlecode.javacv.cpp.opencv_highgui.*;

public class Smoother { public static void smooth(String filename) { IplImage image = cvLoadImage(filename); if (image != null) { cvSmooth(image, image, CV_GAUSSIAN, 3); cvSaveImage(filename, image); cvReleaseImage(image); } } } }}}

JavaCV also comes with helper classes and methods on top of OpenCV and FFmpeg to facilitate their integration to the Java platform. Here is a small demo program demonstrating the most frequently useful parts:

{{{ import java.io.File; import java.net.URL; import com.googlecode.javacpp.Loader; import com.googlecode.javacv.; import com.googlecode.javacv.cpp.; import static com.googlecode.javacv.cpp.opencv_core.; import static com.googlecode.javacv.cpp.opencv_imgproc.; import static com.googlecode.javacv.cpp.opencv_calib3d.; import static com.googlecode.javacv.cpp.opencv_objdetect.;

public class Demo { public static void main(String[] args) throws Exception { String classifierName = null; if (args.length > 0) { classifierName = args[0]; } else { URL url = new URL("https://raw.github.com/Itseez/opencv/2.4/data/haarcascades/haarcascade_frontalface_alt.xml"); File file = Loader.extractResource(url, null, "classifier", ".xml"); file.deleteOnExit(); classifierName = file.getAbsolutePath(); }

    // Preload the opencv_objdetect module to work around a known bug.
    Loader.load(opencv_objdetect.class);

    // We can "cast" Pointer objects by instantiating a new object of the desired class.
    CvHaarClassifierCascade classifier = new CvHaarClassifierCascade(cvLoad(classifierName));
    if (classifier.isNull()) {
        System.err.println("Error loading classifier file \"" + classifierName + "\".");
        System.exit(1);
    }

    // The available FrameGrabber classes include OpenCVFrameGrabber (opencv_highgui),
    // DC1394FrameGrabber, FlyCaptureFrameGrabber, OpenKinectFrameGrabber,
    // PS3EyeFrameGrabber, VideoInputFrameGrabber, and FFmpegFrameGrabber.
    FrameGrabber grabber = FrameGrabber.createDefault(0);
    grabber.start();

    // FAQ about IplImage:
    // - For custom raw processing of data, getByteBuffer() returns an NIO direct
    //   buffer wrapped around the memory pointed by imageData, and under Android we can
    //   also use that Buffer with Bitmap.copyPixelsFromBuffer() and copyPixelsToBuffer().
    // - To get a BufferedImage from an IplImage, we may call getBufferedImage().
    // - The createFrom() factory method can construct an IplImage from a BufferedImage.
    // - There are also a few copy*() methods for BufferedImage<->IplImage data transfers.
    IplImage grabbedImage = grabber.grab();
    int width  = grabbedImage.width();
    int height = grabbedImage.height();
    IplImage grayImage    = IplImage.create(width, height, IPL_DEPTH_8U, 1);
    IplImage rotatedImage = grabbedImage.clone();

    // Objects allocated with a create*() or clone() factory method are automatically released
    // by the garbage collector, but may still be explicitly released by calling release().
    // You shall NOT call cvReleaseImage(), cvReleaseMemStorage(), etc. on objects allocated this way.
    CvMemStorage storage = CvMemStorage.create();

    // The OpenCVFrameRecorder class simply uses the CvVideoWriter of opencv_highgui,
    // but FFmpegFrameRecorder also exists as a more versatile alternative.
    FrameRecorder recorder = FrameRecorder.createDefault("output.avi", width, height);
    recorder.start();

    // CanvasFrame is a JFrame containing a Canvas component, which is hardware accelerated.
    // It can also switch into full-screen mode when called with a screenNumber.
    // We should also specify the relative monitor/camera response for proper gamma correction.
    CanvasFrame frame = new CanvasFrame("Some Title", CanvasFrame.getDefaultGamma()/grabber.getGamma());

    // Let's create some random 3D rotation...
    CvMat randomR = CvMat.create(3, 3), randomAxis = CvMat.create(3, 1);
    // We can easily and efficiently access the elements of CvMat objects
    // with the set of get() and put() methods.
    randomAxis.put((Math.random()-0.5)/4, (Math.random()-0.5)/4, (Math.random()-0.5)/4);
    cvRodrigues2(randomAxis, randomR, null);
    double f = (width + height)/2.0;        randomR.put(0, 2, randomR.get(0, 2)*f);
                                            randomR.put(1, 2, randomR.get(1, 2)*f);
    randomR.put(2, 0, randomR.get(2, 0)/f); randomR.put(2, 1, randomR.get(2, 1)/f);
    System.out.println(randomR);

    // We can allocate native arrays using constructors taking an integer as argument.
    CvPoint hatPoints = new CvPoint(3);

    while (frame.isVisible() && (grabbedImage = grabber.grab()) != null) {
        cvClearMemStorage(storage);

        // Let's try to detect some faces! but we need a grayscale image...
        cvCvtColor(grabbedImage, grayImage, CV_BGR2GRAY);
        CvSeq faces = cvHaarDetectObjects(grayImage, classifier, storage,
                1.1, 3, CV_HAAR_DO_CANNY_PRUNING);
        int total = faces.total();
        for (int i = 0; i < total; i++) {
            CvRect r = new CvRect(cvGetSeqElem(faces, i));
            int x = r.x(), y = r.y(), w = r.width(), h = r.height();
            cvRectangle(grabbedImage, cvPoint(x, y), cvPoint(x+w, y+h), CvScalar.RED, 1, CV_AA, 0);

            // To access or pass as argument the elements of a native array, call position() before.
            hatPoints.position(0).x(x-w/10)   .y(y-h/10);
            hatPoints.position(1).x(x+w*11/10).y(y-h/10);
            hatPoints.position(2).x(x+w/2)    .y(y-h/2);
            cvFillConvexPoly(grabbedImage, hatPoints.position(0), 3, CvScalar.GREEN, CV_AA, 0);
        }

        // Let's find some contours! but first some thresholding...
        cvThreshold(grayImage, grayImage, 64, 255, CV_THRESH_BINARY);

        // To check if an output argument is null we may call either isNull() or equals(null).
        CvSeq contour = new CvSeq(null);
        cvFindContours(grayImage, storage, contour, Loader.sizeof(CvContour.class),
                CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
        while (contour != null && !contour.isNull()) {
            if (contour.elem_size() > 0) {
                CvSeq points = cvApproxPoly(contour, Loader.sizeof(CvContour.class),
                        storage, CV_POLY_APPROX_DP, cvContourPerimeter(contour)*0.02, 0);
                cvDrawContours(grabbedImage, points, CvScalar.BLUE, CvScalar.BLUE, -1, 1, CV_AA);
            }
            contour = contour.h_next();
        }

        cvWarpPerspective(grabbedImage, rotatedImage, randomR);

        frame.showImage(rotatedImage);
        recorder.record(rotatedImage);
    }
    frame.dispose();
    recorder.stop();
    grabber.stop();
}

} }}}

==Acknowledgments== This project was conceived at the Okutomi & Tanaka Laboratory, Tokyo Institute of Technology, where I was supported for my doctoral research program by a generous scholarship from the Ministry of Education, Culture, Sports, Science and Technology (MEXT) of the Japanese Government. I extend my gratitude further to all who have reported bugs, donated code, or made suggestions for improvements!

==Changes==

===January 6, 2014 version 0.7===

===September 15, 2013 version 0.6===

===April 7, 2013 version 0.5===

===March 3, 2013 version 0.4===

===November 4, 2012 version 0.3===

===July 21, 2012 version 0.2===

===May 27, 2012 version 0.1===

===May 12, 2012===

===March 29, 2012===

===February 18, 2012===

===January 8, 2012===

===October 1, 2011===

===August 20, 2011===

===July 5, 2011===

===June 10, 2011===

===May 11, 2011===

===April 7, 2011===

===February 19, 2011===

===December 2, 2010===

===November 4, 2010===

===July 30, 2010===

===May 30, 2010===

===April 16, 2010===

===April 8, 2010===

===April 5, 2010===

===March 21, 2010===

===February 13, 2010===

===January 3, 2010===

===December 22, 2009===

===November 24, 2009===

===October 19, 2009===

===October 14, 2009===

===October 2, 2009===

===August 27, 2009===

===August 26, 2009===

===August 19, 2009===

===August 11, 2009=== Initial release

Copyright (C) 2009-2014 Samuel Audet samuel.audet@gmail.com Project site: http://code.google.com/p/javacv/

Licensed under the GNU General Public License version 2 (GPLv2) with Classpath exception. Please refer to LICENSE.txt or http://www.gnu.org/licenses/ for details.