Iris/pupil detection only

assuming I already have a code that performs 68-point facial landmark detection using dlib, is it possible to extract code that performs solely iris/pupil detection, i.e.

You can run the eye model fitting as a stand alone module. That will fit a global eye model first (eyelids, create, iris + pupil) followed by iris refinement using the eyelid contours for occlusion maps. The global model improves stability of the iris estimate, even if you don't care about eyelids, etc.

There is a sample that can fit eye models on single eye crops (4x3 aspect ratio) here:

https://github.com/elucideye/drishti/blob/master/src/app/eye/eye.cpp

There is also a sample of a minimal eye model fitting application using the public/exported API (no 3rdparty types in the API) in the following link, which also illustrates how to use the hunter package:

https://github.com/elucideye/drishti_hunter_test/blob/master/src/app/eye/drishti-eye-test.cpp

That sounds closest to what you are looking for. The current public eye models refine an iris ellipse model using an affine cascaded pose regressor built around xgboost, with occlusion maps provided by the initial global eye model. In the posted models, an initial approximate pupil ellipse is provided from the global model only. The code supports pupil refinement (CPR, etc), but the public models don't currently include such a step, since the pupil isn't often visible for most of the selfie data I've worked with, and it adds to the model size. I can probably make one available. The approximate pupils from the global fit are often pretty good.

These CMake options should be sufficient to disable builds for all of the face related modules if you only want to fit eye models:

    DRISHTI_BUILD_ACF=OFF
    DRISHTI_BUILD_FACE=OFF
    DRISHTI_BUILD_HCI=OFF
    DRISHTI_BUILD_OGLES_GPGPU=OFF

Background: Most of that stuff was added to support real time processing on mobile devices. Those flags should disable all of it. I've migrated most of that stuff out of the repository to hunter packages.

If you have 68 point ibug landmarks you can generate suitable eye crops for processing using crop geometry based on eye centers as shown here (be sure to specify the left or right side eye appropriately):

https://github.com/elucideye/drishti/blob/40e3370e613ef5933558ce449d0d47c812b6fe79/src/lib/drishti/face/Face.cpp#L38-L52

bool FaceModel::getEyeRegions(cv::Rect2f& eyeR, cv::Rect2f& eyeL, float scale) const
{
    bool okay = false;
    if ((eyeLeft.size() && eyeRight.size()) || (eyeRightCenter.has && eyeLeftCenter.has))
    {
        okay = true;
        cv::Point2f pL = getEyeLeftCenter();
        cv::Point2f pR = getEyeRightCenter();
        cv::Point2f ratio(1.0, 3.0 / 4.0);
        cv::Point2f diag = ratio * cv::norm(pL - pR) * scale * 0.5f;
        eyeL = cv::Rect2f((pL - diag), (pL + diag));
        eyeR = cv::Rect2f((pR - diag), (pR + diag));
    }
    return okay;
}

I really like your project, but it is way too heavy for me -- I am already using dlib for facial landmark detection and tons of other stuff, I already have my own Qt VideoOutput filter

Thanks. The QT stuff is just a sample project, which can be disabled. None of that is actually used in the libs. You can build the lib for eye model fitting only if you want. It will be easiest to use this project through hunter. If you aren't using hunter, you can build the lib with hunter and export a single shared lib for integration. I moved the ACF stuff to a separate module recently. I guess the same thing could be done for the CPR ellipse regression stuff.

I hope that helps.

elucideye / drishti

Iris/pupil detection only #612