janus_create_template

[nxwhite-str]

In the interest of keeping the API as simple as possible, it seems the second definition of janus_create_template, taking a single media object and returning collections of templates and tracks, is unnecessary. The intent of that method seems to be to remove the ground truth association out of the processing and have the algorithm decide how many individual subjects there are in the media. If I'm understanding that correctly, it seems the user could get that same behavior with the calls to janus_detect and the first definition of janus_create_template, and that logic seems more appropriate to implement in the test harness rather than a separate API call that each algorithm needs to implement. This pseudo-code illustrates what I think is the requested behavior:

vector<janus_template> templates;
vector<janus_track> tracks;

janus_media media;
janus_load_media(filename, media);

janus_detect(media, min_face_size, tracks);
for (auto& track: tracks) {
    // assumes janus_association no longer holds vector of tracks per issue #27
    janus_association assoc = { media, track };
    vector<janus_association> assocs =  { assoc };
    janus_template template_;
    janus_create_template(assocs, role, template_);
    templates.push_back(template_);
}

Finally, the second definition of "janus_create_template" in fact creates a collection of templates, and it would be more precise to call this "janus_create_all_templates". One may argue that creating templates from media directly rather than janus_associations allows for some sort of within media tracking and clustering. However, would not this improved functionality already be included in "janus_detect", since "janus_detect" has the same inputs as the second definition of "janus_create_template"?

So, given that it appears that second definition of "janus_create_template" can be decomposed into atomic API calls by the test harness, we recommend that it be removed.

[JordanCheney]

@paddygr: I think NIST added this version of janus_create_templates(). Could you comment on keeping / decomposing it when you have the chance?

[paddygr]

Hi, we added this function because that's what we have used in a COTS evaluation. Moreover, "janus_detect" should excel at detection of faces throughout video sequences, and so will be useful in isolation for applications involving only detection. A template generator, on the other hand, may elect to extract features from partial tracks, potentially even single frames, in which case the second form of "janus_create_template" would return minimal tracks. Because the first form of "janus_create_template" doesn't give output tracks at all, the second form is needed.

To rename to "janus_create_all_templates" assumes it should include detection of all faces. I suggest however, the function's role is not to report detection for all people in a video - rather to detect only those faces that can realistically be matched. "janus_detect" should be evaluated on its ability to find faces. "janus_create_template" should be evaluated on recognition accuracy (low FN and low FP).

[stephenrawls]

Hi Patrick, Nate,

It seems to me that perhaps this second version of janus_create_template() could be removed, and replaced with the following:

1) Run janus_cluster() on a collection of media 2) Build templates with the first version of janus_create_template() for each subject, using all media/tracks attached to that subject via janus_cluster()

I think this addresses Patrick's concern about a free-standing face detector wanting to detect all faces in an image/video, regardless of the quality of that face in terms of matching, VS his desire that the second form of janus_create_template() only include those faces that can be reliably matched. (Because the janus_cluster() function will only be able to cluster those faces it can reliably extract meaningful features for).

Thoughts?

Thanks, Stephen Rawls (ISI)

[stephenrawls]

P.S.

I hope this isn't too off-topic, but since you mentioned evaluation of janus_detect() and the second version of janus_create_template()--when can we expect info on how these things (plus clustering) will be evaluated? The release of CS3 didn't include any protocol files for face detection or clustering, for instance.

Thanks, Stephen

[paddygr]

Stephen,

Thanks for the idea. My answer reflects a drive toward an (operational) end-state. The addition of janus_cluster, I think, is about a "forensic" use-case only - finding all individuals in a large collection of seized media. It has quadratic dependency. I suspect end-users would not expect to call a cluster function in many applications. For example:

1. Suppose we have one video e.g. from a bank robbery, and wish to search its faces against some database of stills. There's no collection of media, there's no clustering to do per se.
2. A real-time application in which the video clips arrive, templates are extracted, and candidate lists returned to some relying system. This is a sequential operation without obvious clustering.

[JordanCheney]

I am going to leave both versions of janus_create_template in the API for the time being