[Good First Issue]: [JAVA API] Async demo for Java API 2.0

likholat commented 8 months ago

Context

OpenVINO Runtime supports inference in either synchronous or asynchronous mode. The key advantage of the Async API is that when a device is busy with inference, the application can perform other tasks in parallel (for example, populating inputs or scheduling other requests) rather than wait for the current inference to complete first.

What needs to be done?

We had async inference support and async demo for Java API 1.0 before, now the async mode should be supported for OV Java API 2.0.

It's necessary to:

[ ] Add Face Detection Async Java Sample for OV API 2.0 based on the deprecated sample and Async Mode.
[ ] If needed, implement missing Java bindings of C++ methods required for OV asynchronous demo via the Java Native Interface (JNI). Such methods as start_async() and wait() have already been added for Java API, but you will probably find more unsupported OV Java methods while developing demo.

Here is an example of OV Java method implementation for infer() method of InferRequest class: Java part of implementation:

// openvino_contrib/blob/master/modules/java_api/src/main/java/org/intel/openvino/InferRequest.java

package org.intel.openvino;

public class InferRequest extends Wrapper {
    ...

    public void infer() {
        Infer(nativeObj);
    }

    private static native void Infer(long addr);
}

CPP part of method implementation:

//openvino_contrib/blob/master/modules/java_api/src/main/cpp/openvino_java.hpp

...
JNIEXPORT void JNICALL Java_org_intel_openvino_InferRequest_Infer(JNIEnv *, jobject, jlong);

// openvino_contrib/blob/master/modules/java_api/src/main/cpp/infer_request.cpp

#include <jni.h>
#include "openvino/openvino.hpp"
#include "openvino_java.hpp"
#include "jni_common.hpp"

using namespace ov;
JNIEXPORT void JNICALL Java_org_intel_openvino_InferRequest_Infer(JNIEnv *env, jobject obj, jlong addr)
{
    JNI_METHOD("Infer",
        InferRequest *infer_request = (InferRequest *)addr;
        infer_request->infer();)
}
...

Example Pull Requests

[JAVA_API][GSOC] Add Java API bindings

Resources

What is OpenVINO toolkit?
OpenVINO Contrib repo
How to build OpenVINO with extra modules
Java bindings for OpenVINO
OV Java Samples
Async Mode
Java Programming Tutorial - Java Native Interface (JNI)
Java async demo for depricated OV API 1.0
OpenVINO™ API 2.0 Transition Guide
Intel DevHub Discord channel - engage in discussions, ask questions and talk to OpenVINO developers

Contact points

@likholat

Ticket

No response

AishaniPandey commented 8 months ago

.take

AishaniPandey commented 8 months ago

Hello. I wish to take this issue in preparation for GSoc 2024

AishaniPandey commented 8 months ago

.take

AishaniPandey commented 8 months ago

@likholat could you kindly add me as an assignee for this issue. Thanks :)

AishaniPandey commented 7 months ago

Implementation Plan for Asynchronous Java Sample in OpenVINO Java API 2.0

Subject: Accelerated Implementation of Async Java Sample for OpenVINO Java API 2.0

Introduction This plan aims to implement a Face Detection Asynchronous Java Sample for OpenVINO Java API 2.0 within a tight timeline. The focus is on integrating necessary Java bindings and developing a robust async demo.

Timezone and Availability My timezone is CET (GMT+1). I can dedicate 3 hours daily to this project.

Objective To create an asynchronous Java sample for the OpenVINO Java API 2.0, ensuring compatibility and efficiency, and to add any missing Java bindings for C++ methods via JNI.

Expedited Implementation Steps and Timeline

Initial Setup and Analysis (Day 1)

Set up the development environment.
Review existing async demos and understand the upgrades in Java API 2.0.

Development of Java Bindings (Day 2-3)

Implement Java bindings for essential C++ methods (like start_async, wait).
Test bindings for functionality and compatibility.

Asynchronous Java Sample Development (Day 4-5)

Develop the core structure of the Face Detection Async Java Sample.
Integrate and test the new Java bindings within the sample.

Sample Refinement and Testing (Day 6)

Optimize the sample for performance and adherence to coding standards.
Conduct comprehensive testing to ensure proper functionality.

Documentation, Review, and Pull Request (Day 7)

Update documentation to reflect new features and instructions.
Finalize the sample and prepare for submission.
Create and submit a pull request with a detailed implementation summary.

Buffer Period (Day 8-10)

Use this time to address feedback, make adjustments, or conduct additional testing if needed.

Conclusion This condensed plan aims for the swift and efficient integration of an asynchronous Java sample in the OpenVINO Java API 2.0. I'm committed to maintaining quality and functionality within this expedited timeframe.

Please feel free to reach out for any clarifications or discussions regarding this plan.

Best regards, Aishani Pandey pandeyaishani@gmail.com

likholat commented 7 months ago

Hi @AishaniPandey, thank you for the detailed integration plan, it looks good to me! Please note that this issue can't be a GSoc project, but can be done as a preparation step for this program.

likholat commented 7 months ago

Hi @AishaniPandey, do you have any update on this task? Please let me know if you have questions, you can also create a draft PR to discuss.

AishaniPandey commented 7 months ago

Hi likholat,

I was occupied with college midterm assessments from the past few days and I missed your message.

I will be back on track with an updated plan within next few days.

HarshG1308 commented 7 months ago

Hello Likholat,

I could resolve this issue.

likholat commented 7 months ago

Hi @AishaniPandey, please let me know your progress on this issue. I'll assign it to @HarshG1308 if you don't have enough time to resolve the issue

AishaniPandey commented 7 months ago

Yes ma'am. There has not been much update ....I had started to work on it but couldn't devote much time due to health issues. You can assign it to a different person. I apologise once again but I am unable to work given my current health. Wishing you luck. Thanks.

On Thu, 15 Feb 2024 at 18:26, Anna Likholat @.***> wrote:

Hi @AishaniPandey https://github.com/AishaniPandey, please let me know your progress on this issue. I'll assign it to @HarshG1308 https://github.com/HarshG1308 if you don't have enough time to resolve the issue

— Reply to this email directly, view it on GitHub https://github.com/openvinotoolkit/openvino_contrib/issues/834#issuecomment-1946046484, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWJBNOQFZJ2HMR2KKQCH7UTYTYAX3AVCNFSM6AAAAABB6ZYJUGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBWGA2DMNBYGQ . You are receiving this because you were mentioned.Message ID: @.***>

likholat commented 7 months ago

@AishaniPandey thank you for the reply. Get well! @HarshG1308 assigned this issue to you

HarshG1308 commented 6 months ago

Unfortunately, I won't be able to work on this at the moment as I am currently swamped with another project. Thank you for understanding."

likholat commented 6 months ago

Unfortunately, I won't be able to work on this at the moment as I am currently swamped with another project. Thank you for understanding."

Thank you for the reply!

MonalSD commented 6 months ago

.take

Pranshu-S commented 6 months ago

.take

github-actions[bot] commented 6 months ago

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

likholat commented 5 months ago

Hello @Pranshu-S, thanks for your interest! Do you have an update on this task? Please let me know if you have any questions. You can also create a draft PR to discuss.

Pranshu-S commented 5 months ago

Hi @likholat Thanks for reaching out. I have gotten the gist of what is to be done on the issue. I'll raise a draft PR by tomorrow and follow up with testings.

Thanks!

p-wysocki commented 5 months ago

Hello @Pranshu-S, are you still working on this?

SerenityCodes commented 2 weeks ago

.take

github-actions[bot] commented 2 weeks ago

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

openvinotoolkit / openvino_contrib