Can I use DJL NDArrays without a engine?

[carlosuc3m]

Hello I am tryng to use the NDArrays as the backbone of my program. They are quite convenient and I am using them almost as Numpy arrays. However, I have notices that they have to be used together with one of the djl engines. Is it possible to use NDArrays without having to use any of the engines as a edpendency? REgards, Carlos

[zachgk]

They can't. The basic design of DJL is that the things in the main API module like NDArray are just interfaces. The actual implementations of the interfaces would then be in the engines.

The goal of this is that different engines can have different implementations. We could have an interface built entirely in Java to work with GC. We could have an interface built in C++ if it is faster. We could have smaller ones that are CPU only or bigger ones with GPU. And, they could also have only NDArray code or all kinds of deep learning operations implemented.

[carlosuc3m]

And for example, should I be able to create an NDArray with the python engine? Ive seen that the full support is only for Pytorch mxnet and Tf

[zachgk]

It depends on the engine, because we have a few different groups of them. There are ones like PT, MX, and TF which are full engines that support all of deep learning and NDArrays. Although, for TF some of that support hasn't been fully implemented on the DJL side.

Then, we could have engines that are NDArrays without deep learning. For example, we might have a numpy engine or a cupy engine. These would probably be smaller and simpler that the full DL engines, but still great for the NDArray parts of the API.

Finally, we have ones that are basically Predictor only. This is what we do with something like XGBoost which is really more ML than DL, but we want to still be able to run it with DJL Serving or using the same DJL model zoo. I think the plan was that the Python engine would fall into this group

[carlosuc3m]

should I be able to create an NDArray simply with the API and the Python engine jars? Also, are the latest APIs (0.13.0, 0.14.0, 0.15.0 and 0.16.0) supported by Java 8? Thanks a lot for your help. Carlos

[zachgk]

The Python engine jar does not have any NDArray support inside it. It is just Predictor only. If you want NDArray operations, only MX and PT fully implement the DJL NDArray API. There might be others in the future, but those are the ones right now

All of our latest versions should support Java8.

[carlosuc3m]

Thanks for the answer. And finally, is it possible to run several engines on the same machine. I have found that as the initialization of engines only takes place whenever the first engine is loaded. If the classes corresponding to the next engine are added dinamically, the new engine never enters the ALL_ENGINES variable of the engine class

[zachgk]

If you know all of the engines you want, then you can add all of them to the classpath and it is fine having multiple engines in the classpath. If you are trying to dynamically add engines into the classpath, you can use Engine.registerEngine(...) to add them to ALL_ENGINES

[frankfliu]

We created a Java engine (was used for testing purpose) that you can achieve what you want. You can only create NDArrays but you cannot run any operators.

You just need to include it in your classpath.

See: https://github.com/deepjavalibrary/djl-serving/tree/master/engines/java.

[carlosuc3m]

Hello, thanks for you help. Is this engine available on maven, I have just seen it on nexus as a snaptshot module, not .jar file. REgards, CArlos

[carlosuc3m]

I have just tried this simple piece of code:

import ai.djl.ndarray.NDManager;

public class TestDjlJavaApi {

    public static void main(String[] args) {
        NDManager aa = NDManager.newBaseManager();
        aa.create(0);
        System.out.print(true);
    }
}

only with the dependency

<dependency>
  <groupId>ai.djl.java</groupId>
  <artifactId>java</artifactId>
  <version>0.18.0-SNAPSHOT</version>
  <type>module</type>
</dependency>

from repository:

<repository>
    <id>oss-sonatype</id>
    <name>oss-sonatype</name>
    <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
    <snapshots>
        <enabled>true</enabled>
    </snapshots>
</repository>

and I got the error saying that no engine was found.

Exception in thread "main" ai.djl.engine.EngineException: No deep learning engine found.
Please refer to https://github.com/deepjavalibrary/djl/blob/master/docs/development/troubleshooting.md for more details.
    at ai.djl.engine.Engine.getInstance(Engine.java:131)
    at ai.djl.ndarray.NDManager.newBaseManager(NDManager.java:115)
    at org.bioimageanalysis.icy.deeplearning.test.TestDJLJavaAPi.main(TestDJLJavaAPi.java:9)

Do I need another dependency or am I missing something? Regards, Carlos

[carlosuc3m]

Hello it's me again I just tried using another engine, more especifically Python 0.17.0 and it seems to work fine without any other engine. This is the dependency I added:

<dependency>
    <groupId>ai.djl.python</groupId>
    <artifactId>python</artifactId>
    <version>0.17.0</version>
</dependency>

Can I go on and keep using this engine or is there any issue? Regards, CArlos

[frankfliu]

@carlosuc3m PyTorch is a full engine, you can do most of NDArray operations with pytorch engine. And PyTorch engine is the recommended engine.

The Java engine we created is purely for testing purpose to create NDArray without depending on a full engine like PyTorch.

[carlosuc3m]

Yess, but I have tried the Python engine, and it also works. Regards, Carlos

[frankfliu]

@carlosuc3m You need pick at least one engine to use NDArray. You can use Python or Java engine, both of them are light-weighted and doesn't requires download any native library binaries.

[frankfliu]

Feel free to reopen the issue if you still have questions.