deepjavalibrary / djl

An Engine-Agnostic Deep Learning Framework in Java
https://djl.ai
Apache License 2.0
4.07k stars 648 forks source link

How to use djl for time series forecasting (如何使用djl进行时间序列预测) #765

Closed MaleicAcid closed 3 years ago

MaleicAcid commented 3 years ago

Hello, I want to use djl for time series forecasting. I have trained a pytorch model(Transformer) and converted it to torchscript format I have consulted the official documents and examples, but I still don’t know how to write Translator and use DJl to call the pt model for time series forecasting.

The input of my model is four tensors, and the shape is as follows: torch.Size([32, 96, 6]) torch.Size([32, 96, 6]) torch.Size([32, 72, 6]) torch.Size([32, 72, 6])

The model will predict 24 steps backwards, the shape of output which is as follows: torch.Size([24, 6]) torch.Size([24, 6]) ...... torch.Size([24, 6]) # A total of 32

Does DJl currently support calling such a time series model for forecasting? How should I write Translator and make predictions? I hope to get some help, thank you very much!


你好,我想用djl来进行时间序列的预测,我已经训练了一个pytorch模型(Transformer模型),并将其转换为torchscript格式 我查阅了官方文档和示例,但我仍然不知道如何编写Translator以及使用DJl调用pt模型进行时间序列预测

我的模型输入为四个tensor, 其形状如下: torch.Size([32, 96, 6]) torch.Size([32, 96, 6]) torch.Size([32, 72, 6]) torch.Size([32, 72, 6])

模型会向后预测24步,输出32个tensor, 其形状如下: torch.Size([24, 6]) torch.Size([24, 6]) ...... torch.Size([24, 6]) # 共32个

DJl目前支持调用这样的时间序列模型进行预测嘛?我应该如何编写Translator以及进行预测?恳请您能为我提供一些帮助,万分感谢!

stu1130 commented 3 years ago

Here is the sample translator

// I don't use input, but you should use the input to generate 4 tensors you mentioned above
public static final class MyTranslator implements Translator<NDList, NDList> {

         @Override
        public NDList processInput(TranslatorContext ctx, NDList input) {
            NDManager manager = ctx.getNDManager();
            NDArray input1 = manager.ones(new Shape(32, 96, 6));
            NDArray input2 = manager.ones(new Shape(32, 96, 6));
            NDArray input3 = manager.ones(new Shape(32, 72, 6));
            NDArray input4 = manager.ones(new Shape(32, 72, 6));
           return new NDList(input1, input2, input3, input4);
        }

        @Override
        public NDList processOutput(TranslatorContext ctx, NDList list) {
            // here you will get 32 tensors in list
           // list.size() == 32
           // list.get(0).getShape() == new Shape()
          // take stock price prediction for instance, you can return a float representing the stock price got from 32 tensors instead of the original output list
           return list;
        }

        @Override
        public Batchifier getBatchifier() {
            return null;
        }
    }

Here is the pytorch prediction jupyter notebook. Let me know if you need more details

MaleicAcid commented 3 years ago

Thank you very much for the sample code, I tried it, When I use ctx.getNDManager() to create a manager in processInput(TranslatorContext ctx, NDList input) like the sample, I get the following error:

[main] WARN ai.djl.pytorch.jni.LibUtils-No matching cuda flavor for win found: cu111.
[main] INFO ai.djl.pytorch.engine.PtEngine-Number of inter-op threads is 1
[main] INFO ai.djl.pytorch.engine.PtEngine-Number of intra-op threads is 2
Exception in thread "main" java.lang.IllegalStateException: Native resource has been release already.
at ai.djl.util.NativeResource.getHandle(NativeResource.java:51)
at ai.djl.pytorch.jni.JniUtils.getDataType(JniUtils.java:1283)
at ai.djl.pytorch.engine.PtNDArray.getDataType(PtNDArray.java:113)
at ai.djl.ndarray.NDList.toString(NDList.java:285)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.io.PrintStream.println(PrintStream.java:897)
at InformerTest.main(InformerTest.java:57)

Then I tried to create a manager in the main() function, and the model can successfully predict

public class InformerTest {
    public static void main(String[] args) {
        Path modelDir = Paths.get("./predictModelTest/model/traced_model2.pt");
        Model model = Model.newInstance("traced_model2");
        try {
            model.load(modelDir);
        } catch (IOException e) {
            e.printStackTrace();
        } catch (MalformedModelException e) {
            e.printStackTrace();
        }

        Predictor<NDList, NDList> predictor = model.newPredictor(new MyTranslator());

        NDManager manager = NDManager.newBaseManager();
        NDArray input1 = manager.ones(new Shape(1, 96, 6), DataType.FLOAT64);
        NDArray input2 = manager.ones(new Shape(1, 96, 6), DataType.FLOAT64);
        NDArray input3 = manager.ones(new Shape(1, 72, 6), DataType.FLOAT64);
        NDArray input4 = manager.ones(new Shape(1, 72, 6), DataType.FLOAT64);
        NDList input = new NDList(input1, input2, input3, input4);
        NDList predictResult = new NDList();
        long start = System.currentTimeMillis();
        try {
            predictResult = predictor.predict(input);
        } catch (TranslateException e) {
            e.printStackTrace();
        }
        long end = System.currentTimeMillis();
        System.out.println(predictResult);
        System.out.println(String.format("end informer test use %s ms", (end-start)));
    }
}
public class MyTranslator implements Translator<NDList, NDList> {
    @Override
    public NDList processInput(TranslatorContext ctx, NDList input) {
        return input;
    }

}

Run output:

[main] WARN ai.djl.pytorch.jni.LibUtils-No matching cuda flavor for win found: cu111.
[main] INFO ai.djl.pytorch.engine.PtEngine-Number of inter-op threads is 1
[main] INFO ai.djl.pytorch.engine.PtEngine-Number of intra-op threads is 2
NDList size: 1
0: (1, 24, 6) float64

end informer test use 295 ms

Why can't I use ctx.getNDManager() to create a manager in processInput(TranslatorContext ctx, NDList input), are there any details I haven't noticed?

Thank you so much for your help!


非常感谢您提供的示例代码,我尝试了一下, 当我向您一样在processInput(TranslatorContext ctx, NDList input) 中使用ctx.getNDManager()创建manager时,我得到如下错误:

[main] WARN ai.djl.pytorch.jni.LibUtils - No matching cuda flavor for win found: cu111.
[main] INFO ai.djl.pytorch.engine.PtEngine - Number of inter-op threads is 1
[main] INFO ai.djl.pytorch.engine.PtEngine - Number of intra-op threads is 2
Exception in thread "main" java.lang.IllegalStateException: Native resource has been release already.
    at ai.djl.util.NativeResource.getHandle(NativeResource.java:51)
    at ai.djl.pytorch.jni.JniUtils.getDataType(JniUtils.java:1283)
    at ai.djl.pytorch.engine.PtNDArray.getDataType(PtNDArray.java:113)
    at ai.djl.ndarray.NDList.toString(NDList.java:285)
    at java.base/java.lang.String.valueOf(String.java:2951)
    at java.base/java.io.PrintStream.println(PrintStream.java:897)
    at InformerTest.main(InformerTest.java:57)

随后我尝试在main() 函数里创建manager, 模型可以成功预测

public class InformerTest {
    public static void main(String[] args) {
        Path modelDir = Paths.get("./predictModelTest/model/traced_model2.pt");
        Model model = Model.newInstance("traced_model2");
        try {
            model.load(modelDir);
        } catch (IOException e) {
            e.printStackTrace();
        } catch (MalformedModelException e) {
            e.printStackTrace();
        }

        Predictor<NDList, NDList> predictor = model.newPredictor(new MyTranslator());

        NDManager manager = NDManager.newBaseManager();
        NDArray input1 = manager.ones(new Shape(1, 96, 6), DataType.FLOAT64);
        NDArray input2 = manager.ones(new Shape(1, 96, 6), DataType.FLOAT64);
        NDArray input3 = manager.ones(new Shape(1, 72, 6), DataType.FLOAT64);
        NDArray input4 = manager.ones(new Shape(1, 72, 6), DataType.FLOAT64);
        NDList input = new NDList(input1, input2, input3, input4);
        NDList predictResult = new NDList();
        long start = System.currentTimeMillis();
        try {
            predictResult = predictor.predict(input);
        } catch (TranslateException e) {
            e.printStackTrace();
        }
        long end = System.currentTimeMillis();
        System.out.println(predictResult);
        System.out.println(String.format("end informer test use %s ms", (end-start)));
    }
}
public class MyTranslator implements Translator<NDList, NDList> {
    @Override
    public NDList processInput(TranslatorContext ctx, NDList input) {
        return input;
    }

运行输出:

[main] WARN ai.djl.pytorch.jni.LibUtils - No matching cuda flavor for win found: cu111.
[main] INFO ai.djl.pytorch.engine.PtEngine - Number of inter-op threads is 1
[main] INFO ai.djl.pytorch.engine.PtEngine - Number of intra-op threads is 2
NDList size: 1
0 : (1, 24, 6) float64

end informer test use 295 ms

为什么不能在processInput(TranslatorContext ctx, NDList input) 中使用ctx.getNDManager()创建manager, 是我有什么细节没有注意到吗?

万分感谢您的帮助!

stu1130 commented 3 years ago

Hi @MaleicAcid, pytorch 1.7.1 doesn't provide official cuda 11.1 support. You need to downgrade to cuda 11.0. We will release latest pytorch 1.8.0 with cuda 11.1 support on 0.11.0. Stay tuned!

frankfliu commented 3 years ago

Feel free to reopen this issue if you have more questions