frankfliu
commented
3 years ago Is that possible you can share your model?
Closed hu-guanwei closed 2 years ago
frankfliu
commented
3 years ago Is that possible you can share your model?
hu-guanwei
commented
3 years ago Is that possible you can share your model?
will do shortly
hu-guanwei
commented
3 years ago Hi.
My newest Python model loading and inference can be found at:
https://aistudio.baidu.com/aistudio/projectdetail/2191399?shared=1
which gives inference result as
DJL loading and inference
import ai.djl.Device;
import ai.djl.MalformedModelException;
import ai.djl.inference.Predictor;
import ai.djl.ndarray.NDArray;
import ai.djl.ndarray.NDList;
import ai.djl.ndarray.NDManager;
import ai.djl.ndarray.types.DataType;
import ai.djl.ndarray.types.Shape;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ModelNotFoundException;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.training.util.ProgressBar;
import ai.djl.translate.TranslateException;
import java.io.IOException;
import java.nio.file.Paths;
public class Main {
public static void main(String[] args) throws MalformedModelException, ModelNotFoundException, IOException, TranslateException {
// load model
Criteria<NDList, NDList> criteria = Criteria.builder()
.setTypes(NDList.class, NDList.class)
.optEngine("PaddlePaddle")
.optModelPath(Paths.get("inference.zip"))
.optModelName("ernie")
.optDevice(Device.cpu())
.optProgress(new ProgressBar())
.build();
ZooModel<NDList, NDList> model = criteria.loadModel();
Predictor<NDList, NDList> predictor = model.newPredictor();
NDManager manager = NDManager.newBaseManager();
long[] inputIdsData = {1, 103, 1138, 422, 2};
NDArray inputIds = manager.create(inputIdsData).reshape(1, inputIdsData.length);
NDArray tokenTypeIds = manager.zeros(new Shape(1, inputIdsData.length), DataType.INT64);
System.out.println(inputIds);
System.out.println(tokenTypeIds);
NDList list = new NDList(inputIds, tokenTypeIds);
NDList res = predictor.predict(list);
System.out.println(res.get(0));
}
}which gives:
model .zip file can be found at: https://www.filemail.com/d/gkzkqngzapjsqcd
lanking520
commented
3 years ago Will take a look
lanking520
commented
3 years ago @hu-guanwei I am facing the following error during testing of your model. Could you try to remove ~/.djl.ai/cache and try to run inference again? Looks like we are not using the same model for testing
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
----------------------
Error Message Summary:
----------------------
InvalidArgumentError: The DataType of equal Op's duplicable Variable Y must be consistent. The current variable type is (int64_t), but the previous variable type is (int). (at /Users/paddle/wangye19/test/Paddle/paddle/fluid/framework/operator.cc:1520)
[operator < equal > error] public static void main(String[] args) throws IOException, MalformedModelException, TranslateException, ModelNotFoundException {
Criteria<NDList, NDList> criteria = Criteria.builder()
.setTypes(NDList.class, NDList.class)
.optEngine("PaddlePaddle")
.optModelPath(Paths.get("inference.zip"))
.optModelName("ernie")
.optDevice(Device.cpu())
.optProgress(new ProgressBar())
.build();
ZooModel<NDList, NDList> model = ModelZoo.loadModel(criteria);
Predictor<NDList, NDList> predictor = model.newPredictor();
NDManager manager = NDManager.newBaseManager();
long[] inputIdsData = {1, 103, 1138, 422, 2};
NDArray inputIds = manager.create(inputIdsData, new Shape(1, inputIdsData.length));
NDArray tokenTypeIds = manager.zeros(new Shape(1, inputIdsData.length), DataType.INT64);
System.out.println(inputIds);
System.out.println(tokenTypeIds);
NDList list = new NDList(inputIds, tokenTypeIds);
NDList res = predictor.predict(list);
System.out.println(res.get(0));
}dependencies {
implementation "ai.djl:api:0.12.0"
implementation "ai.djl.paddlepaddle:paddlepaddle-engine:0.12.0"
implementation "ai.djl.paddlepaddle:paddlepaddle-native-auto:2.0.2"
implementation "org.apache.logging.log4j:log4j-slf4j-impl:2.13.3"
}I found the root cause, checkout this fix:
https://github.com/deepjavalibrary/djl/pull/1111
After this PR merged, the result is consistent.
lanking520
commented
3 years ago The change has been applied to 0.13.0-SNAPSHOT nightly build and will be official with 0.13.0. @hu-guanwei Please try again and see if you can have consistent result
hu-guanwei
commented
3 years ago The change has been applied to 0.13.0-SNAPSHOT nightly build and will be official with 0.13.0. @hu-guanwei Please try again and see if you can have consistent result
will do shortly. thanks in advance.
frankfliu
commented
2 years ago Closing old issues. Feel free to re-open it if you still have questions.
I was trying to use djl to load paddle and do inference, but got different results from using python paddle and djl.
My codes:
gives
Saved model files are archived in a .zip file,
gives
build.gradle