-
楼主您好啊,我服务器去部署该的时候,普通启动项目,nohub java -jar 会一直报错Caused by: java.lang.UnsatisfiedLinkError: /root/.djl.ai/pytorch/1.9.1-cpu-linux-x86_64/libtorch_cpu.so: /lib64/libm.so.6: version `GLIBC_2.23' not found …
fjwnb updated
2 years ago
-
## Description
Tokens not streaming not working with rolling batch
### Expected Behavior
(what's the expected behavior?)
### Error Message
## How to Reproduce?
(If you developed your own…
-
I ran
```
client = mii.serve("mistralai/Mistral-7B-Instruct-v0.2")
response = client.generate(inputs, max_new_tokens=128, tensor_parallel=2, replica_num=2)
```
on AWS ml.g5.12xlarge with 4 GPUs…
-
**Description**
Would be nice to have quarkus-extensions for the following frameworks:
- https://github.com/awslabs/djl (Deep Java Library)
- https://github.com/eclipse/deeplearning4j
- https://…
-
## Description
What's the solutions of concurrency for AI model inference in DJL? Multithreads can access a model in the same time? Support Nvidia Triton?
Will this change the current api? How…
-
I have modified a custom target detection network and trained it to get pytorch weights, can I load this model for inference using DJL? How should I do it? Thank you!
Xhran updated
2 weeks ago
-
## Description
I followed the recipe given [here](https://docs.djl.ai/docs/serving/serving/docs/lmi/tutorials/trtllm_manual_convert_tutorial.html) to manually convert teknium/OpenHermes-2.5-Mistral-7…
-
Hello-
I've been looking into hosting an LLM on AWS Infrastructure. I am mainly looking to host Flan T5 XXL. My question is below
Inquiry: what is the recommended container for hosting Flan T5 X…
-
Hi every one. I am learning DJL(Deep java library) and I have seen [Malicious URL Detector](https://docs.djl.ai/docs/demos/malicious-url-detector/index.html) post and I'm going to try it locally, but …
-
## Description
在使用OnnxRuntime引擎不会回收jvm内存
下面是代码,已经有很多使用者提出内存不会回收,不知道你们为啥这么自信你们一直没有问题,我们已经转到python在生产环境。你们有时间看下吧
```
mport ai.djl.Device;
import ai.djl.MalformedModelException;
import ai.djl…