-
Do anyone tried to convert trained .h5 model to Tensorflow Lite or Tensorflow Serving?
I got some difficulties due to some python functions such as 'yolo_loss' and 'yolo_head'.
-
I am trying to use Llama-2-70b-chat-hf as zero-shot text classifier for my datasets. Here is my setups.
1. vLLM + Llama-2-70b-chat-hf
I used vLLM as my inference engine as run it with:
```
pyt…
-
### Your current environment
```text
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
GCC ve…
-
### Your current environment
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: **** Group Enterprise Linux Server 7.2 (Pala…
-
Hi! I am trying to convert HFNet to ONNX, then convert it to TensorRT, I found when running
```
python frozen2onnx.py
```
It shows
```
Tensorflow op [pred/descriptor_sampling/resampler/Resampl…
-
Sometimes the error mentioned in the title occurs, after which the server stops processing images and keeps giving this error.
If the server is manually restarted, it starts working correctly, conti…
iG8R updated
2 months ago
-
Reference:https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/fastchat_quickstart.md
While testing Fastchat with below steps, I got connectin error as below attachment pic…
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
#####i start serving with this script :
#####bash
export OLLAMA_NUM_GPU=999
export no_proxy=localhost,127.0.0.1
export ZES_ENABLE_SYSMAN=1
source /opt/intel/oneapi/setvars.sh
./ollama serve
…
-
In the benchmark comparison results, could we add a comparison with VLLM to see the acceleration effects?