-
I am also interested in your impressive work! As you are from the Horizon Robotics, will you consider deploying the model on Journey 6 platform in the future?
-
-
**Title of Meeting**
Multi-deployment OPCM model
**Date, Time and Duration**
Thursday, November 7 · 3:00 – 4:00pm
Time zone: America/New_York/EST
**Link to design doc**
https://githu…
-
### Proposal to improve performance
vllm serve /workspace/model/llm/Qwen/Qwen2_5-3B-Instruct\
--host 0.0.0.0 \
--port 2017 \
--tensor-parallel-size 1 \
--gpu-memory-utilization …
-
### What is the issue?
I am unable to generate citations with a custom model deployment by simply yielding a `CITATION_GENERATION` event, nor by including citations in the `STREAM_END` event in the m…
-
### Problem you would like to solve
Setting up GRPC/ HTTP2 in company networks through firewalls, load balancers and such sometimes causes problems with customers.
### Proposed solution
Use t…
-
While initial support was added for Azure OpenAI, there are outstanding issues with the current implementation.
**Currently only one deployment/model is supported.**
* If I want to use gpt-4o via …
-
hello, can i once deploy several models for server ?
whcjb updated
2 months ago
-
Hi there,
I just want to ask that for the pruned model, how can we deploy it using TensorRT-LLM? Since the qkv dimensions in each layer are different, the model is stored using torch.save rather th…
-