-
./scripts/infer_seq.sh
this script is not there
-
Once the inference API has all the capabilities and feature we need, and the inference stack connector is usable for GenAI in Kibana, we should add support for the inference connector to the inference…
-
I have meet this question:
when I run demo to inference two pictures but the result is an all black picture ,I dont know whats wrong with it.
 on Openvino, and the model takes these as inputs:
input_ids
attention_mask
words_mask
text_lengths
span_idx
span…
-
### Is your feature request related to a problem? Please describe.
Models greater than the GPU memory capacity cannot be currently run in inference, whilst parallel implementations in training exist.…
-
Is there any normal inference code for trajectory controls rather than a gradio demo?
-
Hey there,
Love this work! I'm also a big fan of using pre-LLM style models for problems which require far less compute resources.
Do you have any data on the inference time per sample?
E.g…
-
Can you provide a way to inference it with onnx?
This way we'll be able to use the GPU and much less dependencies and also it will be easier to adapt it to other languages such as Rust.
Thanks!