-
I'm running vllm with:
- model = `meta-llama/Meta-Llama-3-8B-Instruct`
- DJL image = `763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.28.0-lmi10.0.0-cu124`
- the env of the deployed…
-
I have a pydantic model that has a field like so:
```python
class MySchemaModel(BaseModel):
my_mapping: Dict[str, str] = Field(default={})
DEFAULT = MySchemaModel()
class Model(models.M…
-
I refined llama3.1 8b bnb 4bits according to your recommendations with my own train+eval dataset and saved as merged 16 bits. I now want to create an inference by loading the 16b merged model and usin…
-
See test NonSharedPrimitiveCollectionsQueryCosmosTest.Array_of_byte_array, where we incorrectly persist a `byte[][]` as an array of arrays (
`[[1, 2], [1, 2]]`) instead of an array of base64 strings. …
-
@tsalo and I have been working on getting the derivatives of our dMRI post-processing workflows into BEP16 compliance. We'd appreciate some thoughts and comments on the file names we're producing. Per…
-
llama-omni) Ubuntu@0008-dsm-prxmx30009:~/TestTwo/LLaMA-Omni$ python -m omni_speech.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000/ --port 40000 --worker http://localhost:40000/ …
-
This is a write up of a plan I've had for a long time to improve the performance of Pydantic/pydantic-core by another big step, hopefully up to 5x-10x in some cases.
# Summary
The core idea is t…
-
### Willingness to contribute
Yes. I can contribute this feature independently.
### Proposal Summary
Langchain flavor does not permit to log a chain in MLflow with a custom LLM model wrapper (llm t…
-
### What happened?
For most data sources (e.g. Static), we can map the annotation details (e.g. title and text) to fields. This makes no sense for builtin data source, hence with #51256 the configu…
-
**Environment (please complete the following information):**
- OS: any
- Browser any
- Version 23.3.0
**Describe the bug**
ex:
curl --location --request POST 'http://127.0.0.1:8080/tes…