-
I am using keybert for a topic extraction task and I have the need of making the system content as a variable so I can change accordingly based on my task. Here below I represent what I mean.
**Cur…
-
For enforcing model to give response in json format, I am using ExLlamaV2TokenEnforcerFilter and ExLlamaV2PrefixFilter classes and appending to to filters list and passing as filters for generating ou…
-
Didn't find the code for DDIM inversion, but DDIM sampling directly from noise, why is that?
@torch.no_grad()
def __call__(
self,
prompt: Union[str, List[str]],
mot…
-
### Your current environment
python3.11 vllm4.1 torch2.21-cu118
### 🐛 Describe the bug
here is my log with vllm, when inference gemma7b ,it shows 6 logs for one request which spend 30s ,…
-
### What behavior of the library made you think about the improvement?
It might be nice to have the option to have a `non_strict` mode when doing generations for large batches of data. This could be …
-
My simple inference script is failing when calling wrapper.merge_to() with Flux Dev as the base model.
```
2024-09-21 19:27:53|[LyCORIS]-INFO: Loading Modules from state dict...
2024-09-21 19:27:…
-
**Is your feature request related to a problem? Please describe.**
There is a huge delay before starting the inference and once the 4th step is complete and there is no callback for that and it feels…
-
I imagine a textarea where the user can enter a prompt using a format like this:
```
masterpiece of a street corner in %atlas_city, in the style of Julia Sanfrancisco, summer, %time_of_day
```
T…
-
- [ ] I'd be willing to implement this feature ([contributing guide](https://github.com/nrwl/nx/blob/master/CONTRIBUTING.md))
## Description
[Enquirer](https://github.com/enquirer/enquirer…
-
# Summary
I am trying to pass command-line options to my plop script, but it instead provides an error message:
> /usr/local/Cellar/node/22.9.0_1/bin/node: bad option: --plopfile=plopfile.ts
ELI…