-
I'm seeing some significant increases in ms/t when running 1.56 across multiple pascal GPUs. It works out to about a 33% speed reduction overall. 103b split across three P40s, identical 6k prompt:
…
-
This thread is dedicated to discussing the setup of the webui on AMD GPUs.
You are welcome to ask questions as well as share your experiences, tips, and insights to make the process easier for all…
-
I'm not sure if the logs are public or not since I don't see reference to them anywhere, but I noticed today that no PRs are being sent in to https://github.com/DavidGregory084/mill-tpolecat even thou…
-
**Describe the bug**
I was testing a model with the team over the Exllama Discord server and encountered a hard server crash while submiting a regular request. The server worked well for about 1 hour…
-
I tried to run it locally in Ubuntu VM on Mac M1 Max
npx fsai gen "Build a clone of Twitter called StackPrompt where people prompt instead of tweet. Allow users to follow one another and to like pr…
-
I am trying to build for iOS and i can not. I have the latest version of flutter_blue, I have created a new project and just added this plugin and when I try to build I get the error below.
I was usi…
-
Langserve is used in:
```
$ git grep -i langserve
comps/llms/summarization/tgi/llm.py:from langserve.serialization import WellKnownLCSerializer
comps/llms/summarization/tgi/requirements.txt:langse…
-
[RFC](https://github.com/rust-lang/rfcs/pull/2033).
This is an *experimental* RFC, which means that we have enough confidence in the overall direction that we're willing to land an early implementa…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
```
import os
import time
import asyncio
from llama_index.core import Settings
from…
-
There are currently problems with high load as demonstrated with [BenchmarkNet](https://github.com/nxrighthere/BenchmarkNet/wiki/Benchmark-Results). Hazel should not fall at any of the BenchmarkNet te…