search

nhatnm0612 / lm-server-rust

Running this code with `cargo run` and let it scan ports from localhost where llm server is hosted, if it found a port to use, it will write down the ip address, port and response time to an output folder
0 stars 0 forks source link

readme

LM-SERVER

Rust Check Python Check codecov

Description

Running this code with cargo run and let it scan ports from localhost where llm server is hosted, if it found a port to use, it will write down the ip address, port and response time to an output folder.

How to run?

Pull with git:

git pull git@github.com:nhatnm0612/lm-server-rust.git

Cd into new directory:

cd lm-server-rust

Run with cargo:

cargo build -vv
cargo run

Run test with cargo:

cargo test -vv

Run with cargo in dev environment:

ENVIRON=dev cargo run

Fake Server

Fake server in ./dev/main.py is a Python FastAPI Server mimicking OpenAI completions route. Used for development only.