LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with a KoboldAI UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.41k stars 319 forks source link

Please Add Socket Binding Check. #781

Open Kas1o opened 3 months ago

Kas1o commented 3 months ago
def RunServerMultiThreaded(addr, port, embedded_kailite = None, embedded_kcpp_docs = None):
    global exitcounter, sslvalid
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    if args.ssl and sslvalid:
        import ssl
        certpath = os.path.abspath(args.ssl[0])
        keypath = os.path.abspath(args.ssl[1])
        context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
        context.load_cert_chain(certfile=certpath, keyfile=keypath)
        sock = context.wrap_socket(sock, server_side=True)

    sock.bind((addr, port))
    numThreads = 20
    sock.listen(numThreads)

koboldcpp.py:1380 When I launch multiple instance of koboldcpp in same port by mistake, It won't get a error.(because of the resuse)

console:

Starting Kobold API on port 5001 at http://localhost:5001/api/
Starting OpenAI Compatible API on port 5001 at http://localhost:5001/v1/
======
Please connect to custom endpoint at http://localhost:5001
LostRuins commented 3 months ago

Good point. I will check if the port is bound and add a warning message text if so.

LostRuins commented 2 months ago

Should be fixed in latest release