Unable to Connect over Local Network.

[someonenameguy]

THe Config file. kcpps:

{"model": null, "model_param": "C:/Users/kevan/Documents/DataSet/Koboldcpp/LLaMA2-13B-Psyfighter2.Q6_K.gguf", "port": 5001, "port_param": 7070, "host": "", "launch": false, "config": null, "threads": 5, "usecublas": ["normal", "0", "mmq"], "usevulkan": null, "useclblast": null, "noblas": false, "contextsize": 4096, "gpulayers": 8, "tensor_split": null, "ropeconfig": [0.0, 10000.0], "blasbatchsize": 64, "blasthreads": null, "lora": null, "noshift": false, "nommap": false, "usemlock": true, "noavx2": false, "debugmode": 0, "skiplauncher": false, "onready": "", "benchmark": null, "multiuser": 1, "remotetunnel": false, "highpriority": false, "foreground": true, "preloadstory": null, "quiet": false, "ssl": ["C:\\Users\\\Documents\\DataSet\\MyCertificate.crt", "C:\\Users\\\Documents\\DataSet\\MyKey.key"], "nocertify": true, "mmproj": null, "password": "Test1234", "ignoremissing": false, "chatcompletionsadapter": null, "flashattention": true, "forceversion": 0, "smartcontext": false, "hordeconfig": null, "sdconfig": null, "hordemodelname": "", "hordeworkername": "", "hordekey": "", "hordemaxctx": 0, "hordegenlen": 0, "sdmodel": "", "sdthreads": 5, "sdquant": false, "sdclamped": false}

Laptop Output:

Load Text Model OK: True
Embedded Kobold Lite loaded.
Embedded API docs loaded.
Starting Kobold API on port 7070 at https://localhost:7070/api/
Starting OpenAI Compatible API on port 7070 at https://localhost:7070/v1/
======
Please connect to custom endpoint at https://localhost:7070

When I try to connect on my phone use firefox at https://192.168.1.224:7070/ the page is stuck on loading, till close koboldcpp, which then it shows timeout(Makes Sense). I tried with and without password and SSL.

Opening localhost page on the laptop works fine.

I have open hardware monitor on the laptop and I am able to connect to its web ui on 8086 on my phone.

Both devices are on the the same wifi. Other than that I did set port-forwarding for 192.168.1.224 from 7000-7999 wan and lan for other stuff but 7070 isn't being used by other programs

IpConfig incase it's useful.

Windows IP Configuration

Ethernet adapter Ethernet:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter vEthernet (Wi-Fi):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::bdc7:8aea:a2f6:4813%37
   IPv4 Address. . . . . . . . . . . : 192.168.48.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . :

Ethernet adapter vEthernet (Default Switch):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::be35:d7c8:33fd:7e5a%44
   IPv4 Address. . . . . . . . . . . : 172.20.144.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . :

Wireless LAN adapter Local Area Connection* 1:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Local Area Connection* 2:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Wi-Fi:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::9a2f:740:7679:4729%20
   IPv4 Address. . . . . . . . . . . : 192.168.1.224
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.1.254

Ethernet adapter Bluetooth Network Connection:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter vEthernet (Ethernet):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::91c8:2ec7:55dd:b7fa%49
   IPv4 Address. . . . . . . . . . . : 172.21.192.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . :

[LostRuins]

First, try troubleshoot with different settings.

1. Does it work without https? Try without the SSL cert instead.
2. Does it work locally? Try to connect on the same device first via localhost instead.

[someonenameguy]

(I Edited the post a few times, I tried these step already, if you are replying from email.)

1. I tried, no luck.

2. Works fine.

On Mon, 20 May 2024, 00:44 LostRuins Concedo, @.***> wrote:

First, try troubleshoot with different settings.

1. Does it work without https? Try without the SSL cert instead.
2. Does it work locally? Try to connect on the same device first via localhost instead.

— Reply to this email directly, view it on GitHub https://github.com/LostRuins/koboldcpp/issues/855#issuecomment-2119296743, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM5HQ7BJXZCYBMHEEG4WTOLZDDJG7AVCNFSM6AAAAABH6OXRV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZGI4TMNZUGM . You are receiving this because you authored the thread.Message ID: @.***>

[someonenameguy]

Other things I now tested:

1. Different Port 8085. Timeout
2. Different Port 5051. Timeout
3. Different Device. Timeout
4. Different Model. Timeout
5. Disable Flash Attention. Timeout
6. Setting Host to "192.168.1.224". Timeout

^^^ These were all tested without ssl. The Localhost was still accessible.

I couldn't quote in the email as I was going to sleep.

When I try to connect on my phone use firefox at https://192.168.1.224:7070/ the page is stuck on loading, till close koboldcpp, which then it shows timeout(Makes Sense). I tried with and without password and SSL.

Opening localhost page on the laptop works fine.

I have open hardware monitor on the laptop and I am able to connect to its web ui on 8086 on my phone.

Both devices are on the the same wifi. Other than that I did set port-forwarding for 192.168.1.224 from 7000-7999 wan and lan for other stuff but 7070 isn't being used by other programs

[someonenameguy]

The only way I was able to connect is to use remote tunnel trycloudflaretunnel. Which seem overkill for connecting from a local device to a local laptop.

[LostRuins]

If you've already tried binding to a specific interface with --host and that still does not work, the next step would be to check your windows firewall settings to ensure that incoming requests are not being blocked. For a test, you can try disable the firewall on your laptop first.

If that still does not work, it may be blocked by your router config. You should check if both devices are on the same networks and are able to see each other, and that nothing is getting filtered.

Another thing to try is to see if both devices can ping each other by their LAN IPs. Though this is not foolproof, as ping works over ICMP which is subject to different firewall rules.

[someonenameguy]

After checking the firewall. Turns out GlassWire didn't add koboldcpp to the private profile although it added it for public and domain profiles, odd plus open hardware monitor was fine local and wan.

I have disabled Glasswire and set the firewall allow all inbound connection and have seem to have fixed the problem.