Thanks in advance.
Running on home PC just fine, followed instructions, no issues besides the normal learning curve.
I've got a Dell Poweredge R720 with Windows Server 2016 Standard with the Hyper-V role installed.
The guest OS is currently Windows 11 Pro w/ 48 cores & 32 GB RAM allocated. 150GB VHDX file for storage. (NVMe drive on server) Fully updated. Already tried with Windows Server 2019 and Windows Server 2022, no luck, I thought it was an OS issue, but everything works just fine on home PC running Windows 11 Pro.
Fresh Install on guest VM - Win 11 Pro. Unzipped Alpaca-Turbo to C:\AI\Alpaca-Turbo. Grabbed ggml-model-q4_1.bin from Huggingface. Pi3141/alpaca-7b-native-enhanced, specifically. I installed Python 3.8 from 64-bit installer, enabled long path. Installed Miniconda3 Windows 64-bit. Conda init, added condabin to environmental variables, created environment, etc, followed instructions, no errors. The web interface loads fine. I load model, ggml-model-q4_1.bin is in the dropdown list. I click load and it just sits there. In task manager, the command window is using about 2% CPU. There's disk activity. It slowly ramps up to about 62MB RAM usage and stops. (Home PC only uses 30MB of RAM while model is fully loaded) CPU and disk activity stay at 2%. One error in the command prompt window that says: "127.0.0.1 - - [18/Apr/2023 11:42:31] "GET /load_model/undefined HTTP/1.1" 404 -" Then just never ending rows of ""GET /status HTTP/1.1" 200 -"
I'm pretty sure it has something to do with the fact that it's running in Hyper-V. All other variables are accounted for.
Possibilities:
Miniconda is a virtualized environment and a VM can't run inside another VM?
I installed Python by downloading the install .exe instead of letting Conda get it?
The guest OS does not have graphics acceleration.
That's all I got. One note, running on home PC, there is no Python item below the command prompt in task manager, whereas in the VM, there is two. One dormant, and one using 2-4% CPU. Did I mess up by installing Python manually instead of letting Conda install it?
Thanks in advance. Running on home PC just fine, followed instructions, no issues besides the normal learning curve. I've got a Dell Poweredge R720 with Windows Server 2016 Standard with the Hyper-V role installed. The guest OS is currently Windows 11 Pro w/ 48 cores & 32 GB RAM allocated. 150GB VHDX file for storage. (NVMe drive on server) Fully updated. Already tried with Windows Server 2019 and Windows Server 2022, no luck, I thought it was an OS issue, but everything works just fine on home PC running Windows 11 Pro. Fresh Install on guest VM - Win 11 Pro. Unzipped Alpaca-Turbo to C:\AI\Alpaca-Turbo. Grabbed ggml-model-q4_1.bin from Huggingface. Pi3141/alpaca-7b-native-enhanced, specifically. I installed Python 3.8 from 64-bit installer, enabled long path. Installed Miniconda3 Windows 64-bit. Conda init, added condabin to environmental variables, created environment, etc, followed instructions, no errors. The web interface loads fine. I load model, ggml-model-q4_1.bin is in the dropdown list. I click load and it just sits there. In task manager, the command window is using about 2% CPU. There's disk activity. It slowly ramps up to about 62MB RAM usage and stops. (Home PC only uses 30MB of RAM while model is fully loaded) CPU and disk activity stay at 2%. One error in the command prompt window that says: "127.0.0.1 - - [18/Apr/2023 11:42:31] "GET /load_model/undefined HTTP/1.1" 404 -" Then just never ending rows of ""GET /status HTTP/1.1" 200 -" I'm pretty sure it has something to do with the fact that it's running in Hyper-V. All other variables are accounted for. Possibilities: