search

iimez / inference-server

8 stars 1 forks source link

Building on Windows not possible because of missing cp command #4

Closed avnigashi closed 4 months ago

avnigashi commented 4 months ago

System: Windows 11, AMD CPU/GPU Node: v20.15.0 installed nativ via nvm, no wsl subsystem.

The cp command used in the build script (cp -r src/engines/node-llama-cpp/grammars dist/engines/node-llama-cpp/grammars) is a Unix command that doesn't exist on Windows by default. 

> lllms@1.0.0-beta.11 pre-build
> rimraf ./dist

> lllms@1.0.0-beta.11 Build
> tsc --build tsconfig.release.json --force && tsc-alias -p tsconfig.release. json && cp -r src/engines/node-llama-cpp/grammars dist/engines/node-llama-cpp/grammars

The command "cp" is either misspelled or
could not be found.

Process terminated with exit code 1

i would suggest, that we use a custom build.mjs where build commands can be executed and its cross plattform and we would get rid of rimraf.

Here is my propose: https://github.com/iimez/lllms/commit/fc2c62a6911ff86c7449b5bf1dc3a0784f7956f0

iimez commented 4 months ago

thanks a lot for the fixes! i have a hard time testing much on windows so its very much appreciated. will release a new beta soon.

avnigashi commented 4 months ago

I have to thank you! A very cool project. Very good work!

iimez commented 4 months ago

Very welcome! I released beta.12 including some changes to types. Grammar API changed a bit as well, they can just be supplied as string dictionary now.

avnigashi commented 4 months ago

@iimez yeah, saw various cool changes. I've followed every change so far, hehe. I'd be happy to support you with that project. I'll see what else needs to be done, hehe. Tell me if you have a pref

iimez commented 4 months ago

awesome!! no specific preference in priority, feel free to pick anything you find useful/important. im currently reviewing the node-llama-cpp engine implementation to make sure im not "hiding" anything thats likely useful (like the json-schema-to-gbnf converter, or lora support). also want to check that ctrl+c works more reliable.

my goal for the summer is to get the whisper+function calling demo to work realtime with a mic. will become an example if i get it working. so i'll probably prioritize transformers.js integration a bit more soon, on the way there.

i enabled discussions, and planning to post some more thoughts on design of the cli and the non-OAI http api there soon. will appreciate your feedback. if you have any ideas or q's feel free to use it as well :)