search

evilsocket / cake

Distributed LLM and StableDiffusion inference for mobile, desktop and server.
Other
2.43k stars 126 forks source link

Error in model.forward: error in forward batch operation for block #13

Open hash-momo opened 1 month ago

hash-momo commented 1 month ago

The first time I call the API, it works fine. However, when I call the REST API for the second time, the master node reports the error: cake/api/mod.rs:98:10: called Result::unwrap() on an Err value: error in model.forward: error in forward batch operation for block 29: error receiving response for Batch Additionally, one of my workers will also trigger an error and then stop: src/cake/worker.rs:225:26: called Result::unwrap() on an Err value: cannot broadcast [28, 28] to [1, 32, 28, 65]

caohuilong commented 1 month ago

The problem still exists and has not been solved. When sending the second request, the master reports an error image

one of the worker reports image

evilsocket commented 1 month ago

@caohuilong are you sure you pulled the fix before compiling?

caohuilong commented 1 month ago

@caohuilong are you sure you pulled the fix before compiling?

yes, i pull the repo today and have check the code is the newest.

angive commented 1 month ago

I hope the author can resolve this issue as soon as possible.

evilsocket commented 1 month ago

the author works at this project for fun during his free time