search

tjake / Jlama

Jlama is a modern LLM inference engine for Java
Apache License 2.0
665 stars 62 forks source link

Error while running command (com.github.tjake.jlama.cli.commands.ChatCommand@166fa74d): java.lang.UnsupportedOperationException #74

Closed urbandroid closed 1 month ago

urbandroid commented 1 month ago

jlama chat tjake/Llama-3.2-1B-Instruct-Jlama-Q4 --auto-download

NOTE: Picked up JDK_JAVA_OPTIONS: --add-modules jdk.incubator.vector --enable-preview WARNING: Using incubator modules: jdk.incubator.vector NOTE: Picked up JDK_JAVA_OPTIONS: --add-modules jdk.incubator.vector --enable-preview WARNING: Using incubator modules: jdk.incubator.vector Using Native SIMD Operations (OffHeap) Model type = Q4, Working memory type = F32, Quantized memory type = I8

Chatting with tjake/Llama-3.2-1B-Instruct-Jlama-Q4...

You: hello

Jlama: Exception in thread "main" picocli.CommandLine$ExecutionException: Error while running command (com.github.tjake.jlama.cli.commands.ChatCommand@166fa74d): java.lang.UnsupportedOperationException
    at picocli.CommandLine.executeUserObject(CommandLine.java:2035)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:2264)
    at picocli.CommandLine.parseWithHandlers(CommandLine.java:2664)
    at picocli.CommandLine.parseWithHandler(CommandLine.java:2599)
    at com.github.tjake.jlama.cli.JlamaCli.main(JlamaCli.java:57)
    at jlama.main(jlama.java:22)
Caused by: java.lang.UnsupportedOperationException
    at com.github.tjake.jlama.tensor.operations.PanamaTensorOperations.accumulate(PanamaTensorOperations.java:2174)
    at com.github.tjake.jlama.tensor.operations.NativeTensorOperations.accumulate(NativeTensorOperations.java:409)
    at com.github.tjake.jlama.model.TransformerBlock.forward(TransformerBlock.java:105)
    at com.github.tjake.jlama.model.AbstractModel.forward(AbstractModel.java:275)
    at com.github.tjake.jlama.model.AbstractModel.batchForward(AbstractModel.java:262)
    at com.github.tjake.jlama.model.AbstractModel.batchForward(AbstractModel.java:252)
    at com.github.tjake.jlama.model.AbstractModel.generate(AbstractModel.java:456)
    at com.github.tjake.jlama.cli.commands.ChatCommand.run(ChatCommand.java:92)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
    ... 9 more
tjake commented 1 month ago

Can you try installing the latest version of jlama?

jbang app install --force --fresh jlama@tjake

urbandroid commented 1 month ago

i have tried and it works

gregbarton commented 1 month ago

I'm getting the same error, even with an uninstall/install with jbang.

jlama chat --auto-download tjake/Llama-3.1-70B-Instruct-Jlama-Q4
NOTE: Picked up JDK_JAVA_OPTIONS: --add-modules jdk.incubator.vector --enable-preview
WARNING: Using incubator modules: jdk.incubator.vector
NOTE: Picked up JDK_JAVA_OPTIONS: --add-modules jdk.incubator.vector --enable-preview
WARNING: Using incubator modules: jdk.incubator.vector
Using Native SIMD Operations (OffHeap)
Model type = Q4, Working memory type = F32, Quantized memory type = I8

Chatting with tjake/Llama-3.1-70B-Instruct-Jlama-Q4...

You: How much is a duck?

Jlama: Exception in thread "main" picocli.CommandLine$ExecutionException: Error while running command (com.github.tjake.jlama.cli.commands.ChatCommand@1f1c7bf6): java.lang.UnsupportedOperationException
    at picocli.CommandLine.executeUserObject(CommandLine.java:2035)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:2264)
    at picocli.CommandLine.parseWithHandlers(CommandLine.java:2664)
    at picocli.CommandLine.parseWithHandler(CommandLine.java:2599)
    at com.github.tjake.jlama.cli.JlamaCli.main(JlamaCli.java:57)
    at jlama.main(jlama.java:22)
Caused by: java.lang.UnsupportedOperationException
    at com.github.tjake.jlama.tensor.operations.PanamaTensorOperations.accumulate(PanamaTensorOperations.java:2174)
    at com.github.tjake.jlama.tensor.operations.NativeTensorOperations.accumulate(NativeTensorOperations.java:409)
    at com.github.tjake.jlama.model.TransformerBlock.forward(TransformerBlock.java:105)
    at com.github.tjake.jlama.model.AbstractModel.forward(AbstractModel.java:275)
    at com.github.tjake.jlama.model.AbstractModel.batchForward(AbstractModel.java:262)
    at com.github.tjake.jlama.model.AbstractModel.batchForward(AbstractModel.java:252)
    at com.github.tjake.jlama.model.AbstractModel.generate(AbstractModel.java:456)
    at com.github.tjake.jlama.cli.commands.ChatCommand.run(ChatCommand.java:92)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
    ... 9 more