search

tjake / Jlama

Jlama is a modern LLM inference engine for Java
Apache License 2.0
669 stars 62 forks source link

ShieldGemma model not supported #104

Closed lordofthejars closed 3 weeks ago

lordofthejars commented 3 weeks ago

Seems like the latest changes for supporting ShieldGemma (Gemma 2 classification model) aren't working in 0.8.0. I have the dependency and did a copy paste from your example but still I got:

Caused by: java.lang.UnsupportedOperationException: Classification not supported by this model
    at com.github.tjake.jlama.model.AbstractModel.loadClassifierWeights(AbstractModel.java:185)
    at com.github.tjake.jlama.model.AbstractModel.<init>(AbstractModel.java:169)
    at com.github.tjake.jlama.model.llama.LlamaModel.<init>(LlamaModel.java:58)
    at com.github.tjake.jlama.model.gemma2.Gemma2Model.<init>(Gemma2Model.java:61)
    at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
    ... 35 more

The example is at: https://github.com/lordofthejars-ai/quarkus-langchain-examples/tree/main/prompt-injection-guard

Produced here: https://github.com/lordofthejars-ai/quarkus-langchain-examples/blob/main/prompt-injection-guard/src/main/java/org/acme/GuardModelsProducer.java#L47

To run the demo run ./mvnw quarkus:dev in the project and curl localhost:8080/test

lordofthejars commented 3 weeks ago

Seems something is wrong with my configuration.