Is this still being worked on? The example program won't run and it appears to require a very specific JVM version.

Hello!

Is this project still being maintained in a major way? I can't quite get it to work.

I tried to run this software by using the example code provided, except modified it to simply use a model I have on my computer already:

`` public static void main(String[] args) {

    File model = new File("models/TinyLlama-1.1B-Chat-v1.0/");

    AbstractModel m = ModelSupport.loadModel(model, DType.F32, DType.I8);

    String prompt = "Say hello.";

    if (m.promptSupport().isPresent()) {

        prompt = m.promptSupport().get().newBuilder()
                .addSystemMessage("You're a helpful chatbot who writes short responses.")
                .addUserMessage(prompt)
                .build();
    }

    System.out.println("Prompt: " + prompt + "\n");

    Response r = m.generate(UUID.randomUUID(), prompt, 0.7f, 256, false, (s, f) -> {
        System.out.println(s);
    });

    System.out.println(r.toString());

}

`` I'll preface by saying I was trying to use a model that wasn't suggested by you on the main readme page, as I was under the impression that you could run any type of chat-based safetensor model with this API. So I apologize if this all turns out to be a major misunderstanding. I'll admit that I'm new to LLM-based projects and I'm trying to learn more in my spare time.

Anyway, first of all, it failed to run at first. It seems to require a specific preview feature that's only currently in Java21(?). A friend said it's something to do with a new vector instruction API. I haven't read about it much myself, but either way- I couldn't get this API to run on anything besides Java21. Here's the error I got in Java22:

Exception in thread "main" java.lang.UnsupportedClassVersionError: com/github/tjake/jlama/tensor/AbstractTensor (class file version 65.65535) was compiled with preview features that are unsupported. This version of the Java Runtime only recognizes preview features for class file version 66.65535

Then of course, in Java21, it gave me an error again, but that was just because I had to enable the preview features. I'll admit that the specific configuration puts me off a little bit from using it in my own projects. This doesn't seem to have been mentioned anywhere in your example.

So, I finally get past this, and it ends up giving me an error that there is no safetensor model found in the models folder I've made. At the time, it looked like this (don't mind the GGUF files):

That said, I ended up just realizing the API probably expects it to be formatted like it is on huggingface, so I adjusted it appropriately and put all of these files in a unique folder, and renamed the model to just "model.safetensors." That one's on me.

And finally, thinking I finally had all the pieces together, triumphantly ran the program to just get... this.

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at com.github.tjake.jlama.model.ModelSupport.loadModel(ModelSupport.java:161) at com.github.tjake.jlama.model.ModelSupport.loadModel(ModelSupport.java:87) at com.github.tjake.jlama.model.ModelSupport.loadModel(ModelSupport.java:77) at com.nokoriware.ai.test.JlamaTest.main(JlamaTest.java:16) Caused by: java.lang.reflect.InvocationTargetException at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:74) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) at com.github.tjake.jlama.model.ModelSupport.loadModel(ModelSupport.java:153) ... 3 more Caused by: java.lang.NoClassDefFoundError: jdk/incubator/vector/Vector at com.github.tjake.jlama.safetensors.Weights.load(Weights.java:128) at com.github.tjake.jlama.safetensors.WeightLoader.load(WeightLoader.java:30) at com.github.tjake.jlama.safetensors.SafeTensorIndex.load(SafeTensorIndex.java:189) at com.github.tjake.jlama.model.llama.LlamaModel.loadInputWeights(LlamaModel.java:61) at com.github.tjake.jlama.model.AbstractModel.<init>(AbstractModel.java:135) at com.github.tjake.jlama.model.llama.LlamaModel.<init>(LlamaModel.java:55) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ... 6 more Caused by: java.lang.ClassNotFoundException: jdk.incubator.vector.Vector at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) ... 13 more

At this point, I decided it was best to...

Anyway, just so that this wasn't a complete failure on my part, I figured I'd report the problems here. If it's a user error, I apologize. But the good news is if that's the case, others can see this and learn from it to have an easier time setting it up.

Take care!

tjake / Jlama

Is this still being worked on? The example program won't run and it appears to require a very specific JVM version. #39