Create a model file from the generated training_state.dat

[manojmanivannan]

Apologies if my question is stupid, is it at all possible to create a model so we can run this generated model on , say, ollama ?

[keyvank]

Unfortunately, the file format femtoGPT is generating is something special to femtoGPT and not a standardized one, so no, you can't directly load training_state.dat into ollama. Although, maybe in the future, we can add the ability to generate standard model formats to femtoGPT :)

[manojmanivannan]

thanks @keyvank for the quick response. So how can i run the model in inference mode ?

[keyvank]

@manojmanivannan Just change the main.rs file and keep this:

let inference = gpt.infer(
    &mut rng,
    &tokenizer.tokenize("YOUR INPUT TO THE MODEL"),
    100,
    inference_temperature,
    |_ch| {},
)?;

// Generate 100 character with the currently trained model before
// starting the training loop.
println!("{}", tokenizer.untokenize(&inference));

[nitirajrathore]

@keyvank : Thanks for developing such a nice project. Can you also help by writing the full code for this inference in the project itself. I am a coder, but don't know rust. I tried the snippet you gave but it is giving lots of error that I don't understand. Now just to test the model generated, I will have learn rust. Can you please complete the inference part as well so that newbies can directly run the generated model.

[keyvank]

@nitirajrathore Please check my last commit