Investigate using Hugging Face for prediction

[gavinhenderson]

This task is to investigate using a LLM via hugging face for prediction. Note this is not to do the work but just to create the separate tasks to do the work.

Ref: #13

[gavinhenderson]

Note: https://github.com/kdv123/PPMLM

[willwade]

I was reading 'for fun' this article https://shyam.blog/posts/beyond-self-attention/

A code snippet got my attention notably

prompt = 'my most gr'
tokens = encoding_helpers.tokenize_string(prompt)
logits, _ = m(tokens)
logits = LogitsWrapper(logits.detach(), tokenizer)
for token, prob in logits.topk_tokens(k=10)[0][-1]:
    print(f'{repr(token)} {prob:.3f}')

It outputs weights

'a' 0.819
'e' 0.081
'i' 0.059
'o' 0.036
'u' 0.004
'y' 0.001
'w' 0.000
'r' 0.000
'g' 0.000
's' 0.000

Something like https://huggingface.co/distilbert/distilbert-base-uncased might be ok

So you have to first convert it for MLKit

import coremltools as ct
import torch

# Load your trained PyTorch model
model = YourTrainedModel()

# Convert to Core ML
model_coreml = ct.convert(model, inputs=[ct.TensorType(shape=(1, seq_length))])

# Save the Core ML model
model_coreml.save("YourModel.mlmodel")

and then use it (NB: I cant see a quick way like that python snippet that prints weights.. I'm sure there must be - this is not pretty below

import CoreML
import SwiftUI

class TextPredictionModel: ObservableObject {
    private var model: YourModel? // Replace YourModel with the actual model class generated by Xcode

    init() {
        do {
            model = try YourModel(configuration: MLModelConfiguration())
        } catch {
            print("Error loading model: \(error)")
        }
    }

    func predictNextLetters(inputText: String) -> [(String, Double)] {
        guard let model = model else {
            return []
        }

        do {
            let input = YourModelInput(inputText: inputText) // Adjust based on your actual model's input
            let prediction = try model.prediction(input: input)

            // Assuming the model's output is a dictionary of [String: Double] where keys are letters and values are probabilities
            let probabilities = prediction.output // Adjust `output` based on your actual model's output property

            // Convert dictionary to sorted array of tuples and return
            return probabilities.sorted { $0.value > $1.value }
        } catch {
            print("Prediction error: \(error)")
            return []
        }
    }
}

struct ContentView: View {
    @State private var inputText: String = ""
    @ObservedObject private var textPredictionModel = TextPredictionModel()

    var body: some View {
        VStack {
            TextField("Type something...", text: $inputText)
                .padding()

            Button("Predict Next Letters") {
                let predictions = textPredictionModel.predictNextLetters(inputText: inputText)
                // Handle the predictions as needed, e.g., display them
                print(predictions)
            }

            // Display predictions or handle them as needed
        }
    }
}