pepperoni21 / ollama-rs

A Rust library allowing to interact with the Ollama API.
MIT License
451 stars 68 forks source link

How to save History and Store It in a File #66

Open heydocode opened 2 weeks ago

heydocode commented 2 weeks ago

Request for Assistance: Implementing AI Response Function with History Management

Objective

I am working on a function that should take a String parameter (prompt) and return a String (AI response) with history tracking. Currently, my implementation does not handle history, and I need guidance on how to incorporate it. Additionally, I am interested in storing this history into a file.

Current Code Without History

// Error Handling
use thiserror::Error;

// Ollama API crate
use ollama_rs::Ollama;
use ollama_rs::generation::completion::request::GenerationRequest;

// Error Handling Definitions
#[derive(Debug, Error)]
pub enum AppError {
    #[error("Ollama API error: {0}")]
    OllamaError(String),
}

pub type Result<T> = std::result::Result<T, AppError>;

async fn generate_response(model: &str, text_prompt: &str) -> Result<String> {
    let ollama = Ollama::default();
    let request = GenerationRequest::new(model.to_string(), text_prompt.to_string());
    let response = ollama.generate(request).await.map_err(|e| AppError::OllamaError(e.to_string()))?;
    Ok(response.response)
}

pub async fn send_ollama_request(text_prompt: &str) -> Result<String> {
    // Generate a response from the AI model
    let model = "nex";  // Replace with your actual model name
    let prompt = &format!("{}", text_prompt);
    let response = generate_response(model, prompt).await?;

    Ok(response)
}

// other code non-related to this

Desired Enhancement

My situation

I am using Tauri, Svelte Kit, and Vite plugin in my project, which affects how I can integrate and test this feature. All my Rust code is available here.

I am open to sharing my code and collaborating with others who might be interested in contributing to this project.

Thank you in advance for your help!

The issue template that I've followed to create this issue: ### Issue Title: [Provide a concise title for the issue] #### Objective [Clearly state the goal or problem you want to address.] #### Current Implementation [Provide the current code or setup that you're working with. Include any relevant code snippets, configuration, or system information.] ```programming_language // Your current code or configuration here ``` Desired Enhancement [Feature 1]: [Describe the first feature or change you want to implement.] [Feature 2]: [Describe the second feature or change you want to implement.] [List all the features or changes you are seeking.] Example Request [Describe what kind of example or solution you are looking for.] Additional Context [Include any additional information that could be helpful.] References [If applicable, include links to related issues, discussions, or documentation that may provide additional context.] Key Points: Clear Headers: Use bold text for key points in the desired enhancements. Code Blocks: Maintain consistent formatting for code snippets and markdown. Detailed Context: Ensure context about the development environment and the repository is clear. Template in Details: Providing the template in a collapsible
section is a good practice for keeping the issue clean and organized. This format will help maintain clarity and provide a structured way for contributors to understand and address your request.
heydocode commented 2 weeks ago

LMAO right now I'm writing a sequential library to encode previous messages into cached size + stock them in hash maps and load them in AI which has in SYSTEM message instructions about explaining this history and how to understand it well. It's working really fine in rust even if the hash maps are created on the typescript side. I think that's far away from performance & saving resources so I hope someone will add the logging history into a file feature.