Add Ollama API integration with AgentOps

[sarath59]

Add Ollama API Integration with AgentOps

Overview

This pull request integrates the Ollama API with AgentOps, enabling tracking of both synchronous and asynchronous interactions. The following changes were made:

Changes Made

• llm_tracker.py:

  • Added support for Ollama API.
  • Implemented methods to handle synchronous and asynchronous responses.
  • Patched Ollama chat methods for tracking.

• client.py:

  • Initialized and utilized the updated LlmTracker.
  • Ensured override_api method in LlmTracker is called during client initialization.

Test Script

Created a test script to verify the integration. The script performs synchronous and asynchronous calls to the Ollama API and logs the interactions.

Test Script:

import ollama
from agentops import Client
import asyncio

# Initialize the AgentOps client with your API key
client = Client(api_key="YOUR_API_KEY_HERE")

# Start a session
session_id = client.start_session(tags=["ollama_test"])

try:
    # Synchronous chat call to Ollama API
    sync_response = ollama.chat(
        model='orca-mini',
        messages=[{'role': 'user', 'content': 'Why is the sky blue?'}]
    )
    print("Sync Response:", sync_response)

    # Asynchronous chat call to Ollama API using asyncio.to_thread to run the sync method
    async def async_chat():
        response = await asyncio.to_thread(ollama.chat,
                                           model="orca-mini",
                                           messages=[{"role": "user", "content": "Why is the sky blue?"}])
        return response

    async def main():
        async_response = await async_chat()
        print("Async Response:", async_response)

    asyncio.run(main())

    # End the session successfully
    client.end_session("Success")

except Exception as e:
    # End the session with failure if there is an exception
    client.end_session("Fail", str(e))
    raise
Results
The script successfully logged interactions with the Ollama API and provided session replay links for review. Both synchronous and asynchronous interactions were tracked and logged correctly.

Conclusion
This integration enables comprehensive monitoring and analysis of interactions with Ollama, facilitating better understanding and debugging of API usage within the AgentOps framework.

[gitguardian[bot]]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

| GitGuardian id | GitGuardian status | Secret | Commit | Filename | | | -------------- | ------------------ | ------------------------------ | ---------------- | --------------- | -------------------- | | [-](https://dashboard.gitguardian.com/workspace/514623/incidents/secrets) | | Generic High Entropy Secret | 3c2aac3defec9660607c73d8007e9cbaf3bb705e | tests/test_ollama.py | [View secret](https://github.com/AgentOps-AI/agentops/commit/3c2aac3defec9660607c73d8007e9cbaf3bb705e#diff-ac68b8016970e812f1cc52b10bec0c46062ee5d4725555dbb34444e7b16dd387L6) |

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code. 2. Replace and store your secret safely. [Learn here](https://blog.gitguardian.com/secrets-api-management?utm_source=product&utm_medium=GitHub_checks&utm_campaign=check_run_comment) the best practices. 3. Revoke and [rotate this secret](https://docs.gitguardian.com/secrets-detection/secrets-detection-engine/detectors/generics/generic_high_entropy_secret#revoke-the-secret?utm_source=product&utm_medium=GitHub_checks&utm_campaign=check_run_comment). 4. If possible, [rewrite git history](https://blog.gitguardian.com/rewriting-git-history-cheatsheet?utm_source=product&utm_medium=GitHub_checks&utm_campaign=check_run_comment). Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data. To avoid such incidents in the future consider - following these [best practices](https://blog.gitguardian.com/secrets-api-management/?utm_source=product&utm_medium=GitHub_checks&utm_campaign=check_run_comment) for managing and storing secrets including API keys and other credentials - install [secret detection on pre-commit](https://docs.gitguardian.com/ggshield-docs/integrations/git-hooks/pre-commit?utm_source=product&utm_medium=GitHub_checks&utm_campaign=check_run_comment) to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[bboynton97]

237 also exists for ollama support

[sarath59]

237 also exists for ollama support

I thought I could give it a try. Let me know how can I proceed. Thanks.

[areibman]

237 also exists for ollama support

I thought I could give it a try. Let me know how can I proceed. Thanks.

Thanks @sarath59 for helping. Perhaps we can combine the changes in this PR with #237? Namely, you've provided the unit tests for this, whereas that branch has not.

[sarath59]

237 also exists for ollama support

I thought I could give it a try. Let me know how can I proceed. Thanks.

Thanks @sarath59 for helping. Perhaps we can combine the changes in this PR with #237? Namely, you've provided the unit tests for this, whereas that branch has not.

Sure @areibman , I am good with that if that helps. Let me know if I could help with anything else. Thanks.

[bboynton97]

closed in favor of #237