Luxadevi
commented
6 months ago Hey,
Cool to hear there are more Kagglers out there! We never really dropped Kaggle support but the scope and usecase shifted when working on the companion part. Kaggle doesnt allow proxy-passing/having public endpoints via Cloud-Flare, and blocks alot of connections, and whenever you launch certain parts of gradio or streamllit they kill the notebook instance. I myself use Kaggle with Ollama alot but in my experience running the UI also within that ENV made it reach a couple bottlenecks. So we kept with the old way of running Ollama within a notebook defined in this Kaggle-Notebook:
Here we use R0lfs Nat python tunneling script that won't false flag for a remote connection and stays generally on. Then you have the Ollama endpoint available and what i personally do is just run somewhere on a linux install since we can just kick off companion with a one liner:
curl https://raw.githubusercontent.com/Luxadevi/Ollama-Companion/main/install.sh | shThere are some old parts about this notebook that can be optimized like
- Removing-Cuda installer - not needed anymore
- Changing some parts of the httpx and asyncio back (Jupyter is not well equipped for this out of the box)
- Removing Ipython model-maker (yes this is how it all started :)
I myself removed these things but are filled with other testing things and my variables like IP and Ports for the backend, so i dont need to do that again and again.
If you would like to try this with another way feel free, always open for a PR if works well generally. as said i did run streamlit over on kaggle but all around performance was too low with loading in big models, checking hash and the like while running inference and so on. You can also use Ngrok as so many but Ngrok kicked me out or closed connections too many times to even consider :)
I hope i informed you well enough with this and will change the mentioned optimizations when im available.
Kind regards, Luxa,
Hello,
I am currently facing challenges with integrating the Olama Collab Integration repository with Kaggle. Although the repository mentions compatibility with Kaggle.com, it lacks detailed instructions or notebooks for seamless integration.
Despite my efforts to adapt the code and directory structure to Kaggle's environment, I have encountered persistent issues in achieving the desired functionality. As a result, I am seeking the assistance of the repository creator to collaborate on resolving these issues and ensuring successful integration with Kaggle.
I am eager to work together with the creator to overcome these obstacles and optimize the repository's performance on Kaggle. If possible, I would appreciate direct contact through my GitHub account or any linked social media platforms to facilitate communication and collaboration.
Furthermore, I have established a separate GitHub repository titled "Ollama Colab Integration for Kaggle," housing the modified versions of the files. You can access it here.
Thank you for considering my request for assistance
Warm regards, Md.Fatin Ishraq facebook