search

OthersideAI / self-operating-computer

A framework to enable multimodal models to operate a computer.
https://www.hyperwriteai.com/self-operating-computer
MIT License
8.68k stars 1.15k forks source link

OpenSource free Vision model use Instead of openAI #181

Closed ahsan3219 closed 6 months ago

ahsan3219 commented 6 months ago

Is your feature request related to a problem? Please describe. The current reliance on OpenAI for vision models poses limitations and concerns regarding accessibility and transparency due to its proprietary nature.

Describe the solution you'd like I propose integrating an open-source vision model into the project to replace the dependency on OpenAI. This solution would ensure greater accessibility, transparency, and long-term sustainability.

Describe alternatives you've considered Alternative solutions include continuing to use OpenAI, but this maintains the issues of proprietary technology and potential barriers to access for users. Another option is developing a custom vision model, but this could be time-consuming and resource-intensive.

Additional context Incorporating an open-source vision model aligns with the principles of open access and transparency, fostering community engagement and innovation. It offers benefits such as greater customization, flexibility, and trust within the user community.

Real-Experienes commented 6 months ago

That is the plan. Thank you ashan3219

joshbickett commented 6 months ago

Hi @ahsan3219 @Real-Experienes there is an open-source model available to use in the project. Check out the Llava section in the README.md: https://github.com/OthersideAI/self-operating-computer?tab=readme-ov-file#locally-hosted-llava-through-ollama.

Let me know if you have any questions about it!

ahsan3219 commented 6 months ago

@joshbickett

It is only for Linux and macOS. No option available for windows

joshbickett commented 6 months ago

@ahsan3219, oh ok. We can reopen this issue in that case. I noticed it didn't mention Windows specifically in the initial issue creation. It may be helpful to create a new issue that mentions Windows specifically so that the open-source community can quickly understand and address this issue specifically

Real-Experienes commented 6 months ago

Hi @ahsan3219 @Real-Experienes there is an open-source model available to use in the project. Check out the Llava section in the README.md: https://github.com/OthersideAI/self-operating-computer?tab=readme-ov-file#locally-hosted-llava-through-ollama.

Let me know if you have any questions about it!

Thanks @joshbickett !! I'm on it!