search

OpenAdaptAI / OpenAdapt

Open Source Generative Process Automation (i.e. Generative RPA). AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models
https://www.OpenAdapt.AI
MIT License
918 stars 121 forks source link

Implement LaVIN/BLIP2 #353

Open abrichr opened 1 year ago

abrichr commented 1 year ago

Feature request

We would like to implement https://github.com/luogen1996/LaVIN#demo and/or https://huggingface.co/docs/transformers/main/model_doc/blip-2as a ReplayStrategyMixin.

In particular we would like to understand how this compares to huggingface transformers.agent : https://github.com/OpenAdaptAI/OpenAdapt/pull/192

Motivation

Answering questions about screenshots

LaPetiteSouris commented 1 year ago

@abrichr is this still up for grab or already taken ?

angelala3252 commented 1 year ago

@LaPetiteSouris I'm currently working on this (PR #412) but if you want to collaborate I would be down!

LaPetiteSouris commented 1 year ago

@LaPetiteSouris I'm currently working on this (PR #412) but if you want to collaborate I would be down!

Thanks @angelala3252 No I'm good on other topic. Just snicking around to see available topics. Please go ahead and thanks for the tips.

rafikmatta commented 1 year ago

hi @angelala3252 I'll collab on this, seems interesting