search

SawyerHood / draw-a-ui

Draw a mockup and generate html for it
https://draw-a-ui.com
MIT License
13.31k stars 1.6k forks source link

Please add LLaVA #45

Open Xyndra opened 1 year ago

Xyndra commented 1 year ago

Please I want LLaVA. https://llava-vl.github.io/

eletroswing commented 1 year ago

I loved the idea of ​​having this opensource and completely free

AltayYuzeir commented 1 year ago

Maybe even combination between LLaVA 1.5 and DeepSeek Coder.

Anon2578 commented 1 year ago

Yes this is a very cool idea I’ll be here for it once it’s up and ready

eletroswing commented 1 year ago

@AltayYuzeir how llava 1.5 and deepseek coder can be implemented to work together or what it would look like in theory?

AltayYuzeir commented 1 year ago

Hmmm, perhaps I was inaccurate in my statement. I have been thinking about it, and I cannot comprehend direct communication on the neural network level between the two LLMs. I came up with two concepts: 1) If LLaVA is asked to produce a detailed description of the drawing, also including spacing between elements and alignments, the description can be passed on DeepSeek Coder 2) Maybe similar to LLaVA-Med, LLaVA can be fine-tuned on UI layouts, HTML and CSS

eletroswing commented 1 year ago

@AltayYuzeir I'm going to create a dataset of 'images to tailwind' using deepseek coder, and then we can train this custom model, what do you think?

AltayYuzeir commented 1 year ago

I am not the best AI or UI programmer, but I believe such resource will be beneficial in any case for the future. As far as I understand it, Tailwind is a high-level framework on top of CSS, featuring custom classes absent in native CSS. Tailwind is not explicitly supported on DeepSeek Coder, as per info from their GitHub page. I was thinking it might be perhaps more flexible to use DeepSeek to generate more general webpages built on top of raw HTML and CSS. Please, correct me if I am wrong.

eletroswing commented 1 year ago

@AltayYuzeir You are right! Similar to what vercel is doing with v0.dev. Anyway, I'm going to continue creating the dataset using tailwind used in public projects!

AltayYuzeir commented 1 year ago

I guess one could make a dataset for fine tuning based on UIs made with Dash (Python) and Shiny (R)? I wonder about the capabilities of DeepSeek Coder for Dash and Shiny specifically, since they are libraries for their respective languages, but are also frameworks, which function quite differently than most packages.

orkutmuratyilmaz commented 11 months ago

@eletroswing and @AltayYuzeir, supporting Ollama can solve all these requirements at once. Please check #52

bocklucas commented 1 month ago

99