search

SawyerHood / draw-a-ui

Draw a mockup and generate html for it
https://draw-a-ui.com
MIT License
13.18k stars 1.58k forks source link

Please add LLaVA #45

Open Xyndra opened 9 months ago

Xyndra commented 9 months ago

Please I want LLaVA. https://llava-vl.github.io/

eletroswing commented 9 months ago

I loved the idea of ​​having this opensource and completely free

AltayYuzeir commented 9 months ago

Maybe even combination between LLaVA 1.5 and DeepSeek Coder.

Anon2578 commented 9 months ago

Yes this is a very cool idea I’ll be here for it once it’s up and ready

eletroswing commented 9 months ago

@AltayYuzeir how llava 1.5 and deepseek coder can be implemented to work together or what it would look like in theory?

AltayYuzeir commented 9 months ago

Hmmm, perhaps I was inaccurate in my statement. I have been thinking about it, and I cannot comprehend direct communication on the neural network level between the two LLMs. I came up with two concepts: 1) If LLaVA is asked to produce a detailed description of the drawing, also including spacing between elements and alignments, the description can be passed on DeepSeek Coder 2) Maybe similar to LLaVA-Med, LLaVA can be fine-tuned on UI layouts, HTML and CSS

eletroswing commented 9 months ago

@AltayYuzeir I'm going to create a dataset of 'images to tailwind' using deepseek coder, and then we can train this custom model, what do you think?

AltayYuzeir commented 9 months ago

I am not the best AI or UI programmer, but I believe such resource will be beneficial in any case for the future. As far as I understand it, Tailwind is a high-level framework on top of CSS, featuring custom classes absent in native CSS. Tailwind is not explicitly supported on DeepSeek Coder, as per info from their GitHub page. I was thinking it might be perhaps more flexible to use DeepSeek to generate more general webpages built on top of raw HTML and CSS. Please, correct me if I am wrong.

eletroswing commented 9 months ago

@AltayYuzeir You are right! Similar to what vercel is doing with v0.dev. Anyway, I'm going to continue creating the dataset using tailwind used in public projects!

AltayYuzeir commented 9 months ago

I guess one could make a dataset for fine tuning based on UIs made with Dash (Python) and Shiny (R)? I wonder about the capabilities of DeepSeek Coder for Dash and Shiny specifically, since they are libraries for their respective languages, but are also frameworks, which function quite differently than most packages.

orkutmuratyilmaz commented 8 months ago

@eletroswing and @AltayYuzeir, supporting Ollama can solve all these requirements at once. Please check #52