search

aylabs / chatgpt

Experiments with ChatGPT
Apache License 2.0
0 stars 0 forks source link

Research Open Source language models #6

Open acs opened 1 year ago

acs commented 1 year ago

Talking with jgb he has a presentation about this topic, specially focused in StableDiffusion:

https://jgbarah.github.io/presentations/stable-diffusion/slides-2022-12-01.pdf

The video for the talk: https://m.youtube.com/watch?v=NOGB2BrDG88

acs commented 1 year ago

StableDiffusion (this is not text, but images) BLIP-2 https://github.com/nomic-ai/gpt4all #9 https://laion.ai/blog/open-flamingo/ (vision-language models) https://github.com/hpcaitech/ColossalAI #10 (the best one to learn the architecture behind LLM) https://github.com/microsoft/DeBERTa

acs commented 1 year ago

An alternative for ChatGPT with the goal of having your own service:

https://github.com/nsarrazin/serge https://github.com/aylabs/chatgpt/issues/8

Let's explore it a bit! It is not open source, but you can download the models and build your own service. It is based in the model: https://huggingface.co/decapoda-research/llama-7b-hf-int4#llama-model-card (also in Alpaca?)

LLama (Large Language Model) comes from Facebook and it was leaked to internet? Alpaca is from Standford university. Both are at GPT3 level.

Some quick intro about them: https://youtu.be/YBWM5Uz-wd8

Alpaca is not available any more because problems with hallucinations.

The goal of Alpaca and it seems that also LLama is to have LLM in low computer hardware. Interesting reading: https://news.ycombinator.com/item?id=35136624

acs commented 1 year ago

Transferring knowledge between models is easier than expected: https://www.reddit.com/r/singularity/comments/11tmkqi/alpacapromptchaining_considerations_on_the_latest/

AI models training each other

During this trip, nice dissemination article in spanish: https://www.youtube.com/watch?v=FdZ8LKiJBhQ