search

punith300i / nlp-vlm-project

CS544 NLP project Team
0 stars 0 forks source link

Find open-source implementations of Flamingo #6

Open punith300i opened 1 year ago

MeMihir commented 1 year ago
zvovov commented 1 year ago

https://github.com/snap-stanford/med-flamingo by Stanford

vermaprince17 commented 1 year ago

Hi @punith300i

I am taking up lucidrains flamingo implementation and you can take up open-flamingo implementation.

vermaprince17 commented 1 year ago
  1. Inference code for open-flamingo implementation with 3B params. OOM error for 9B model on my pc.
  2. lucid-rains implementation has just API calls, which we should integrate (not an easy task) to build a flamingo model.
vermaprince17 commented 1 year ago

explore-

  1. play w diff params ->
  2. changing videos .. text. images -> inference time vs input size ->
  3. scaled down version of LM s and image encoders.
  4. fine-tuning (explore code base) ->
  5. netron (graph pictorial) ->