Add a notebook with BakLLaVA inference from Python

kopyl commented 8 months ago

Also add notebooks directory in the root of this repo

kopyl commented 8 months ago

You can preview the notebook here: https://colab.research.google.com/drive/1yn-25Np_6OiJ7IDBBcuPOsPF9vqWBjzt

@nisten @Sentdex @pharaouk @marianna13

Guys, could you please review and merge?

mshumer commented 7 months ago

Hey @kopyl, I'm having trouble with the notebook -- getting this error when running the last cell:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-9-b3405b5fe08b> in <cell line: 1>()
      1 with torch.inference_mode():
----> 2     output_ids = model.generate(
      3         input_ids=input_ids,
      4         images=image_tensor,
      5         do_sample=True,

2 frames
/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py in sample(self, input_ids, logits_processor, stopping_criteria, logits_warper, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus, streamer, **model_kwargs)
   2768             # sample
   2769             probs = nn.functional.softmax(next_token_scores, dim=-1)
-> 2770             next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
   2771 
   2772             # finished sentences should have their next token be a padding token

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Any ideas as to how this can be fixed?

kopyl commented 7 months ago

@mshumer try Runpod. Google Colab sometimes has issues.

SkunkworksAI / BakLLaVA

Add a notebook with BakLLaVA inference from Python #6