search

h-zhao1997 / cobra

Cobra: Extending Mamba to Multi-modal Large Language Model for Efficient Inference
MIT License
219 stars 7 forks source link

How to generate sequence with more than one word. #7

Open xuy1234 opened 2 months ago

xuy1234 commented 2 months ago

In cobra/cobra/models/mamba/modeling_mamba.py line 1772:

assert hidden_states.shape[1] == 1, "Only support decoding with 1 token at a time for now"

which prevent me from generating sequence. But in the paper there are examples of sequence generating like Fig. 4 in page 13.

So, how can I change the code to achieve this?

xuy1234 commented 2 months ago

maybe set cg =True when generating is a solution.