mit-han-lab / TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library
https://mit-han-lab.github.io/TinyChatEngine/
MIT License
756 stars 73 forks source link

Error encountered during inference #90

Open plasm0r opened 9 months ago

plasm0r commented 9 months ago

Hi

I encountered the following error using the 'LLaMA2_7B_chat' model:

USER: Which dinosaurs survived the giant asteroid impact? ASSISTANT: The giant asteroid impact that occurred around 65 million years ago, commonly referred to as the Chicxulub Asteroid Impact, had a devastating effect on life on Earth. While many species went extinct as a result of this event, some dinosaurs are believed to have survived in various ways. Here's what we know:

  1. Non-avian dinosaurs: The asteroid impact is thought to have caused the extinction of all non-avian dinosaurs, including the well-known Tyrannosaurus Rex and Velociraptor. These species were likely unable to adapt to the sudden environmental changes that followed the impact.
  2. Avian dinosaurs: On the other hand, some avian dinosaurs (birds) are believed to have survived the asteroid impact. The earliest known bird, Archaeopteryx, lived around 150 million years ago, during the Late Jurassic period. While it's unclear how many avian dinosaurs survived the impact, some scientists suggest that birds may have been better equipped to withstand the environmental changes that followed due to their smaller size and more flexible diets.
  3. Other species: Some other species of dinosaurs, such as pterAssertion failed: max_sqlen > len + start_idx, file src/ops/RotaryPosEmb.cc, line 15