search

LargeWorldModel / LWM

Large World Model -- Modeling Text and Video with Millions Context
https://largeworldmodel.github.io/
Apache License 2.0
7.15k stars 552 forks source link

Any experiments on reasoning or world modeling ? #32

Open Randolph-zeng opened 8 months ago

Randolph-zeng commented 8 months ago

This paper talks about long context length for a language model which is extended to be a vision-language model. I wonder why is it called World Model. It is not obvious in the paper. This paper seems focus more on the long context and evaluation of related retrieval ability with little discussion on the world modelling.

I wonder is there any specific discovery on the model ability that improves along with long context training. Does it make it more robust against prompt variations? More robust on reasoning ? More semantically riched in concept representations ? Better ontological/hierarchical learning towards the meaning ?

Will be curious on hearing more about the findings from the authors.

Thanks a lot for any insights in advance : )

Yindong-Zhang commented 8 months ago

the same wondering about world modeling? It's seems to be a tricky claim.