search

locuslab / deq

[NeurIPS'19] Deep Equilibrium Models
MIT License
724 stars 80 forks source link

DEQ for Vision Transformer #18

Closed wyvernbai closed 2 years ago

wyvernbai commented 2 years ago

Since the DEQ tech has achieved performance competitive with the state-of-the-art deep networks on Transformer based LM and CNN-based Image Recognition tasks, do the authors have plans to adapt DEQ for vision transformer architecture?

jerrybai1995 commented 2 years ago

Hi @wyvernbai ,

Thanks for the question. I have thought about it but never got my hands on it.

wyvernbai commented 2 years ago

Thanks for your reply.