search

HazyResearch / H3

Language Modeling with the H3 State Space Model
Apache License 2.0
513 stars 54 forks source link

Training code? #6

Open NtaylorOX opened 1 year ago

NtaylorOX commented 1 year ago

Amazing work - has some really interesting implications for the research field.

Is there the possibility to see the scripts for actually training these models?

Thanks

DanFu09 commented 1 year ago

Releasing the full training script is in our roadmap - will post an update here when we have more details about timing.

NtaylorOX commented 1 year ago

Awesome - thanks

ekg commented 1 year ago

This is great work! Congrats!

I strongly agree with @NtaylorOX. We need to know how the models are built to understand it and expand its scope.

I'd actually been hoping to do exactly that, but I won't be able to until there is public code to back up your paper. This is a negative cycle that will decrease the impact of your work dramatically.

Please put a priority on publishing your training code so that others can reproduce and confirm your work!

DanFu09 commented 1 year ago

These are available here now: https://github.com/HazyResearch/safari