Command R and R+ support

microsoft / TransformerCompression

For releasing code related to compression methods for transformers, accompanying our publications

MIT License

352 stars 31 forks source link

Open Steel-skull opened 4 months ago

Steel-skull commented 4 months ago

Hello! first off thanks for the hard work!

is there plans to add Command R and R+ support?

I am attempting too but im a little out of my depth.

Thanks again!

nailimixaM commented 2 months ago

Hi @Steel-skull, thanks for asking - we're looking into MoEs, watch this space!