search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
128.37k stars 25.47k forks source link

Add Flax diverse group search #25355

Open sanchit-gandhi opened 10 months ago

sanchit-gandhi commented 10 months ago

Feature request

Add diverse beam search decoding to Flax, an "alternative to BS that decodes a list of diverse outputs by optimising for a diversity-augmented objective", as described in the paper: https://arxiv.org/pdf/1610.02424.pdf

This feature would mimic the PyTorch equivalent, added in #9006.

@yeandy made a great start on adding this feature in the PR #24508. The PR is still open, and anyone in the community is free to pick-up the PR and see it through to completion!

Motivation

There's a promising PR for this feature that is partway there - it would be a shame not to see this through to completion!

Your contribution

Happy to answer any questions/queries on the PR and provide PR reviews 🤗 Think this would be a fun one for any Flax contributors who are interested!

chiral-carbon commented 10 months ago

Hey @sanchit-gandhi, can I continue the PR? I don't have experience writing with Flax though so let me know if you expect a very quick turnaround time for this PR! But if that's not a roadblock then I'd be interested to contribute.

sanchit-gandhi commented 10 months ago

Hey @chiral-carbon! There's no time pressure for this, so feel free to pick it up if you're interested! Would be a fun first Transformers Flax contribution

chiral-carbon commented 10 months ago

@sanchit-gandhi thanks! In that case I would love to pick it up 👍

sanchit-gandhi commented 10 months ago

Awesome! Feel free to continue the PR or open a new one!