search

hlt-mt / FBK-fairseq

Repository containing the open source code of works published at the FBK MT unit.
Other
41 stars 1 forks source link

question about "Efficient yet Competitive Speech Translation: FBK@IWSLT2022" #1

Closed zhhao1 closed 1 year ago

zhhao1 commented 1 year ago

A simple method, char-ratio filter, produce great gains. I've tried it before, but it didn't work out very well. I want to know how you do it and can't find the relevant code. Hopefully you can point out where your code is implemented. There is no file processing flow in the Readme file.

mgaido91 commented 1 year ago

Thanks for your question. The related code can be found at:

https://github.com/hlt-mt/FBK-fairseq/blob/master/examples/speech_to_text/scripts/filter_on_char_ratio.py.

We will update the README in the next days with your suggestion. Or if you want to create a PR to update it, we appreciate that.

Unfortunately, as stated in the paper, we tested these methods only on MuST-C en-de. As such we do not know how and if our findings generalise to other datasets. In case you have more insights and you want to share with us, we would appreciate it. Thanks.

mgaido91 commented 1 year ago

I'm closing this as I updated the description, now it should be clearer. Please feel free to re-open if something else is needed.

zhhao1 commented 1 year ago

I'm closing this as I updated the description, now it should be clearer. Please feel free to re-open if something else is needed.

Sorry for the delay in reply. In the past I didn't remove punctuation from src text, so the filter ratio was set incorrectly. After setting the correct ratio, it will work properly. The updated description is very clear. This is very good work, and I believe that this data processing configuration should be widely used in the future because it is simple, but really effective.

mgaido91 commented 1 year ago

Thank you for your feedback, and glad it helped. Please let me know if you have other questions.