search

DanielLin94144 / DUAL-textless-SQA

Textless (ASR-transcript free) Spoken Question Answering. The official release of NMSQA dataset and the implementation of "DUAL: Textless Spoken Question Answering with Speech Discrete Unit Adaptive Learning" paper.
Creative Commons Attribution Share Alike 4.0 International
34 stars 11 forks source link

Preprocessed units for the segments #4

Open mutiann opened 1 year ago

mutiann commented 1 year ago

Hello!

I'm recently doing some experiments on NMSQA, and the code for DUAL provided here are really helpful! While I encountered some difficulty building the units using scripts provided to reproduce the results. Particularly, I'm trying to extract the units for each segment of context, while the preprocessed ones currently provided in the repo are already concatenated for each article following the standard QA scheme (using the merge_passage.py, I guess). May I know if the preprocessed units for each segment could be provided?

Thank you!

P.S. Just seen you and had some chat on Interspeech at the poster. The work was really impressive and useful for us :)

mutiann commented 1 year ago

(Or do you have any number for the performance of DUAL with the view only on each segment?)

DanielLin94144 commented 1 year ago

Hi @mutiann ! Thanks for the question. Here is the segment hubert units from hubert large 22-th layer with 128 number of clusters. google drive link

mutiann commented 1 year ago

Thank you very much! Let me have a look at them.

mutiann commented 1 year ago

Thank you very much! Actually I am looking for the HuBERT units for each segment (for, e.g., context-0_0_1, context-0_0_2, ...), while it seems that the provided units above and in the README are for each paragraph as in standard SQuAD (e.g. context-0_0, context-0_1, context-0_2, ...) that are merged from those from each segment. May I know if those for the each segment could be provided?

BTW, out of curiousity, have you tried to have any experiment that works on each segment instead of having a view on the paragraph?

Thanks in advance!