search

bootphon / ABXpy

ABX discrimination task in python
https://docs.cognitive-ml.fr/ABXpy
MIT License
43 stars 16 forks source link

Triplets computed when each item under a label is unique #20

Open maureendss opened 4 years ago

maureendss commented 4 years ago

It looks like triplets are computed - and scores calculated - even when each item has a different label, and that the abx score is computed "on" this label category. This should probably not happen as there can't be any triplet with A and X sharing the same label?

RobinAlgayres commented 4 years ago

Did you check the "analyse" file that is created during computation? In this file you ll find all ABX types and their scores. If you find one ABX type that should not be computed could you paste it here?

maureendss commented 4 years ago

Here is a snippet of the analyse file. None of those triplets should be computed as the triplets are supposed to be on a category which has a different label for every single item. It doesn't make sense to run such a scoring - but I was still wondering how these triplets and scores could even be computed. I can also attach the .item and "analyse" files if this is of any help?

sent_1  sent_2  by  score   n
FIN_0001    ENG_0001    0   0.5 2
FIN_0001    ENG_0002    0   0.5 2
FIN_0001    ENG_0003    0   1.0 2
FIN_0001    ENG_0004    0   1.0 2
FIN_0002    ENG_0098    0   0.5 2
FIN_0002    ENG_0099    0   0.5 2
FIN_0002    ENG_0100    0   0.5 2
FIN_0002    FIN_0001    0   0.25    4
FIN_0002    FIN_0003    0   0.0 4
RobinAlgayres commented 4 years ago

So I need to know how you created your abx task ( only the "abx-task" line in your script) and a snippet of your item file.

maureendss commented 4 years ago

Sure, this is the 'abx-task' line: abx-task $item $task --verbose --on sent And here's a snippet of the item file:

#file onset offset #lang spk sent
EF3_ENG_0001_0 0 150 ENG EF3 ENG_0001
EF3_ENG_0002_0 0 150 ENG EF3 ENG_0002
EF3_ENG_0003_0 0 150 ENG EF3 ENG_0003
EF3_ENG_0004_0 0 150 ENG EF3 ENG_0004
EF3_ENG_0005_0 0 150 ENG EF3 ENG_0005
EF3_ENG_0006_0 0 150 ENG EF3 ENG_0006
EF3_ENG_0007_0 0 150 ENG EF3 ENG_0007

Thanks!

RobinAlgayres commented 4 years ago

So you should use: abx-task $item $task --verbose --on sent --by lang

If you do not specify the "by" attribute it will compute all comparison of items in your item file with no distinction.