search

usnistgov / ccu_validation_scoring

Other
5 stars 0 forks source link

Norm Discovery Scoring 0 per norm category for modality All #7

Open yrf1 opened 1 year ago

yrf1 commented 1 year ago

Potential bug? The scores for each norm category aggregated across all the data modality comes out as 0 - a noticeable bug when running from performer end.

jfiscus commented 1 year ago

Please be more specific (like adding a command line or send data to jfiscus@nist.gov) I do not know how to track this down. Also, the initial issue report included a command line that failed. It looks like pip install needed re-ran.

yrf1 commented 1 year ago

I used the standard command from README:

CCU_scoring score-nd \ -s test/pass_submissions/pass_submissions_LDC_reference_sample/ND/CCU_P1_TA1_ND_NIST_mini-eval1_20220815_164235 \ -ref test/reference/LDC_reference_sample \ -i test/reference/LDC_reference_sample/index_files/LC1-SimulatedMiniEvalP1.20220909.ND.scoring.index.tab

jfiscus commented 1 year ago

Please try this test:

Test 1: cd python -m CCU_validation_scoring.cli score-nd -s test/pass_submissions/pass_submissions_LDC_reference_sample/ND/CCU_P1_TA1_ND_NIST_mini-eval1_20220815_164235 -ref test/reference/LDC_reference_sample -i test/reference/LDC_reference_sample/index_files/LC1-SimulatedMiniEvalP1.20220909.ND.scoring.index.tab -o /tmp

Test 2: cd python3 -m pytest test