search

facebookresearch / SimulEval

SimulEval: A General Evaluation Toolkit for Simultaneous Translation
Creative Commons Attribution Share Alike 4.0 International
102 stars 36 forks source link

Fix: typos and logic errors #108

Closed Epic-Eric closed 2 months ago

Epic-Eric commented 3 months ago

Description

Fixes Issue #44. Problem 1: Speech type displayed even though text is given for target. Solution 1: Typo fix.

Problem 2: The latency metrics are the same between each pair, like LAAL is the same is LAAL_CA, where LAAL_CA should be bigger due to considering elapsed time which counts computer processing time along with delays. Solution 2: The problem was that when “--computation-aware” argument was passed in args, it was set globally to True, causing all the metric object to take on ‘computation_aware’=True. The fix was very simple, just one line of code. In from_args which instantiates the object, I put ‘computation_aware’ to False. If it’s _CA, then it goes through a different pathway and instantiates the object directly not through from_args.

Screenshot 2024-07-28 at 12 37 05 PM (Metrics shown are now correct)

Type of change

How Has This Been Tested?

Successful locally. No test cases needed.

xutaima commented 3 months ago

Hi @Epic-Eric, thanks for the PR. Could you separate the change for the issue from the #107?