Add CI Tests for Canary/AEDMultitask "lang_field"

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html

Apache License 2.0

11.93k stars 2.47k forks source link

Add CI Tests for Canary/AEDMultitask "lang_field" #10103

Open tbartley94 opened 2 months ago

tbartley94 commented 2 months ago

Is your feature request related to a problem? Please describe.

Some of the Canary changes lead to unreliable use of the lang_field slot. There should be a test to check for it in the future for CI integration.

Describe the solution you'd like

Add a CI test to aed_models that switches out the lang_field and confirms same outputs regardless
Make this CI test take into account cases where target_lang has prompts that are not in the tokenizer.

pzelasko commented 2 months ago

I'll look into that.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

tbartley94 commented 1 month ago

@pzelasko did you fix this? (no pressure, just checking if I should take down)

github-actions[bot] commented 2 days ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.