mozilla / translations

The code, training pipeline, and models that power Firefox Translations
https://mozilla.github.io/translations/
Mozilla Public License 2.0
154 stars 33 forks source link

Pass W&B suffix to publish_group_logs (offline experiments) #818

Closed vrigal closed 2 months ago

vrigal commented 2 months ago

I tested the offline publication command using parse_tc_group WDNdtYFhTG6gh7c1wnAEyw --no-recursive-lookup, and noticed the suffix was missing from group logs (and metrics not related to a training task).

The results have been published to https://wandb.ai/teklia/offline-en-lt?nw=nwuserrigal and can be compared with the moz-translations namespace.

Extra notes:

eu9ene commented 2 months ago

@vrigal I see 8 custom charts in the moz-translations Group: spring-2024_WDNdtYFhTG6gh7c1wnAEyw and 9 ones on your new dashboard Group: spring-2024_WDNdtYFhTG6gh7c1wnAEyw. I see that flores-devtest is duplicated.

vrigal commented 2 months ago

I think this is still related to the W&B display bug (#716). I cannot see the duplicated flores-devtest, and I am able to list the 9 tables with corresponding names from moz-translations using the API:

>>> import wandb
>>> api = wandb.Api()
>>> run = api.run(f"moz-translations/en-lt/5u7r4tvy")
>>> [f.name for f in run.files() if f.name.startswith("media/table/")]
['media/table/flores_aug-inline-noise_devtest_table_3_5cad38ff706b388967e7.table.json',
 'media/table/flores_aug-mix_devtest_table_6_904e363d1ee0dc1a0b78.table.json',
 'media/table/flores_aug-noise_devtest_table_8_8dd7548d18d24f67a868.table.json',
 'media/table/flores_aug-title_devtest_table_7_5c54e4f59d3421123d4d.table.json',
 'media/table/flores_aug-typos_devtest_table_4_f1bb8df26a397b3a7d3d.table.json',
 'media/table/flores_aug-upper_devtest_table_2_87bed4d20feb9267ef22.table.json',
 'media/table/flores_devtest_table_0_6b6c668f710045f649e6.table.json',
 'media/table/mtdata_Neulab-tedtalks_test-1-eng-lit_table_1_3c0bef4225a0472f0156.table.json',
 'media/table/sacrebleu_wmt19_table_5_0c5ad8f647415fe81c61.table.json']