Document impact on HCA Language Evaluations

[Manssur94]

• First German Update: https://shopify.slack.com/archives/C06T9U6THPU/p1727292779002369 Impact

  • Anusha reported some of the issues mentioned - https://shopify.slack.com/archives/C076Y3GSDJ4/p1727348458595059
  • Bot speaks English
  • Admin buttons in English
  • Github issues created
    • https://github.com/Shopify/help/issues/51078#issuecomment-2379574102
    • https://github.com/Shopify/help-center/issues/5137#issuecomment-2393957650
  • Eyes from Delali to implement more robust translation evaluation and acknowledgement that manual reviews are helpful

• Reported a lot of English messages in BR-PT conversation: https://shopify.slack.com/archives/C06T9U6THPU/p1727355954708579

  • This has been fixed - https://shopify.slack.com/archives/C06T9U6THPU/p1727447816351419?thread_ts=1727378381.666949&cid=C06T9U6THPU

• Second German Update: https://shopify.slack.com/archives/C06T9U6THPU/p1727378395128679

• Highlighted that bot makes suggestions which are not available to German merchants which resulted in having this Github issue created - https://github.com/Shopify/help/issues/51078#issuecomment-2379571988

• Spanish second update: Repetitive bot information being looked into - https://shopify.slack.com/archives/C076Y3GSDJ4/p1727900263246339

• AI Athena translation suggested to have AI look at context and not only message at one time. Suggestion will be shipped in the future: https://shopify.slack.com/archives/C06T9U6THPU/p1727795568204139?thread_ts=1727466845.454139&cid=C06T9U6THPU

• Shopify Terminology: Raised awareness that HCA terms do not match HCA and Admin which needs to be fixed

  • https://shopify.slack.com/archives/C06T9U6THPU/p1727305857225869?thread_ts=1727292779.002369&cid=C06T9U6THPU
  • https://shopify.slack.com/archives/C076Y3GSDJ4/p1727817328858429?thread_ts=1727815470.303109&cid=C076Y3GSDJ4

• Bot saying "no results found" raised with devs, which ended up in a P0 ticket to investigate

  • https://shopify.slack.com/archives/C076Y3GSDJ4/p1729094157047709
  • Devs created this Github issue: https://github.com/Shopify/help-center/issues/5462

• Athena AI screenshot issues

  • https://shopify.slack.com/archives/C076Y3GSDJ4/p1729780632786599

• Japanese report resulting in not going live with Japanese at this stage