Garbage in translation (http & Trados & OmegaT)

[claude-ws01]

Hello, I'm getting "@ @ " garbage in the translations.

ENVIRONMENT:

• win7 ult english
• OpusCAT v1.2.0.0 (tested v1 & v1.2.3 as well)
• OmegaT 5.7.1
• Trados 2021 (& tested sr2)
• Firefox ESR

In OpusCatMTEngine's window, I do "translate with model" and I get no garbage in the result. I managed to hook Opus in OmegaT & Trados, get lots of "@ @ "... Tested with browser: same bad result.

I tested with different versions of the plugin, and engine... always the same.

Am I missing something ? Any ideas anyone ?

Thank you.

[claude-ws01]

Just couldn't stop my brain from juggling with that issue... (since it did work once with OmegaT)... so I came back to the "workbench" and here's what I did, step by step.

• deleted

  • c_users_admin_local_opuscat
  • c_users_admin_local_OpusCatMTEngine

• run OpusCATMTEngine_v1.1.0.7

  • install en-es zip
  • HTTP request -> no garbage
  • exit app

• run OpusCATMTEngine_v1.2.0 (previously imported en-es still there)

  • HTTP request -> no garbage
  • exit app

• run OpusCATMTEngine_v1.2.3

  • dbase update - ok
  • HTTP request -> no garbage
  • exit app

• run OpusCATMTEngine_v1.2.3 (again)

  • HTTP request -> no garbage
  • test OmegaT -> no garbage
  • test Trados -> no garbage
  • test memoQ -> no result (at this time, enough testing, don't even like memoQ)

** My current conclusion. Deleting the "c_users_admin_local...opus folders" and restart resolved the issue... for now. (time will tell)

** Note:

• I wanted to save data in installation folder ("Store OPUS-CAT MT Engine data in..." UNCHECKED) and I got exception erros in both v.1.1.0.7 & v1.2.0
• did the same test with v1.2.3... no errors but previous en-es gone.
  • exited OpusCATMTEngine_v1.2.3 then moved "c:\users\admin\local\opuscat" folder to the running opus folder, restarted opus and en-es was there
    • tested HTTP again -> no garbage & perfect translation

** Suggestion improvements:

• add, under "Store OPUS-CAT MT engine...", a field containing the current path of storage.
• When changing the location of data folder (tick-untick "Store OPUS-CAT..."), if OPUS is not doing the move, display a warning/instruction message that the user can/must move the folder to maintain the list of model (with source path to new destination path).
• in the translate model tab, beside the "translate" button, add a button/function to jump to browser & test the same string but with the "http://localhost". It would allow to confirm the results are the same through HTTP (which relate to plugin results, if I'm not mistaking).

Note, I know a lot of it is in the documentation... spreaded here and there (you have to admit), still these are details that are time savers if included as I suggest.

Note 2, I (I) would have benefited from readiing your debug procedure/tools when it come to plugins. (ie, I was getting good results in the tranalstion tab of OPUS, but not in trados...)(how do you debug that???)

That's it! Relieved that it now works... because when it works, hell it works nice.

Opus is a awesome tool... be proud. :)

[claude-ws01]

Post note. My issue is resolved. "thank god" as would say my mexican friends. Kind regards Claude.

[claude-ws01]

I thought my issue was solved, but no.

Finally, pinpointed the issue: the 2019 model files.

The issue can be reproduced by adding a 2019 model.

(at first I downloaded what I could find on opus, then went to tatoeba, that's why I had 2019 & 2020 model files)

Furthermore, got a InvalidOperationException every time I attempted to "delete selected model", Opus crash, then I re-run, then "deleted" again the same model without error. But no issues at all deleting non-2019 models. Included the log file.

opuscat_log_DELETING-MODEL.txt

I leave the ticket open for you to see. cheers.

[TommiNieminen]

Thanks for your thorough testing, I'll keep this open as an enhancement issue, since the UI fixes you mention should be fairly simple to implement.

The root cause of the garbage output seems to be a fix I made in v1.1.0.8 to get rid of batch file post-processing (this was causing problems to some users). Unfortunately this broke some older models, which used BPE subword segmentation. All the newer models use SentencePiece subword segmentation, and since I've only done testing on them recently, this bug went unnoticed. I will either fix it or remove the BPE models from the model download list. A workaround is to use only the newest models, since I think all language pairs have SentencePiece models available for them now.

[claude-ws01]

Thx for your reply, genuinely appreciated.

Regarding downloading models...

Some thoughts of Enhancements:

• size of the models
• a cancel "x" in/beside the progress bar (instead of having to exit Opus)

(where I am, in a small village in the "3rd world", I'm limited to 3Mbps. With the DL size, I may decide for a more appropriate time to download.)

BTW, really happy with Opus, it works really great, with great results. A jewel. :) Take care. Claude.