27 of 6474 translated BUG issue

[johnfelipe]

source json files here:

https://github.com/KnowageLabs/Knowage-Server/tree/master/knowage-vue/src/i18n/en_US

pls help me to solve this

# jsontt knowage/messages.json
? From which language? English
? To which language | languages? (Can select more than one with space bar) Czech
, French, German
° Translating. Please wait. 27 of 6474 translated.
<--- Last few GCs --->

[2541:0x5ab56d0]   114407 ms: Mark-sweep 1808.2 (2079.1) -> 1782.2 (2063.7) MB, 222.3 / 0.2 ms  (average mu = 0.373, current mu = 0.369) allocation failure scavenge might not succeed
[2541:0x5ab56d0]   114743 ms: Mark-sweep 1804.2 (2078.4) -> 1777.8 (2062.5) MB, 212.2 / 0.2 ms  (average mu = 0.370, current mu = 0.368) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb09c10 node::Abort() [jsontt cli]
 2: 0xa1c193 node::FatalError(char const*, char const*) [jsontt cli]
 3: 0xcf8dbe v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [jsontt cli]
 4: 0xcf9137 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [jsontt cli]
 5: 0xeb09d5  [jsontt cli]
 6: 0xeb14b6  [jsontt cli]
 7: 0xebf9de  [jsontt cli]
 8: 0xec0420 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [jsontt cli]
 9: 0xec339e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [jsontt cli]
10: 0xe848da v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [jsontt cli]
11: 0x11fd626 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [jsontt cli]
12: 0x15f2099  [jsontt cli]
Aborted (core dumped)

[johnfelipe]

i run again with other languages

? To which language | languages? (Can select more than one with space bar) Polis
h, Portuguese, Spanish
o Translating. Please wait. 2206 of 6474 translated./root/json-translator/node_modules/got/source/as-promise.js:74
                                const error = new HTTPError(response, options);
                                              ^

GotError [HTTPError]: Response code 302 (Found)
Url: https://translate.google.com/_/TranslateWebserverUi/data/batchexecute?rpcids=MkEWBc&f.sid=7429583905198015450&bl=boq_translate-webserver_20220615.04_p0&hl=en-US&soc-app=1&soc-platform=1&soc-device=1&_reqid=8517&rt=c
    at EventEmitter.<anonymous> (/root/json-translator/node_modules/got/source/as-promise.js:74:19)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  host: 'translate.google.com',
  hostname: 'translate.google.com',
  method: 'POST',
  path: '/_/TranslateWebserverUi/data/batchexecute?rpcids=MkEWBc&f.sid=7429583905198015450&bl=boq_translate-webserver_20220615.04_p0&hl=en-US&soc-app=1&soc-platform=1&soc-device=1&_reqid=8517&rt=c',
  socketPath: undefined,
  protocol: 'https:',
  url: 'https://translate.google.com/_/TranslateWebserverUi/data/batchexecute?rpcids=MkEWBc&f.sid=7429583905198015450&bl=boq_translate-webserver_20220615.04_p0&hl=en-US&soc-app=1&soc-platform=1&soc-device=1&_reqid=8517&rt=c',
  gotOptions: {
    path: '/_/TranslateWebserverUi/data/batchexecute?rpcids=MkEWBc&f.sid=7429583905198015450&bl=boq_translate-webserver_20220615.04_p0&hl=en-US&soc-app=1&soc-platform=1&soc-device=1&_reqid=8517&rt=c',
    protocol: 'https:',
    slashes: true,
    auth: null,
    host: 'translate.google.com',
    port: null,
    hostname: 'translate.google.com',
    hash: null,
    search: '?rpcids=MkEWBc&f.sid=7429583905198015450&bl=boq_translate-webserver_20220615.04_p0&hl=en-US&soc-app=1&soc-platform=1&soc-device=1&_reqid=8517&rt=c',
    pathname: '/_/TranslateWebserverUi/data/batchexecute',
    href: 'https://translate.google.com/_/TranslateWebserverUi/data/batchexecute?rpcids=MkEWBc&f.sid=7429583905198015450&bl=boq_translate-webserver_20220615.04_p0&hl=en-US&soc-app=1&soc-platform=1&soc-device=1&_reqid=8517&rt=c',
    retry: {
      retries: [Function (anonymous)],
      methods: Set(6) { 'GET', 'PUT', 'HEAD', 'DELETE', 'OPTIONS', 'TRACE' },
      statusCodes: Set(7) { 408, 413, 429, 500, 502, 503, 504 },
      errorCodes: Set(8) {
        'ETIMEDOUT',
        'ECONNRESET',
        'EADDRINUSE',
        'ECONNREFUSED',
        'EPIPE',
        'ENOTFOUND',
        'ENETUNREACH',
        'EAI_AGAIN'
      }
    },
    headers: {
      'user-agent': 'got/9.6.0 (https://github.com/sindresorhus/got)',
      'content-type': 'application/x-www-form-urlencoded;charset=UTF-8',
      'accept-encoding': 'gzip, deflate',
      'content-length': 173
    },
    hooks: {
      beforeRequest: [],
      beforeRedirect: [],
      beforeRetry: [],
      afterResponse: [],
      beforeError: [],
      init: []
    },
    decompress: true,
    throwHttpErrors: true,
    followRedirect: true,
    stream: false,
    form: false,
    json: false,
    cache: false,
    useElectronNet: false,
    body: 'f.req=%5B%5B%5B%22MkEWBc%22%2C%22%5B%5B%5C%22Error%20persisting%20table%5C%22%2C%5C%22en%5C%22%2C%5C%22pt%5C%22%2Ctrue%5D%2C%5Bnull%5D%5D%22%2Cnull%2C%22generic%22%5D%5D%5D&',
    method: 'POST'
  },
  statusCode: 302,
  statusMessage: 'Found',
  headers: {
    location: 'https://www.google.com/sorry/index?continue=https://translate.google.com/_/TranslateWebserverUi/data/batchexecute%3Frpcids%3DMkEWBc%26f.sid%3D7429583905198015450%26bl%3Dboq_translate-webserver_20220615.04_p0%26hl%3Den-US%26soc-app%3D1%26soc-platform%3D1%26soc-device%3D1%26_reqid%3D8517%26rt%3Dc&hl=en-US&q=EgS-nvpKGIawupUGIhDdHjcTDqGQlsztv0wt2xAwMgFy',
    date: 'Sun, 19 Jun 2022 03:29:11 GMT',
    pragma: 'no-cache',
    expires: 'Fri, 01 Jan 1990 00:00:00 GMT',
    'cache-control': 'no-store, no-cache, must-revalidate',
    'content-type': 'text/html; charset=UTF-8',
    server: 'HTTP server (unknown)',
    'content-length': '556',
    'x-xss-protection': '0',
    'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"',
    connection: 'close'
  },
  body: '<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\n' +
    '<TITLE>302 Moved</TITLE></HEAD><BODY>\n' +
    '<H1>302 Moved</H1>\n' +
    'The document has moved\n' +
    '<A HREF="https://www.google.com/sorry/index?continue=https://translate.google.com/_/TranslateWebserverUi/data/batchexecute%3Frpcids%3DMkEWBc%26f.sid%3D7429583905198015450%26bl%3Dboq_translate-webserver_20220615.04_p0%26hl%3Den-US%26soc-app%3D1%26soc-platform%3D1%26soc-device%3D1%26_reqid%3D8517%26rt%3Dc&amp;hl=en-US&amp;q=EgS-nvpKGIawupUGIhDdHjcTDqGQlsztv0wt2xAwMgFy">here</A>.\r\n' +
    '</BODY></HTML>\r\n',
  code: 'BAD_REQUEST'
}

[ParvinEyvazov]

Thank you for reporting the problem. I investigated the problem and unfortunately, this package doesn't perform well in bigger JSON files because of an asynchronous operation.

For the current situation, you can try to split your file into a bunch of different files and translate them one by one and combine them all until I fix this problem.

How will you do this? Please, divide your JSON file into more pieces, translate them separately and merge it. For example: You have 1000 lines JSON file (a.json): Create valid JSON files:

a1.json (from a.json, take 0 to 250 lines)
a2.json (from a.json, take 251 to 500 lines)
a3.json (from a.json, take 501 to 750 lines)
a4.json (from a.json, take 751 to 1000 lines)

And translate them separately. (Don't forget to change the name of the file which will be created each time. Because the new English file name will always be en.json).

Also sometimes operations are blocked because of Google Translate API. My best advice is to split your JSON file as much as possible (200-300 objects per file are perfect for now), and try again when it fails.

I will inform you via this issue page when I solve it.

[RoyalC]

same trouble with me and my json file is 340 translated word long, i try to split it in 2 piece or 3 and same resultat. Hope you can find what's wrong. Did you manage to add deepl.com translation service ? thank for this great piece of code.

[ParvinEyvazov]

Hello @RoyalC , thank you for your comment!

I am investigating a new solution for big translation files and unexpected errors. I will look for the deepl.com service. Thank you for your contribution.

[johnfelipe]

Pls use azure or google

El dom, 17 de jul. de 2022, 10:33 a. m., Parvin Eyvazov < @.***> escribió:

Hello @RoyalC https://github.com/RoyalC , thank you for your comment!

I am investigating a new solution for big translation files and unexpected errors. I will look for the deepl.com service. Thank you for your contribution.

— Reply to this email directly, view it on GitHub https://github.com/mololab/json-translator/issues/3#issuecomment-1186548467, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADIWFE35MWFC35RG7QF6QTVUQRS5ANCNFSM5ZFVZRHQ . You are receiving this because you authored the thread.Message ID: @.***>

[ParvinEyvazov]

jsontt now uses a queue system for translating (instead of sending whole requests).

This problem should be solved by this implementation.