search

soimort / translate-shell

:speech_balloon: Command-line translator using Google Translate, Bing Translator, Yandex.Translate, etc.
https://www.soimort.org/translate-shell
The Unlicense
6.94k stars 391 forks source link

unexpected characters changed to encoding when tried with the tool #420

Closed mundey closed 3 years ago

mundey commented 3 years ago

scenario: tried to translate parts of text in source (html/go/js). encoding for '=' didn't change back in output and converted to u003d

command used:

trans -brief zh-CN:en file://example.txt -o example-out.txt

input example file content: 

          <el-form-item label="字典名称" prop="dictType">

actual output file:

<el-form-item labelu003d"Dictionary name" propu003d"dictType">

expected / output using web browser:

<el-form-item label="Dictionary name" prop="dictType">

i tried with -no-ansi but i guess it's not right option here is output with -D

  16 bytes > HTTP/1.1 200 OK
  46 bytes > Content-Type: application/json; charset=utf-8
  32 bytes > X-Content-Type-Options: nosniff
  31 bytes > Access-Control-Allow-Origin: *
  62 bytes > Cache-Control: no-cache, no-store, max-age=0, must-revalidate
  17 bytes > Pragma: no-cache
  39 bytes > Expires: Mon, 01 Jan 1990 00:00:00 GMT
  36 bytes > Date: Tue, 10 Aug 2021 10:33:01 GMT
  80 bytes > Content-Disposition: attachment; filename="json.txt"; filename*=UTF-8''json.txt
 101 bytes > Content-Security-Policy: require-trusted-types-for 'script';report-uri /_/TranslateApiHttp/cspreport
 192 bytes > Content-Security-Policy: script-src 'report-sample' 'nonce-5TXzZFq8qxe3qGstrORw6g' 'unsafe-inline';object-src 'none';base-uri 'self';report-uri /_/TranslateApiHttp/cspreport;worker-src 'self'
  43 bytes > Cross-Origin-Resource-Policy: cross-origin
  40 bytes > Cross-Origin-Opener-Policy: same-origin
  12 bytes > Server: ESF
  20 bytes > X-XSS-Protection: 0
  28 bytes > X-Frame-Options: SAMEORIGIN
  20 bytes > Accept-Ranges: none
  22 bytes > Vary: Accept-Encoding
  18 bytes > Connection: close
  27 bytes > Transfer-Encoding: chunked
   1 bytes >
   4 bytes > 352
 823 bytes > [[["\u003cel-form-item label\u003d\"Dictionary name\" prop\u003d\"dictType\"\u003e","\u003cel-form-item label\u003d\"字典名称\" prop\u003d\"dictType\"\u003e",null,null,3,null,null,[[]],[[["51171d70a5b6c2485a24361038890141","zh_en_2020q2.md"]]]],[null,null,null,"\u003cEl-form-item label\u003d\"zìdiǎn míngchēng\" prop\u003d\"dictType\"\u003e"]],null,"zh-CN",null,null,[["\u003cel-form-item label\u003d\"字典名称\" prop\u003d\"dictType\"\u003e",null,[["\u003cel-form-item label\u003d\"Dictionary name\" prop\u003d\"dictType\"\u003e",0,true,false,[3]],["\u003cEl-form-item label \u003d \"dictionary name\" prop \u003d \"dictType\"\u003e",0,true,false,[0]]],[[0,43]],"\u003cel-form-item label\u003d\"字典名称\" prop\u003d\"dictType\"\u003e",0,0]],1,[],[["zh-CN"],null,[1],["zh-CN"]],null,null,null,null,null,null,null,null,null,[null,2]]
   2 bytes > 0
   1 bytes >
content = '''
352
[[["\u003cel-form-item label\u003d\"Dictionary name\" prop\u003d\"dictType\"\u003e","\u003cel-form-item label\u003d\"字典名称\" prop\u003d\"dictType\"\u003e",null,null,3,null,null,[[]],[[["51171d70a5b6c2485a24361038890141","zh_en_2020q2.md"]]]],[null,null,null,"\u003cEl-form-item label\u003d\"zìdiǎn míngchēng\" prop\u003d\"dictType\"\u003e"]],null,"zh-CN",null,null,[["\u003cel-form-item label\u003d\"字典名称\" prop\u003d\"dictType\"\u003e",null,[["\u003cel-form-item label\u003d\"Dictionary name\" prop\u003d\"dictType\"\u003e",0,true,false,[3]],["\u003cEl-form-item label \u003d \"dictionary name\" prop \u003d \"dictType\"\u003e",0,true,false,[0]]],[[0,43]],"\u003cel-form-item label\u003d\"字典名称\" prop\u003d\"dictType\"\u003e",0,0]],1,[],[["zh-CN"],null,[1],["zh-CN"]],null,null,null,null,null,null,null,null,null,[null,2]]
0

'''
tokens = ["352", "[", "[", "[", "\"\\u003cel-form-item label\\u003d\\\"Dictionary name\\\" prop\\u003d\\\"dictType\\\"\\u003e\"", ",", "\"\\u003cel-form-item label\\u003d\\\"字典名称\\\" prop\\u003d\\\"dictType\\\"\\u003e\"", ",", "null", ",", "null", ",", "3", ",", "null", ",", "null", ",", "[", "[", "]", "]", ",", "[", "[", "[", "\"51171d70a5b6c2485a24361038890141\"", ",", "\"zh_en_2020q2.md\"", "]", "]", "]", "]", ",", "[", "null", ",", "null", ",", "null", ",", "\"\\u003cEl-form-item label\\u003d\\\"zìdiǎn míngchēng\\\" prop\\u003d\\\"dictType\\\"\\u003e\"", "]", "]", ",", "null", ",", "\"zh-CN\"", ",", "null", ",", "null", ",", "[", "[", "\"\\u003cel-form-item label\\u003d\\\"字典名称\\\" prop\\u003d\\\"dictType\\\"\\u003e\"", ",", "null", ",", "[", "[", "\"\\u003cel-form-item label\\u003d\\\"Dictionary name\\\" prop\\u003d\\\"dictType\\\"\\u003e\"", ",", "0", ",", "true", ",", "false", ",", "[", "3", "]", "]", ",", "[", "\"\\u003cEl-form-item label \\u003d \\\"dictionary name\\\" prop \\u003d \\\"dictType\\\"\\u003e\"", ",", "0", ",", "true", ",", "false", ",", "[", "0", "]", "]", "]", ",", "[", "[", "0", ",", "43", "]", "]", ",", "\"\\u003cel-form-item label\\u003d\\\"字典名称\\\" prop\\u003d\\\"dictType\\\"\\u003e\"", ",", "0", ",", "0", "]", "]", ",", "1", ",", "[", "]", ",", "[", "[", "\"zh-CN\"", "]", ",", "null", ",", "[", "1", "]", ",", "[", "\"zh-CN\"", "]", "]", ",", "null", ",", "null", ",", "null", ",", "null", ",", "null", ",", "null", ",", "null", ",", "null", ",", "null", ",", "[", "null", ",", "2", "]", "]", "0"]
ast = {
"0,0,0,0"   "\"\\u003cel-form-item label\\u003d\\\"Dictionary name\\\" prop\\u003d\\\"dictType\\\"\\u003e\""
"0,0,0,1"   "\"\\u003cel-form-item label\\u003d\\\"字典名称\\\" prop\\u003d\\\"dictType\\\"\\u003e\""
"0,0,0,2"   "null"
"0,0,0,3"   "null"
"0,0,0,4"   "3"
"0,0,0,5"   "null"
"0,0,0,6"   "null"
"0,0,0,8,0,0,0" "\"51171d70a5b6c2485a24361038890141\""
"0,0,0,8,0,0,1" "\"zh_en_2020q2.md\""
"0,0,1,0"   "null"
"0,0,1,1"   "null"
"0,0,1,2"   "null"
"0,0,1,3"   "\"\\u003cEl-form-item label\\u003d\\\"zìdiǎn míngchēng\\\" prop\\u003d\\\"dictType\\\"\\u003e\""
"0,1"   "null"
"0,2"   "\"zh-CN\""
"0,3"   "null"
"0,4"   "null"
"0,5,0,0"   "\"\\u003cel-form-item label\\u003d\\\"字典名称\\\" prop\\u003d\\\"dictType\\\"\\u003e\""
"0,5,0,1"   "null"
"0,5,0,2,0,0"   "\"\\u003cel-form-item label\\u003d\\\"Dictionary name\\\" prop\\u003d\\\"dictType\\\"\\u003e\""
"0,5,0,2,0,1"   "0"
"0,5,0,2,0,2"   "true"
"0,5,0,2,0,3"   "false"
"0,5,0,2,0,4,0" "3"
"0,5,0,2,1,0"   "\"\\u003cEl-form-item label \\u003d \\\"dictionary name\\\" prop \\u003d \\\"dictType\\\"\\u003e\""
"0,5,0,2,1,1"   "0"
"0,5,0,2,1,2"   "true"
"0,5,0,2,1,3"   "false"
"0,5,0,2,1,4,0" "0"
"0,5,0,3,0,0"   "0"
"0,5,0,3,0,1"   "43"
"0,5,0,4"   "\"\\u003cel-form-item label\\u003d\\\"字典名称\\\" prop\\u003d\\\"dictType\\\"\\u003e\""
"0,5,0,5"   "0"
"0,5,0,6"   "0"
"0,6"   "1"
"0,8,0,0"   "\"zh-CN\""
"0,8,1" "null"
"0,8,2,0"   "1"
"0,8,3,0"   "\"zh-CN\""
"0,9"   "null"
"0,10"  "null"
"0,11"  "null"
"0,12"  "null"
"0,13"  "null"
"0,14"  "null"
"0,15"  "null"
"0,16"  "null"
"0,17"  "null"
"0,18,0"    "null"
"0,18,1"    "2"
"0" "0"
}
❯ trans -V
Translate Shell       0.9.6.12-release

platform              Linux
terminal type         xterm-256color
bi-di emulator        [N/A]
gawk (GNU Awk)        5.1.0
fribidi (GNU FriBidi) 1.0.10
audio player          mpg123
terminal pager        less
web browser           xdg-open
user locale           en_CA.UTF-8 (English)
home language         en
source language       auto
target language       en
translation engine    google
proxy                 [NONE]
user-agent            Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36
ip version            [DEFAULT]
theme                 default
init file             [NONE]

Report bugs to:       https://github.com/soimort/translate-shell/issues

Thank you for peeking into the issue.

soimort commented 3 years ago

Thanks for reporting. Fixed now.