Closed chandrasg closed 4 years ago
Same here, randomly stopped after going through about half my list. What's weird is I broke it down into two loops and it also stopped after about half of that loop as well (from ~2500 entries to ~1200 entries). Seems like a rate limit as I added sleep(.4) to every loop of my script and it finally worked.
There should be some way of gracefully showing what the actual cause is
Did a workaround. Check this PR: #51
Do we have fix here? I can't pass my string variable includes lot mixed German and English into. And it just break at this point translator.translate(cleared_text) with the same above error.
@VindhyaSRajan -- for your update
I have this error too, and I found emoji in translating text cause this error.
same thing here with emojis, I had to fallback to plain text to be able to translate.
Yup, just made for sure emojis break it - for the record, I get
>>> t = trans.translate("this is life")
[[["this is life","this is life",null,null,0]],null,"en",null,null,null,1,null,[["en"],null,[1],["en"]],null,null,null,null,null,[["this","life","is","this is"]]]
without emojis and this with emojis
>>> t = trans.translate("this is a lemon \N{LEMON}")
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 403 (Forbidden)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>403.</b> <ins>That’s an error.</ins>
<p>Your client does not have permission to get URL <code>/translate_a/single?client=t&sl=auto&tl=en&hl=en&dt=at&dt=bd&dt=ex&dt=ld&dt=md&dt=qca&dt=rw&dt=rm&dt=ss&dt=t&ie=UTF-8&oe=UTF-8&otf=1&ssel=0&tsel=0&tk=888770.784426&q=this+is+%F0%9F%8D%8B</code> from this server. <ins>That’s all we know.</ins>
I got this by modifying utils.legacy_format_json
def legacy_format_json(original):
# save state
states = []
text = original
# save position for double-quoted texts
for i, pos in enumerate(re.finditer('"', text)):
# pos.start() is a double-quote
p = pos.start() + 1
if i % 2 == 0:
nxt = text.find('"', p)
states.append((p, text[p:nxt]))
# replace all weired characters in text
while text.find(',,') > -1:
text = text.replace(',,', ',null,')
while text.find('[,') > -1:
text = text.replace('[,', '[null,')
# recover state
for i, pos in enumerate(re.finditer('"', text)):
p = pos.start() + 1
if i % 2 == 0:
j = int(i / 2)
nxt = text.find('"', p)
# replacing a portion of a string
# use slicing to extract those parts of the original string to be kept
text = text[:p] + states[j][1] + text[nxt:]
print(converted) # I added this line
converted = json.loads(text) # error stems from here
return converted
(btw there is a spelling mistake in # replace all weird characters in text
)
I think I might have an idea on what's causing this. I noticed that some emojis do work, but not the ones with code points within a special range (e.g. like the ones above 1F000). I think this is because the encoding Python uses to count characters, as opposed to JS which counts in UTF-16, causing the token generator to create the wrong token (the tokens didn't match when I tested it with the web version of Google Translate). Check out this issue from the PHP version of the library for more info.
I'm not sure about this.
Creating the request url in chrome. By entering the emoji, then using requests.get seems to respond with the correct response.
This is certainly how this library handles this specifically.
From: Stephanie Wagner notifications@github.com Sent: Thursday, August 30, 2018 4:44:33 AM To: ssut/py-googletrans Cc: Modelmat; Comment Subject: Re: [ssut/py-googletrans] json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) (#47)
I think I might have an idea on what's causing this. I noticed that some emojis do work, but not the ones with code points within a special range (e.g. like the ones above 1F000https://unicode.org/Public/emoji/11.0/emoji-data.txt). I think this is because the encoding Python uses to count characters, as opposed to JS which counts in UTF-16, causing the token generator to create the wrong token (the tokens didn't match when I tested it with the web version of Google Translate). Check out this issue from the PHP versionhttps://github.com/Stichoza/google-translate-php/issues/42 of the library for more info.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ssut/py-googletrans/issues/47#issuecomment-417062510, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYhwzUM9nAQxEWtS_MdaWki8hLDDT6Vzks5uVuERgaJpZM4QScW5.
hello, i found when i translate an Chinese sentences to English. The error happened, i had read the issues that the length limit and the emoji influence. Obviously, Not these two reasons. Coding is here:
from googletrans import Translator
text = "唐高宗又在李世𪟝等朝廷武勋的模棱两可下"
translate = Translator()
en = translate.translate(text=text, dest='en').text
print(en)
And the error information:
JSONDecodeError Traceback (most recent call last)
<ipython-input-18-8ecffbfeebab> in <module>()
2 text = "唐高宗又在李世𪟝等朝廷武勋的模棱两可下"
3 translate = Translator()
----> 4 en = translate.translate(text=text, dest='en').text
5 print(en)
~/anaconda3/lib/python3.6/site-packages/googletrans/client.py in translate(self, text, dest, src)
170
171 origin = text
--> 172 data = self._translate(text, dest, src)
173
174 # this code will be updated when the format is changed.
~/anaconda3/lib/python3.6/site-packages/googletrans/client.py in _translate(self, text, dest, src)
79 r = self.session.get(url, params=params)
80
---> 81 data = utils.format_json(r.text)
82 return data
83
~/anaconda3/lib/python3.6/site-packages/googletrans/utils.py in format_json(original)
60 converted = json.loads(original)
61 except ValueError:
---> 62 converted = legacy_format_json(original)
63
64 return converted
~/anaconda3/lib/python3.6/site-packages/googletrans/utils.py in legacy_format_json(original)
52 text = text[:p] + states[j][1] + text[nxt:]
53
---> 54 converted = json.loads(text)
55 return converted
56
~/anaconda3/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
352 parse_int is None and parse_float is None and
353 parse_constant is None and object_pairs_hook is None and not kw):
--> 354 return _default_decoder.decode(s)
355 if cls is None:
356 cls = JSONDecoder
~/anaconda3/lib/python3.6/json/decoder.py in decode(self, s, _w)
337
338 """
--> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
340 end = _w(s, end).end()
341 if end != len(s):
~/anaconda3/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
355 obj, end = self.scan_once(s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
𪟝
It's seem that the translator can't process the 𪟝
character....
Really painful to work with this.(sometime works , not sure when it works :) )
Traceback (most recent call last):
File "D:\workspace\EBpythonP\src\co\sudheeksha\eb\transltep.py", line 9, in
Same error here. Problem now is that any attempt for translating is showing the same error
translator.translate("Hola", src = "es", dest = "en").text
Throws:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "../py-googletrans/googletrans/client.py", line 172, in translate data = self._translate(text, dest, src) File "../py-googletrans/googletrans/client.py", line 81, in _translate data = utils.format_json(r.text) File "../py-googletrans/googletrans/utils.py", line 62, in format_json converted = legacy_format_json(original) File "../py-googletrans/googletrans/utils.py", line 54, in legacy_format_json converted = json.loads(text) File "/usr/lib/python3.5/json/__init__.py", line 319, in loads return _default_decoder.decode(s) File "/usr/lib/python3.5/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
@ssut could you please take a look at this problem? I have exactly the same issue for a lot of the texts
Hey Guys,
I've faced the same problem, and I think, that it's caused by google's restrictions. My temporary fix is to find and use proxy, which works.
from py_translator import Translator
proxy = {
'http': 'http://username:password@1.1.1.1:1234',
'https': 'http://username:password@1.1.1.1:1234',
}
s = Translator(proxies=proxy).translate(text='Hello my friend', dest='es').text
print(s)
I also have this problem. It seems that it's due to Google usage limits; however, sleeping or using a proxy hasn't given any results so far.
Any idea?
I had the same issue when trying to translate Chinese to English. Anyone has a doable solution?
I've also run into the same issue. What I'm sure is I was able to do the same thing before, and all a sudden this error showed up, which is a indication I believe has sth. to do with Google's restriction as @elzeard91 pointed out. I also dug into the source code of the json library which throw out this error and here is the related code:
def decode(self, s, _w=WHITESPACE.match):
"""Return the Python representation of ``s`` (a ``str`` instance
containing a JSON document).
"""
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
end = _w(s, end).end()
if end != len(s):
raise JSONDecodeError("Extra data", s, end)
return obj
def raw_decode(self, s, idx=0):
"""Decode a JSON document from ``s`` (a ``str`` beginning with
a JSON document) and return a 2-tuple of the Python
representation and the index in ``s`` where the document ended.
This can be used to decode a JSON document from a string that may
have extraneous data at the end.
"""
try:
obj, end = self.scan_once(s, idx)
except StopIteration as err:
raise JSONDecodeError("Expecting value", s, err.value) from None
return obj, end
Is it because the translator didn't get a proper response so that the package was empty??
I have same problem. Google probably is banning ip. I tried different ip. First some time, it works. then this error occurs
I have same problem. Google probably is banning ip. I tried different ip. First some time, it works. then this error occurs
I tried to use some kind of rotating proxy. I found a project on github called multitor which uses tor, it create as many tor proxy as asked. Even after setting up 20 differents proxy nodes the error occurs again.
I think I have figured out the emoji issue. here is the emoji website: https://pypi.org/project/emoji/ install: pip install emoji
just need to demoji before translate, here is the sample str = '아니, 여기 왜 이르케 좋아앙?🙀♡' translator = Translator() str_demoji = emoji.demojize(str) tran = translator.translate(str_demoji,src='ko', dest='en').text str_emoji = emoji.emojize(tran.replace(': ',':').capitalize())
There is Quite a simple Solution to this Reason Google thinks you are a bot and is Banning your IP Solution Use a VPN like Psiphon I created a try-except in translation, if there is an exception I just turn on VPN and it works fine. If it even blocks you on the VPN no Problem Just Change your Country, You can also automate this in Linux with Torghost. By Creating another .py file In that .py file os.system("torghost -r") Basically requests new exit Node and You are ready to go. You could use other services in Windows but in Linux use this one. Enjoy Cause we ain't not paying google 20 bucks for google translate api Happy Coding
I can confirm that changing IP ( public ) is solving this error for a while.
I am getting the bellow error while executing the code python code
C:\Users\moraj\Desktop\BOT\Selfbot>python helloworld.py
Open this link on your LINE for smartphone in 2 minutes
line://au/q/qqomAuq9WpzzaTQzTvex091eHy5dE7tj
Traceback (most recent call last):
File "helloworld.py", line 14, in
C:\Users\moraj\Desktop\BOT\Selfbot>
can any one help me on this issue
This problem is due to the restrictions of Google. My solution to it is to change the network, like from WiFi to mobile phone's hotpot.
Everyone Listen! Carefully,
Google restricts use of this after certain requests form your IP. So, that is why our IP is black listed and it gives the JSON DECODE ERROR!
Change your IP using a Free VPN while you are running your code. i.e. select a German IP, when it is blocked goto an American IP.
#/usr/bin/python3
try:
# Your Translation Code here
except:
# Command to change IP using torghost or any other CLI based VPN
Just change ur IP's lads! That is it. Do it Manually if you like. I use Psiphon3 VPN it is free. By using the above approach your code will be stable and it will work fine.
I translated over 600K Words without an error using this approach. Linux/Ubuntu
#Start Torghost
os.system('torghost -s')
#Change IP when exception
os.system('torghost -x')
On MacOS
See https://pypi.org/project/PyMultitor/
On Windows
Stick to the VPN Method. Or use services like torghost on Windows. But the Tor Approach is better
You can also get errors when there is an emoji in your text or a square box.
Filter your text to the standards that Google Translate can process and that will do the thing.
Thanks! But i still got the error message after I demoji, changed the IP address and even run the code in try-except loop as @UBISOFT-1 suggested. Does anyone face the same issue?
Here is the code:
from googletrans import Translator
from emoji import demojize
import requests
from stem import Signal
from stem.control import Controller
def get_tor_session():
session = requests.session()
session.proxies = {'http': 'socks5://127.0.0.1:9050',
'https': 'socks5://127.0.0.1:9050'}
return session
def renew_connection():
with Controller.from_port(port = 9051) as controller:
controller.authenticate(password="your password")
controller.signal(Signal.NEWNYM)
str = '我住在地球🙀♡'
de_str = demojize(str)
print(de_str)
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
renew_connection()
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
translator = Translator()
translated = translator.translate(de_str, dest='en').text
Error message:
Traceback (most recent call last): File "C:/Users/user/desktop/Module test/[google_translate]2.py", line 29, in
translated = translator.translate(de_str, dest='en').text File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\googletrans\client.py", line 172, in translate data = self._translate(text, dest, src) File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\googletrans\client.py", line 81, in _translate data = utils.format_json(r.text) File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\googletrans\utils.py", line 62, in format_json converted = legacy_format_json(original) File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\googletrans\utils.py", line 54, in legacy_format_json converted = json.loads(text) File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\json__init__.py", line 357, in loads return _default_decoder.decode(s) File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
@edmangog No, Ideas. But I am working a Script that solves this problem as a whole, give me the time of a week k, guys. The script works for 5K Words at one time and no limitation. No Json Decode Errors Ok Hold tight Guys. It is coming. Now, I am just testing it. And adding Languages
@edmangog No, Ideas. But I am working a Script that solves this problem as a whole, give me the time of a week k, guys. The script works for 5K Words at one time and no limitation. No Json Decode Errors Ok Hold tight Guys. It is coming. Now, I am just testing it. And adding Languages
please update me
@UBISOFT-1 I would like to get update on your progress with the script for the google translator issue
I'm getting this error while just trying to import streamlit (current version - 1.16.0). I tried installing previous versions as well. I'm using PyCharm CE on a Mac version 12.4
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/code.py", line 90, in runcode exec(code, self.locals) File "<input>", line 1, in <module> File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/streamlit/__init__.py", line 55, in <module> from streamlit.delta_generator import DeltaGenerator as _DeltaGenerator File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/streamlit/delta_generator.py", line 45, in <module> from streamlit.elements.arrow_altair import ArrowAltairMixin File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/streamlit/elements/arrow_altair.py", line 34, in <module> import altair as alt File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/__init__.py", line 4, in <module> from .vegalite import * File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/vegalite/__init__.py", line 2, in <module> from .v4 import * File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/vegalite/v4/__init__.py", line 2, in <module> from .schema import * File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/vegalite/v4/schema/__init__.py", line 2, in <module> from .core import * File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/vegalite/v4/schema/core.py", line 4, in <module> from altair.utils.schemapi import SchemaBase, Undefined, _subclasses File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/utils/__init__.py", line 1, in <module> from .core import ( File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/altair/utils/core.py", line 13, in <module> import jsonschema File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/jsonschema/__init__.py", line 23, in <module> from jsonschema.validators import ( File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/jsonschema/validators.py", line 432, in <module> meta_schema=_utils.load_schema("draft3"), File "/Users/pratoshbharani/Documents/Documents - Pratosh’s MacBook Pro/MOOC/UDEMY/Python_40Days/venv/lib/python3.10/site-packages/jsonschema/_utils.py", line 62, in load_schema return json.loads(data) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I am trying to translate a list of tokens from chinese to english. I get the following error after a few attempts of translation. Is this because of rate limits?