search

dataculturegroup / DataBasic

A suite of focused and simple tools and activities for journalists, data journalism classrooms and community advocacy groups
http://www.databasic.io/
MIT License
62 stars 16 forks source link

WordCounter not processing pasted/uploaded text #392

Closed rahulbot closed 4 years ago

rahulbot commented 4 years ago

Reported by a user. Verified - type anything into the text box and it throws an error. The log file on production says:

2020-05-20T17:37:05.464858830Z app[web.1]: 2020-05-20 17:37:05,464 - databasic - ERROR - Exception on /en/wordcounter/ [POST]
2020-05-20T17:37:05.464889181Z app[web.1]: Traceback (most recent call last):
2020-05-20T17:37:05.464895518Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
2020-05-20T17:37:05.464901292Z app[web.1]:     response = self.full_dispatch_request()
2020-05-20T17:37:05.464906504Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
2020-05-20T17:37:05.464911919Z app[web.1]:     rv = self.handle_user_exception(e)
2020-05-20T17:37:05.464916996Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
2020-05-20T17:37:05.464935122Z app[web.1]:     reraise(exc_type, exc_value, tb)
2020-05-20T17:37:05.464940239Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
2020-05-20T17:37:05.464966288Z app[web.1]:     raise value
2020-05-20T17:37:05.464971345Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
2020-05-20T17:37:05.464976119Z app[web.1]:     rv = self.dispatch_request()
2020-05-20T17:37:05.464980440Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
2020-05-20T17:37:05.464996919Z app[web.1]:     return self.view_functions[rule.endpoint](**req.view_args)
2020-05-20T17:37:05.465001919Z app[web.1]:   File "/app/databasic/views/wordcounter.py", line 86, in index
2020-05-20T17:37:05.465006286Z app[web.1]:     doc_id = mongo.save_words('wordcounter', counts, ignore_case, ignore_stopwords, title, sample_id, btn_value, extras_to_save)
2020-05-20T17:37:05.465010770Z app[web.1]:   File "/app/databasic/logic/db.py", line 30, in save_words
2020-05-20T17:37:05.465015102Z app[web.1]:     return str(self._db[collection].save(data_to_save))
2020-05-20T17:37:05.465019354Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/collection.py", line 3136, in save
2020-05-20T17:37:05.465023864Z app[web.1]:     return self._insert(
2020-05-20T17:37:05.465027900Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/collection.py", line 605, in _insert
2020-05-20T17:37:05.465032256Z app[web.1]:     return self._insert_one(
2020-05-20T17:37:05.465036352Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/collection.py", line 594, in _insert_one
2020-05-20T17:37:05.465041047Z app[web.1]:     self.__database.client._retryable_write(
2020-05-20T17:37:05.465045153Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1243, in _retryable_write
2020-05-20T17:37:05.465049553Z app[web.1]:     return self._retry_with_session(retryable, func, s, None)
2020-05-20T17:37:05.465053783Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1196, in _retry_with_session
2020-05-20T17:37:05.465058449Z app[web.1]:     return func(session, sock_info, retryable)
2020-05-20T17:37:05.465063476Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/collection.py", line 582, in _insert_command
2020-05-20T17:37:05.465068136Z app[web.1]:     result = sock_info.command(
2020-05-20T17:37:05.465072247Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/pool.py", line 584, in command
2020-05-20T17:37:05.465076604Z app[web.1]:     self._raise_connection_failure(error)
2020-05-20T17:37:05.465080656Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/pool.py", line 745, in _raise_connection_failure
2020-05-20T17:37:05.465085083Z app[web.1]:     raise error
2020-05-20T17:37:05.465089061Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/pool.py", line 570, in command
2020-05-20T17:37:05.465093456Z app[web.1]:     return command(self.sock, dbname, spec, slave_ok,
2020-05-20T17:37:05.465097691Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/network.py", line 121, in command
2020-05-20T17:37:05.465102031Z app[web.1]:     request_id, msg, size = message.query(
2020-05-20T17:37:05.465110664Z app[web.1]:   File "/app/.heroku/python/lib/python3.8/site-packages/pymongo/message.py", line 746, in query
2020-05-20T17:37:05.465115577Z app[web.1]:     return _query_uncompressed(options, collection_name, num_to_skip,
2020-05-20T17:37:05.465120053Z app[web.1]: bson.errors.InvalidDocument: Cannot encode object: l'your text'
rahulbot commented 4 years ago

I think the bug here was that the translation shortcut (_('some text)) wasn't returning a str. So I fixed this by forcing the title of the doc to be a string.