LoganMeitz / votefinder

GNU General Public License v3.0
2 stars 2 forks source link

Unicode characters in votes #27

Open rjmagley opened 1 year ago

rjmagley commented 1 year ago

Currently, Votefinder seems to totally ignore votes with certain characters in them, like ‽. I have a suspicion that it fails with basically any character outside of the ASCII set.

Interestingly, this wasn't a problem before - I've had votes in games where ε was used for my name, so it's either only some characters it chokes on, or that when we switched hosting something got weird with the database. I have a suspicion it's the second option, but will need to check to make sure.

rjmagley commented 1 year ago

Tried this on BNR as a test, this is the result:

Traceback (most recent call last):
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)
MySQLdb.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8mb4_0900_ai_ci,COERCIBLE) for operation 'like'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/views.py", line 203, in update
    new_game = page_parser.update(game)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/BNRPageParser.py", line 44, in update
    return self.download_and_update(game.thread_id, page)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/BNRPageParser.py", line 32, in download_and_update
    game = self.parse_page(thread, threadid)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/BNRPageParser.py", line 91, in parse_page
    post_parser.read_votes(post, self.gamePlayers, self.players)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/PostParser.py", line 61, in read_votes
    self.search_line_for_actions(post, line)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/PostParser.py", line 24, in search_line_for_actions
    vote.target = self.autoresolve_vote(vote.target_string, post.game)
  File "/home/epsilon/Documents/projects/votefinder/votefinder/main/PostParser.py", line 66, in autoresolve_vote
    player = Player.objects.get(bnr_uid__isnull=False, name__iexact=text)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/models/query.py", line 633, in get
    num = len(clone)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/models/query.py", line 380, in __len__
    self._fetch_all()
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/home/epsilon/Documents/projects/votefinder/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8mb4_0900_ai_ci,COERCIBLE) for operation 'like'")

Definitely is choking on that particular character.

rjmagley commented 1 year ago

The same error occurs when I try to cast a vote using "ε", which has definitely worked before!