peter-wangxu / persist-queue

A thread-safe disk based persistent queue in Python
BSD 3-Clause "New" or "Revised" License
335 stars 50 forks source link

FILOSQLiteAckQueue _SQL_SELECT broken #169

Closed juha-ylikoski closed 3 years ago

juha-ylikoski commented 3 years ago

It seems that FILOSQLliteAckQueue's _SQL_SELECT is not formatted properly.

with q.get(raw=True)I'm getting:

  File "/home/*************/.venv/lib/python3.8/site-packages/persistqueue/sqlackqueue.py", line 290, in get
    serialized = self._pop(
  File "/home/*************/.venv/lib/python3.8/site-packages/persistqueue/sqlackqueue.py", line 193, in _pop
    return {'pqid': row[0], 'data': item, 'tiemstamp': row[2]}
IndexError: tuple index out of range

This can be fixed by setting

class FILOSQLiteAckQueue(SQLiteAckQueue):
    """SQLite3 based FILO queue with ack support."""

    _TABLE_NAME = 'ack_filo_queue'
    # SQL to select a record
    _SQL_SELECT = (
        'SELECT {key_column}, data FROM {table_name} '
        'WHERE {key_column} < {rowid} and status < %s '
        'ORDER BY {key_column} DESC LIMIT 1' % AckStatus.unack
    )

->

class FILOSQLiteAckQueue(SQLiteAckQueue):
    """SQLite3 based FILO queue with ack support."""

    _TABLE_NAME = 'ack_filo_queue'
    # SQL to select a record
    _SQL_SELECT = (
        'SELECT {key_column}, data, timestamp, status FROM {table_name} '
        'WHERE {key_column} < {rowid} and status < %s '
        'ORDER BY {key_column} DESC LIMIT 1' % AckStatus.unack
    )

Like in SQLiteAckQueue.

juha-ylikoski commented 3 years ago

I can create fix + tests for this prob later today and create pr. (It seems there are currently no tests for FILOSQLiteAckQueue)

juha-ylikoski commented 3 years ago

After trying to replicate this issue I cannot seem to be able to replicate it. I will try to replicate this tomorrow with my other computer where I found this issue.

juha-ylikoski commented 3 years ago

The problems was with q.get(raw=True. The FILO-queue had missing parts of the row in the query -> index error. I'm gonna submit pr containing fix + tests soon.