pudo / dataset

Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
https://dataset.readthedocs.org/
MIT License
4.78k stars 298 forks source link

Cannot get newly inserted data with an old connection #244

Closed xg1990 closed 6 years ago

xg1990 commented 6 years ago

I was trying to use the following code to obtain records that are inserted within recent 30 seconds ( using the column start_t)

The problems is that after 30 seconds, the table.find function get zero records. But I am sure there are new records in the database. And I have also tried to use table.query instead of table.find, the issue remains the same.

One temporary solution is to create a new dataset connection using dataset.connect before table.find

import time

import dataset

from config import config

def main():
    db = dataset.connect('mysql://{user}:{password}@{host}:{port}/{database}'.format(
        user = 'root',
        password = config['database']['root_password'],
        host = config['database']['host'],
        database = config['database']['database'],
        port = config['database']['port']
    ) )

    while True:
        re = db.get_table('table_name').find("start_t > {}".format(time.time() - 30))
        print list(re)
        time.sleep(5)

if __name__ == '__main__':
    main()

And I am using the latest dataset:

pip2 install dataset -U
Requirement already up-to-date: dataset in /usr/local/lib/python2.7/site-packages
Requirement already up-to-date: six>=1.11.0 in /usr/local/lib/python2.7/site-packages (from dataset)
Requirement already up-to-date: alembic>=0.6.2 in /usr/local/lib/python2.7/site-packages (from dataset)
Requirement already up-to-date: normality>=0.5.1 in /usr/local/lib/python2.7/site-packages (from dataset)
Requirement already up-to-date: sqlalchemy>=1.1.0 in /usr/local/lib/python2.7/site-packages (from dataset)
Requirement already up-to-date: python-editor>=0.3 in /usr/local/lib/python2.7/site-packages (from alembic>=0.6.2->dataset)
Requirement already up-to-date: python-dateutil in /usr/local/lib/python2.7/site-packages (from alembic>=0.6.2->dataset)
Requirement already up-to-date: Mako in /usr/local/lib/python2.7/site-packages (from alembic>=0.6.2->dataset)
Requirement already up-to-date: chardet in /usr/local/lib/python2.7/site-packages (from normality>=0.5.1->dataset)
Requirement already up-to-date: banal in /usr/local/lib/python2.7/site-packages (from normality>=0.5.1->dataset)
Requirement already up-to-date: MarkupSafe>=0.9.2 in /usr/local/lib/python2.7/site-packages (from Mako->alembic>=0.6.2->dataset)
xg1990 commented 6 years ago

Solved It is because of the isolation of mysql (https://stackoverflow.com/questions/21974627/mysql-connector-not-showing-inserted-results)

Add db.query("SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED") can solve the problem