tobymao / saq

Simple Async Queues
https://saq-py.readthedocs.io/en/latest/
MIT License
586 stars 41 forks source link

cron jobs running at millisecond interval in tests #108

Closed euri10 closed 11 months ago

euri10 commented 11 months ago

Greetings, I have saq running cron jobs only.

While testing the job, I sometimes have some flaky tests.

So looking at the logs I discovered that when failing I can see saq is running the very same cron job on a very short interval, I'm talking microseconds according to the logs, while my test fixture is supposed to run it every second.

Below this is the way I'm using the fixture below in this test

so at first I thought I may have another saq process (my dev env) that interferes with my tests, but I'm pretty confident it doesnt pick any job, queues names are different, I dont see any jobs processed in it,

I secondly thought maybe invoking await worker.schedule() would "add" jobs but it is flaky commented or not.

So I'm at a loss as to why the very same job supposed to run every second is running twice milliseconds appart sometimes

Hope you may have an idea :)

@pytest.mark.anyio
async def test_order_seller_expired_before_approved(
    _ = asyncio.create_task(worker.start())
    # await worker.schedule()
    await worker.process()
    await asyncio.sleep(5)

@pytest.fixture(scope="session")
async def saq_queue(
    app_settings_test: AppSettings,
) -> AsyncGenerator[Queue, None]:
    _redis = Queue.from_url(app_settings_test.redis_url, name="worker-tests")
    yield _redis
    await _redis.redis.flushall()
    await _redis.disconnect()

@pytest.fixture
async def worker(
    saq_queue: Queue, app_settings_test: AppSettings
) -> AsyncGenerator[Worker, None]:
    from .worker import check_not_paid_not_expired
    from .worker import startup, shutdown

    _worker = Worker(
        saq_queue,
        functions=[],
        cron_jobs=[
            CronJob(check_not_paid_not_expired, cron="* * * * * */1", unique=False),
        ],
        startup=partial(startup, app_settings=app_settings_test),
        shutdown=shutdown,
    )
    yield _worker
    await _worker.stop()
    logger.debug("worker end")
tobymao commented 11 months ago

set unique to true

euri10 commented 11 months ago

ok this is it, I think I misunderstood its meaning and thought it would run only once :) thanks !