Develop Integration Guide for PgQueuer with Django

janbjorge / PgQueuer

PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.

https://pgqueuer.readthedocs.io/en/latest/index.html

MIT License

208 stars 1 forks source link

Develop Integration Guide for PgQueuer with Django #13

Open janbjorge opened 2 months ago

janbjorge commented 2 months ago

Description

Write a concise guide on integrating PgQueuer with Django to leverage PostgreSQL for task management effectively. This guide will serve as a resource for Django developers to implement PgQueuer in their projects, optimizing task management capabilities. Objectives

Setup Instructions: Steps to integrate PgQueuer within Django
Configuration: Adjustments necessary in Django settings for
Examples: Code examples for defining and enqueuing
Management Commands: Integration of Django commands for PgQueuer
Best Practices: Tips for efficient use and optimal configuration.

baseplate-admin commented 2 months ago

Hi, if you want to support django, i think it will be better if you use the postgres driver that django uses under the hood, psycopg

janbjorge commented 2 months ago

Hi, if you want to support django, i think it will be better if you use the postgres driver that django uses under the hood, psycopg

I think i need to make driver abstraction for this to come out nice. Another approach might be to sniff connections parameters from psycopg and try to establish a connection with asyncpg insted (to only be used by asyncpg).

Ive mostly been using fastapi/flask, so Django is foreign to me.

baseplate-admin commented 2 months ago

The problem is asyncpg is a 3.3MB dependency. Size optimization for django is a must for deploying into aws lambda.

I think i need to make driver abstraction for this to come out nice.

Do you have any particular problem that is not implementable using psycopg? pgadmin itself is based on psycopg, so in theory it should implement every postgres feature

janbjorge commented 1 month ago

I used to use psycopg2 back in the day (before it got async). Since most of my projects moved to async using asyncpg was natural at that time. I do not have any objects to psycopg, but i still want to support asyncpg. So i think i need to spend a bit of time to figure out how i want to make an abstraction for the drivers(psycopg/asyncpg etc...) to plug into.

janbjorge commented 2 weeks ago

Short update, ive added an abstraction layer for database connection. By passing the the a async psycopg connection to a 'psycopg-driver-class'. Still working on a few details to make this neater.

PR: https://github.com/janbjorge/PgQueuer/pull/37