tortoise / tortoise-orm

Familiar asyncio ORM for python, built with relations in mind
https://tortoise.github.io
Apache License 2.0
4.37k stars 355 forks source link

Custom generated primary key (Snowflake) ignored, a serial is saved in the backend #1647

Closed sakuragasaki46 closed 5 days ago

sakuragasaki46 commented 2 weeks ago

Describe the bug I have a function that generates Snowflakes (i.e. long integers based on timestamp) for use as PK's in the database. When I call .create() on a database model, looks like the function (passed as callable in the default= parameter of the BigIntField) is ignored altogether and a serial number (like 1, 2, 3...) is generated instead in the backend.

Proof:

Schermata del 2024-06-13 01-08-23_e

To Reproduce I have a custom function to generate a Snowflake (64 bit integer), like this:

machine_counter = 0
machine_id =  0 # whatever value

def new_id():
    global machine_counter

    return (
        ((int(time.time() * 1000) - epoch) << 22) | 
        ((machine_id % 32) << 17) |
        ((os.getpid() % 32) << 12) |
        ((machine_counter := machine_counter + 1) % 1024)
    )

I have this line I copy paste in every Model as a primary key field (please note new_id passed as callable):

class User(Model): # or whatever
    id = fields.BigIntField(primary_key=True, default=new_id)
    username = fields.CharField(32, unique=True, validators=[RegexValidator('^[a-z_][a-z0-9_]+$', 0)])
    ...

I did not supply id = new_id() as argument to create(), therefore creation looks like this:

async def foo():
    ...
    u = await User.create(
        username = username,
        # ...
    )

Expected behavior When calling Model.create() with all the appropriate fields, the PK should be populated with a snowflake value.

Additional context I use PostgreSQL (asyncpg) as the backend, Quart web framework, Python 3.12. I use the Aerich migration tool.

sakuragasaki46 commented 2 weeks ago

Update Passing generated=False as field parameter for the PK makes things work.

I updated the lines:

class User(Model): # or whatever
    id = fields.BigIntField(primary_key=True, default=new_id)

to:

class User(Model): # or whatever
    id = fields.BigIntField(primary_key=True, default=new_id, generated=False)
abondar commented 5 days ago

In documentation we have note about it

If this is used on an Integer Field, ``generated`` will be set to ``True`` unless you explicitly pass ``generated=False`` as well.

generated=True enforces generation of pk value on db side, glad you figured it out