hjpotter92 / sonyflake-py

A distributed unique ID generator inspired by Twitter's Snowflake, rewritten in python
https://pypi.org/project/sonyflake-py/
MIT License
33 stars 10 forks source link

not generating unique id for single machine in celery tasks/threads. #8

Closed anupsingh3292 closed 2 months ago

anupsingh3292 commented 11 months ago

I am using this library to generate primary keys. but this doesnt work in celery tasks. it generates duplicate keys. I dont know how to solve it. if possible then please guide me on how I should do it.

hjpotter92 commented 11 months ago

I am using this library to generate primary keys. but this doesnt work in celery tasks. it generates duplicate keys. I dont know how to solve it. if possible then please guide me on how I should do it.

if celery is set across multiple workers, each worker should be assigned a different machine_id.

anupsingh3292 commented 9 months ago

sorry for late reply.

I was fixed it by snowflake.machind_id=os.getpid function. so have tested it was working I was nit getting duplicate id errors. but sometimes I am getting error sometimes even for a small number of concurrent requests ( like from js webwroker request). as for now i have set it up like this

import os
from sonyflake import SonyFlake

class SnowFlakeIdGenerator:
    @classmethod
    def generate(cls):
        return SonyFlake(machine_id=os.getpid).next_id()

that's it only and nothing extra in celery or anywhere. you can suggest to me what should i do like any additional config. then I will try to run load testing from multiple PCs and will troubleshoot.