algo: after application start they try to get lock(with params .CreateLockAsync(key, expiryTime: 2 minutes, waitTime: 1minute, retryTime: 30 seconds)) and if this operation ends with success start process distributed queue, also in that time started timer that check lock status every 2 minutes and if it is not aquired then stop processing and try to lock again.
problem: sometimes, quite often after restart, firstly lock gets one instance of app, but after 2 minutes another app gets it too, its not a problem for algo, but first instance still think that isAquired == true, so two instance process data in same time.
problem 2: today after restart lock gets only one app so im delete lock from redis for restart immitation and catch more strange behavior - app that have lock before removal notice that lock lost and try to relock, app 2 gets lock, but then every 2 minutes another app get lock, and after all apps locked they stops to extend, all have 0 extends, and there is no lock key in redis
timelapse, read from down
app5: 2022-02-03 09:48:21.1104|DEBUG|Lock IsAcquired with id 731334ee-2548-4ff3-8f9c-a3036abd12ae
app4: 2022-02-03 09:46:18.7026|DEBUG|Lock IsAcquired with id 8921325f-2dfc-4f18-921a-8aaf0c662eec
app3: 2022-02-03 09:44:09.1532|DEBUG|Lock IsAcquired with id 0405daf7-cf8e-42f4-8838-e03ba5234af6
app1: 2022-02-03 09:41:47.5693|DEBUG|Lock IsAcquired with id 129741b1-f85e-4aa6-9def-95b7d30a1831
app1: 2022-02-03 09:41:17.2084|DEBUG|Lock lost with id d954b8e8-dbec-44a7-80ff-695897faf4b5
app2: 2022-02-03 09:39:45.7486|DEBUG|Lock IsAcquired with id 5368b2ec-a456-49fe-80f8-1894f39610dd
somewhere here lock key deleted from redis
app1: 2022-02-03 09:19:17.1875|DEBUG|Lock IsAcquired with id d954b8e8-dbec-44a7-80ff-695897faf4b5
enviroment: 1 noded redis 5 nodes of app
algo: after application start they try to get lock(with params .CreateLockAsync(key, expiryTime: 2 minutes, waitTime: 1minute, retryTime: 30 seconds)) and if this operation ends with success start process distributed queue, also in that time started timer that check lock status every 2 minutes and if it is not aquired then stop processing and try to lock again.
problem: sometimes, quite often after restart, firstly lock gets one instance of app, but after 2 minutes another app gets it too, its not a problem for algo, but first instance still think that isAquired == true, so two instance process data in same time.
problem 2: today after restart lock gets only one app so im delete lock from redis for restart immitation and catch more strange behavior - app that have lock before removal notice that lock lost and try to relock, app 2 gets lock, but then every 2 minutes another app get lock, and after all apps locked they stops to extend, all have 0 extends, and there is no lock key in redis
timelapse, read from down
app5: 2022-02-03 09:48:21.1104|DEBUG|Lock IsAcquired with id 731334ee-2548-4ff3-8f9c-a3036abd12ae
app4: 2022-02-03 09:46:18.7026|DEBUG|Lock IsAcquired with id 8921325f-2dfc-4f18-921a-8aaf0c662eec
app3: 2022-02-03 09:44:09.1532|DEBUG|Lock IsAcquired with id 0405daf7-cf8e-42f4-8838-e03ba5234af6
app1: 2022-02-03 09:41:47.5693|DEBUG|Lock IsAcquired with id 129741b1-f85e-4aa6-9def-95b7d30a1831
app1: 2022-02-03 09:41:17.2084|DEBUG|Lock lost with id d954b8e8-dbec-44a7-80ff-695897faf4b5
app2: 2022-02-03 09:39:45.7486|DEBUG|Lock IsAcquired with id 5368b2ec-a456-49fe-80f8-1894f39610dd
somewhere here lock key deleted from redis
app1: 2022-02-03 09:19:17.1875|DEBUG|Lock IsAcquired with id d954b8e8-dbec-44a7-80ff-695897faf4b5