Occasionally, when a sale is reported to Discord, some items will be missing from the list. And there have also been instances where the entire sale is missed.
Context
It seems there is something fundamentally buggy with our architecture. There are a few checks going on to make sure we don't report manual inventory changes. We also check the total number of sales against what we last recorded as the total number of sales. I suspect one of these checks is causing things to go wonky, perhaps a race condition or something.
This is unfortunately difficult to debug. At first, we suspected we were sending too many Discord messages and some were getting dropped. https://github.com/allisonking/wacks-by-warby/commit/62f2757087310250c99bf9e316402ae779970207 changed the architecture to send all sale reports as one message instead of each sale being a message in an attempt to fix this problem. However, the problem still persists, so it does not appear to be the case that Discord was dropping messages.
Suggested path forward
Since this is pretty difficult to debug, I think our first step is to add more logging, perhaps to another file that is easier to consult than the current output log file. We should also take a harder look at our system architecture to see where it may be possible that messages are getting dropped.
What's wrong
Occasionally, when a sale is reported to Discord, some items will be missing from the list. And there have also been instances where the entire sale is missed.
Context
It seems there is something fundamentally buggy with our architecture. There are a few checks going on to make sure we don't report manual inventory changes. We also check the total number of sales against what we last recorded as the total number of sales. I suspect one of these checks is causing things to go wonky, perhaps a race condition or something.
This is unfortunately difficult to debug. At first, we suspected we were sending too many Discord messages and some were getting dropped. https://github.com/allisonking/wacks-by-warby/commit/62f2757087310250c99bf9e316402ae779970207 changed the architecture to send all sale reports as one message instead of each sale being a message in an attempt to fix this problem. However, the problem still persists, so it does not appear to be the case that Discord was dropping messages.
Suggested path forward
Since this is pretty difficult to debug, I think our first step is to add more logging, perhaps to another file that is easier to consult than the current output log file. We should also take a harder look at our system architecture to see where it may be possible that messages are getting dropped.