ParabolInc / parabol

Free online agile retrospective meeting tool
https://www.parabol.co/
Other
1.87k stars 326 forks source link

Chronos: errors on SendBatchNotificationEmails and UpdateOAuthTokens #9755

Open rafaelromcar-parabol opened 1 month ago

rafaelromcar-parabol commented 1 month ago

Issue - Bug

App version v7.31.0

🌱 Chronos Job autoPause: STARTED                                                                                                                                                                                  
🌱 Chronos Job dailyPulse: STARTED                                                                                                                                                                                 
🌱 Chronos Job weeklyPulse: STARTED                                                                                                                                                                                
🌱 Chronos Job batchEmails: STARTED                                                                                                                                                                                
🌱 Chronos Job scheduleJobs: STARTED                                                                                                                                                                               
🌱 Chronos Job updateTokens: STARTED                                                                                                                                                                               
🌱 Chronos Job processRecurrence: STARTED                                                                                                                                                                          

🌾🌾🌾 Server ID: 175. Ready for Chronos           🌾🌾🌾                                                                                                                                                          
Trace: SEND TO SENTRY TIMEOUT {"query":"mutation SendBatchNotificationEmails { sendBatchNo"} undefined                                                                                                             
    at sendToSentry (/home/node/parabol/dist/chronos.js:17395:11)                                                                                                                                                  
    at publishWebhookGQL (/home/node/parabol/dist/chronos.js:17366:28)                                                                                                                                             
Trace: SEND TO SENTRY TIMEOUT {"query":"mutation SendBatchNotificationEmails { sendBatchNo"} undefined                                                                                                             
    at sendToSentry (/home/node/parabol/dist/chronos.js:17395:11)                                                                                                                                                  
    at publishWebhookGQL (/home/node/parabol/dist/chronos.js:17366:28)                                                                                                                                             
Trace: SEND TO SENTRY TIMEOUT {"query":"mutation UpdateOAuthTokens($updatedBefore: DateTim"} undefined                                                                                                             
    at sendToSentry (/home/node/parabol/dist/chronos.js:17395:11)                                                                                                                                                  
    at publishWebhookGQL (/home/node/parabol/dist/chronos.js:17366:28)

Datadog logs

Acceptance Criteria (optional)

Chronos do not show the error anymore Triage is performed to find the root cause of the bug, timeboxed to ~1 hour.

Estimated effort: 1 hour to triage. More if root cause is already identified.

Dschoordsch commented 1 month ago

Looks like it just took long and timed out. We should investigate why it's so slow.

jordanh commented 1 month ago

Assigned priority during scrub