split-sync impression errors

[sreeramjayan]

We have been receiving the following alerts from split-synchronizer today. Can someone explain why this is happening and what can be done to remediate it? Do I need to configure a timeout?

split-synchronizer version - 5.0.2

Split-Sync - ERROR - 2022/05/05 17:54:02 pipelined.go:265: [pipelined/impressions] error posting: Post "https://events.split.io/api/testImpressions/bulk": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Split-Sync - ERROR - 2022/05/05 17:54:02 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/05 17:54:02 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/05 17:54:03 pipelined.go:265: [pipelined/impressions] error posting: Post "https://events.split.io/api/testImpressions/bulk": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Split-Sync - ERROR - 2022/05/05 17:54:03 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/05 17:54:03 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400

[chillaq]

Hi @sreeramjayan , I don't see any incidents on our side on 5/5, it looks like a network time out error, did the errors only occur at that time? It might be a transient network issue.

Thanks Bilal

[sreeramjayan]

These errors are still occurring today and they began on May 5th 2022.

Split-Sync - ERROR - 2022/05/17 16:04:38 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/17 16:04:38 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/17 21:30:48 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 502
Split-Sync - ERROR - 2022/05/17 21:30:48 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/17 21:30:48 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/17 21:55:30 pipelined.go:265: [pipelined/impressions] error posting: Post "https://events.split.io/api/testImpressions/bulk": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Split-Sync - ERROR - 2022/05/17 21:55:30 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/17 21:55:30 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/18 19:23:29 pipelined.go:265: [pipelined/impressions] error posting: Post "https://events.split.io/api/testImpressions/bulk": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Split-Sync - ERROR - 2022/05/18 19:23:29 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/18 19:23:29 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/18 19:23:31 pipelined.go:265: [pipelined/impressions] error posting: Post "https://events.split.io/api/testImpressions/bulk": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Split-Sync - ERROR - 2022/05/18 19:23:31 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400
Split-Sync - ERROR - 2022/05/18 19:23:31 pipelined.go:270: [pipelined/impressions] bad status code when sinking data: 400

[chillaq]

Hi @sreeramjayan , Can you upgrade the synchronizer to latest version (5.0.5)? While I don't see a direct fix to this issue, we did implement lot of improvements on how Synchronizer cache and post impressions.

Is it possible to enable debug logging for Synchronizer? If we can get the error while debugging is turned on, it would be great.

The common issues we see is synchronizer http calls either time out, or receive 500 errors from split cloud, I have not seen 400 errors, which is why this issue is puzzling.

[chillaq]

Hi @sreeramjayan , did you have the chance to upgrade to latest synchronizer? let me know if you still need help.

Thanks

[sreeramjayan]

Sorry for the late response @chillaq. We will be working on this in 2 weeks. Can this issue be open until then?

[chillaq]

Sure, no worries.

[mmelograno]

Hi. @sreeramjayan It has been a while since we heard back from you.

Is there any news regarding updating synchronizer to latest version?

Cheers Matias

[chillaq]

Hi @sreeramjayan , I am following up on this issue, do you still need help?

[chillaq]

Hi @sreeramjayan , I am closing this thread, feel free to reopen if you still see the issue in the latest synchronizer version.