DealsFlow stream never stops running

[GeorgeTouros]

This change seems to have broken our data pipeline using Stitch. Since March 12, we have been experiencing infinite retries on the object, returning NULL values and status 200, causing the pipeline to continue running in perpetuity. See the latency logs attached.

The max number of retries seems to also not work here, because every few times we get a response with 1 row, thus re-initiating the cycle all along.

Originally posted by @GeorgeTouros in https://github.com/singer-io/tap-pipedrive/issues/131#issuecomment-1491533729

[kethan1122]

@GeorgeTouros Thanks for reporting the issue. I would like to know what could be the reason for getting Null response with 200 status code from Pipedrive API? because I couldn't find anything about this kind of behaviour on Pipedrive docs. Do you have any more insights on this ?

I've opened a support ticket on Developer community for this. Please add if you have more info.

Thanks

[GeorgeTouros]

@kethan1122 I guess it's for cases where it's valid not to get any new data? It should have been 204, but perhaps they have implemented it differently?

[kethan1122]

@GeorgeTouros The reason for implementing retries for this scenario is that we were getting the valid content upon retrying the same request for one of our clients.

And we were also retrying for only 3 times with a factor of 3. If we get same None response even on the 3rd retry then it should fail with TypeError: argument of type 'NoneType' is not iterable.

I see in your case it's not failing but kept on running for longer without completing which means there could be many dealIds for which API isn't able to send the data on first request itself.

Did u also experience this error TypeError: argument of type 'NoneType' is not iterable on the earlier versions of v1.1.8 ? I strongly believe you should have experienced it since you see retries for /dealflows with v1.1.8

[GeorgeTouros]

@kethan1122 we have been receiving (at least 1-2 times per week) some general error-500 notifications in the past, without explanations:

HTTP-error-code: 500, Error: Internal Server Error occurred. Pipedrive staff was notified about this.

We would also get this error pretty often (like once per two weeks):

HTTP-error-code: 503, Error: Schedule maintenance on Pipedrive's end.

These have stopped since March 27th, so I guess they are somehow connected with this fix?

[kethan1122]

Thanks @GeorgeTouros . I'm not sure how to proceed with this issue as the retry solution is kind of getting us rid of TypeError: argument of type 'NoneType' is not iterable. I wrote to Pipedrive's support to get the reason behind sending None as API response. I would wait until they respond. Thanks for your inputs 👍

[GeorgeTouros]

Hi @kethan1122 Just an FYI that today we received a different error

HTTP-error-code: 502, Error: Unknown Error

I dug a bit in the logs and saw the following:

2023-04-26 17:33:50,144Z tap - INFO HTTP request to "dealflow" endpoint took 0.271s, returned status code 200 2023-04-26 17:33:51,336Z tap - INFO METRIC Point(metric_type='timer', metric='http_request_duration', value=0.1731860637664795, tags={'endpoint': 'dealflow', 'status': 'failed'}) 2023-04-26 17:33:51,337Z tap - CRITICAL HTTP-error-code: 502, Error: Unknown Error

Could that be related to our discussion?

[kethan1122]

Hello @GeorgeTouros, I hope you're doing well.

Yeah, I could see there are 17 more connections(of different stitch customers) failed with same error. Pipedrive hasn't provided any information about 502 error in their docs but all the 5xx errors would be caused by the server(Pipedrive's) itself. I hope your connection had picked up from where it left off in the very next extraction

[GeorgeTouros]

It did, thankfully.

[kethan1122]

Hello @GeorgeTouros Is it safe to say that your connection is not facing this issue anymore ? I hope you've reselected deal_flows object. If that's case can u please close this issue ?