Error processing `delete_log` when there are no entries

[cayleyh]

Got this error on the first run against our Pipedrive instance:

2018-03-19 22:35:17,173Z    tap - INFO HTTP request to "delete_log" endpoint took 0.153s, returned status code 200
2018-03-19 22:35:17,174Z    tap - INFO replicated 0 records from "delete_log" endpoint
2018-03-19 22:35:17,174Z    tap - CRITICAL Errors during transform
2018-03-19 22:35:17,174Z    tap -   : None does not match {'properties': {'id': {'type': 'integer'}}, 'type': 'object'}
2018-03-19 22:35:17,174Z    tap - 
2018-03-19 22:35:17,174Z    tap - 
2018-03-19 22:35:17,174Z    tap - Errors during transform: [: None does not match {'properties': {'id': {'type': 'integer'}}, 'type': 'object'}]
2018-03-19 22:35:17,175Z    tap - Traceback (most recent call last):
2018-03-19 22:35:17,175Z    tap -   File "tap-env/bin/tap-pipedrive", line 11, in <module>
2018-03-19 22:35:17,176Z    tap -     sys.exit(main())
2018-03-19 22:35:17,176Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_pipedrive/cli.py", line 30, in main
2018-03-19 22:35:17,176Z    tap -     raise e
2018-03-19 22:35:17,176Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_pipedrive/cli.py", line 27, in main
2018-03-19 22:35:17,176Z    tap -     main_impl()
2018-03-19 22:35:17,176Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_pipedrive/cli.py", line 22, in main_impl
2018-03-19 22:35:17,176Z    tap -     pipedrive_tap.do_sync()
2018-03-19 22:35:17,177Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_pipedrive/tap.py", line 94, in do_sync
2018-03-19 22:35:17,177Z    tap -     row = optimus_prime.transform(row, stream.get_schema())
2018-03-19 22:35:17,177Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/transform.py", line 94, in transform
2018-03-19 22:35:17,177Z    tap -     raise SchemaMismatch(self.errors)
2018-03-19 22:35:17,177Z    tap - singer.transform.SchemaMismatch: Errors during transform
2018-03-19 22:35:17,177Z    tap -   : None does not match {'properties': {'id': {'type': 'integer'}}, 'type': 'object'}
2018-03-19 22:35:17,178Z    tap - 
2018-03-19 22:35:17,178Z    tap - 
2018-03-19 22:35:17,178Z    tap - Errors during transform: [: None does not match {'properties': {'id': {'type': 'integer'}}, 'type': 'object'}]

Looks like the delete_log is empty, and iterate_response in tap.py doesn't handle the empty format / case correctly (looks from the error above that it's returning [None] or similar).

[cayleyh]

I wasn't able to reproduce this as calls to delete_log on our Pipedrive are now returning content, but I assume there is still an underlying issue with empty delete_log calls.

However, Stitch continued to fail on this error & I had to end up deleting & recreating the integration to get it to start syncing again.

[ajmers]

It looks like there might be elements in that iteration with a structure like { 'data': None } (the 'recent' stream picks out a data property from the result). I think we can safely just skip over those (not sure why a structure like that would get returned). PR: https://github.com/singer-io/tap-pipedrive/pull/27