search

MeltanoLabs / tap-hubspot

Other
1 stars 5 forks source link

enable export of archived records #75

Open bardrink opened 1 month ago

bardrink commented 1 month ago

I would like to ask if you are planning to add the ability to export also archived records from Hubspot. With the use of incremental loading and subsequent historization of data, I am unable to determine which records have been deleted (archived in HubSpot) since the tap is not returning them. These records remain in my raw table (which is incrementally filled) marked as valid, even though they have already been deleted in HubSpot.

I attempted to prepare a workaround and was partially successful, can be found here. I managed to add it in streams that utilize the HubspotStream class, but it is not working in streams that use the DynamicIncrementalHubspotStream class. I checked the code and saw that they use search endpoints for incremental fetch, for which I was not able to implement archived parameter.

Does it make sense to add the ability to fetch archived records and include this option as a parameter? I would really appreciate it. Thank you!

edgarrmondragon commented 4 weeks ago

Hi @bardrink!

This makes sense to me. There are other taps that do similar things, such as tap-clickup.

It's probably a good idea to put it behind a config flag. PRs welcome!