DeputyApp / singer-tap-deputy

Deputy Stitch Stream
MIT License
3 stars 7 forks source link

INTS-1482 - Support for JSON data types #14

Open RhysHutchison opened 1 year ago

RhysHutchison commented 1 year ago

What?

Issue raised by #13 where recent changes to the EmployeeAgreement & PayRules resource types include new data types that aren't currently supported within this tap.

Why?

Suggestion made by P-Arnold to add support for the following data type within the tap proves to be ideal, however the current default value being returned is that of a string array.

Resulting in the underlining execution mishandling the value Config: [] does not match {'type': ['object', 'null']}

Testing

Without making any changes, using the existing implementation; image

Excluding any attribute that includes the json data type; image

kethan1122 commented 1 year ago

@RhysHutchison Thanks for this PR. Would like to understand why Deputy returns "[]" instead of null/None and Is there any plan to change the default nature from empty string array to None ?

Thanks

kethan1122 commented 1 year ago

and also I see this is happening for field Config. Is this a new field you guys added in your recent changes or an already existing field for which the data type has been changed ?

And can you point me to your documentation page where you people updated about these recent changes ?

Thanks

RhysHutchison commented 1 year ago

@kethan1122, we recently added the config attribute to several resources, in order to support an upcoming product feature. Unknowingly this broke the existing tap we've previously released.

This initial change PR was allow customer to continue using the tap whilst our team revisits the attribute in question at a later date.

RhysHutchison commented 1 year ago

I've spoken directly with the team responsible for this new feature and they're raised an internal PR which will reflect null instead of the current json encoded string array.

I'll update this change in order to support the new json data type that supports a string fallback.

kethan1122 commented 1 year ago

Thanks @RhysHutchison. Is there a documentation available to more understand the data structure of this new attribute ?

RhysHutchison commented 1 year ago

I've reached out to my counterparts back in Australia to provide proper documentation for you and your team surrounding this one.

kethan1122 commented 1 year ago

Thanks @RhysHutchison, really appreciate your support 🙇‍♂️