GtheSheep / tap-tableau

Tap for Tableau built using the Meltano SDK
GNU General Public License v3.0
1 stars 4 forks source link

Flattening workbook view data #34

Open Ahaberling opened 1 year ago

Ahaberling commented 1 year ago

Hello G my favorite sheep,

I adjusted your tap a little bit for our use case. Having the "workbooks" stream returning the view data as a list, turned out a little bit inconvenient for us. I excluded it from the "workbooks" stream and created a separate stream for it. Are you interested in a PR implementing this change in your tap as well, or do you prefer your current version?

Cheers Alex

Ahaberling commented 1 year ago

Ps. sorry for not replying to the other issue, I forgot about it. The tap is and has been working very well for us since we started using it this year. We are starting to pull more data entities right now

GtheSheep commented 1 year ago

Hey @Ahaberling - do you mean flattening it out into a child stream? did it make an extra api request to gain further details about the view? sounds interesting!! Sounds like a cleaner way than this big JSON list that's there now 😸

and no worries! glad to hear it's been working ok 👌

Ahaberling commented 1 year ago

Right now Im just making the same api call as in the "workbooks" stream and then filter the records differently in order to arrive at a schema like this:

schema = th.PropertiesList( th.Property("id", th.StringType), th.Property("name", th.StringType), th.Property("view.id", th.StringType), th.Property("view.name", th.StringType), th.Property("view.total_views", th.NumberType), th.Property("view.content_url", th.StringType), th.Property("view.owner_id", th.StringType), ).to_dict()

The first two properties refer to workbook id and name. If I find some times the next days, I will try to find a more elegant solution that does not simply replicate the api call