gouline / dbt-metabase

dbt + Metabase integration
https://pypi.org/project/dbt-metabase/
MIT License
462 stars 71 forks source link

Compatibility with Metabase newest version 0.40 #33

Closed maxenceroux closed 3 years ago

maxenceroux commented 3 years ago

Metabase 0.40 has breaking changes: /api/database does not anymore return an array but now returns a dictionary. /api/fields/{id}does not anymore return an array but now returns a dictionary.

z3z1ma commented 3 years ago

Hello @maxenceroux

Thanks for posting. I believe @gouline has accounted for this already in the branch available on github. We are awaiting wrap up on any remaining breaking changes / big PRs before pushing to pypi.

I would challenge goulline though that we see what a paginated response with more than 1 page looks like so we can build a loop that appends to the results set

a scribbled example below (I just havent seen the response body from metabase for a multi paged paginated object yet)

response = requests.request(
    method, f"{self.protocol}://{self.host}{path}", verify=self.verify, **kwargs
)
response_json = json.loads(response.text)
if "data" in response_json:
  while bool(response_json.get("next_page")): 
    next_resp = requests.request(...).json()  # btw as I understand the .json() method in the requests library is preferred and more robust but either way is fine
    response_json["data"].append(next_resp["data"])
    response_json["next_page"] = next_resp.get("next_page")
  return response_json["data"]

that way our api interface is fully agnostic to pagination for edge cases where we see it.

gouline commented 3 years ago

As @z3z1ma mentioned, this is already addressed in #32 released as 0.7.1.

gouline commented 3 years ago

Sorry, @z3z1ma only read the second half of your response now (skimmed through it before).

From what I can see, none of the Metabase endpoints we use implement pagination yet, the new response structure seems to just allow for it. No mention of it in the documentation either. Considering how their API evolves quickly without warning, I prefer only implementing the functionality that we need now.