The usage metrics update for 2024-10-21 seems to have hit a snag in the github metrics. Maybe the data coming back from the API changed? It says the column user_view_type doesn't exist when attempting to load it into the database.
The above exception was caused by the following exception:
psycopg2.errors.UndefinedColumn: column "user_view_type" of relation "core_github_stargazers" does not exist
LINE 1: ...repos_url, events_url, received_events_url, type, user_view_...
^
Stack Trace:
File "/home/runner/micromamba/envs/pudl-usage-metrics/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/runner/micromamba/envs/pudl-usage-metrics/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 941, in do_execute
cursor.execute(statement, parameters)
2024-10-21 07:25:46 +0000 - dagster - ERROR - all_nonpartitioned_metrics_etl - 102f9b54-0720-40cb-a105-66d5400aa9aa - 1870 - RUN_FAILURE - Execution of run for "all_nonpartitioned_metrics_etl" failed. An exception was thrown during execution.
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column "user_view_type" of relation "core_github_stargazers" does not exist
LINE 1: ...repos_url, events_url, received_events_url, type, user_view_...
^
[SQL: INSERT INTO core_github_stargazers (id, login, node_id, url, html_url, followers_url, following_url, gists_url, starred_url, subscriptions_url, organizations_url, repos_url, events_url, received_events_url, type, user_view_type, site_admin, starred_at) VALUES (%(id)s, %(login)s, %(node_id)s, %(url)s, %(html_url)s, %(followers_url)s, %(following_url)s, %(gists_url)s, %(starred_url)s, %(subscriptions_url)s, %(organizations_url)s, %(repos_url)s, %(events_url)s, %(received_events_url)s, %(type)s, %(user_view_type)s, %(site_admin)s, %(starred_at)s)]
[parameters: {'id': 27873769, 'login': 'tonezz', 'node_id': 'MDQ6VXNlcjI3ODczNzY5', 'url': 'https://api.github.com/users/tonezz', 'html_url': 'https://github.com/tonezz', 'followers_url': 'https://api.github.com/users/tonezz/followers', 'following_url': '[https://api.github.com/users/tonezz/following{/other_user}](https://api.github.com/users/tonezz/following%7B/other_user%7D)', 'gists_url': '[https://api.github.com/users/tonezz/gists{/gist_id}](https://api.github.com/users/tonezz/gists%7B/gist_id%7D)', 'starred_url': '[https://api.github.com/users/tonezz/starred{/owner}{/repo}](https://api.github.com/users/tonezz/starred%7B/owner%7D%7B/repo%7D)', 'subscriptions_url': 'https://api.github.com/users/tonezz/subscriptions', 'organizations_url': 'https://api.github.com/users/tonezz/orgs', 'repos_url': 'https://api.github.com/users/tonezz/repos', 'events_url': '[https://api.github.com/users/tonezz/events{/privacy}](https://api.github.com/users/tonezz/events%7B/privacy%7D)', 'received_events_url': 'https://api.github.com/users/tonezz/received_events', 'type': 'User', 'user_view_type': 'public', 'site_admin': False, 'starred_at': datetime.datetime(2024, 10, 16, 4, 51, 53, tzinfo=datetime.timezone.utc)}]
(Background on this error at: https://sqlalche.me/e/20/f405)
Success Criteria
How will we know that we're done?
[ ] We can kick off a workflow_dispatch run that updates the metrics and it works.
Overview
The usage metrics update for 2024-10-21 seems to have hit a snag in the github metrics. Maybe the data coming back from the API changed? It says the column
user_view_typedoesn't exist when attempting to load it into the database.Success Criteria
How will we know that we're done?
workflow_dispatchrun that updates the metrics and it works.