davidgasquez / gitcoin-grants-data-portal

🌲 Open source, serverless, and local-first data hub for Gitcoin Grants data!
https://grantsdataportal.xyz/
MIT License
25 stars 3 forks source link

Fix Hypercert error #88

Open davidgasquez opened 6 days ago

davidgasquez commented 6 days ago

Seems the hypercerts endpoint changed.

dagster._core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "raw_hypercert_claims"::

tenacity.RetryError: RetryError[<Future at 0x7f2db20f6010 state=finished raised HTTPError>]

Stack Trace:
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/dagster/_utils/__init__.py", line 468, in iterate_with_context
    next_output = next(iterator)
                  ^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/dagster/_core/execution/plan/compute_generator.py", line 141, in _coerce_op_compute_fn_to_iterator
    result = invoke_compute_fn(
             ^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/dagster/_core/execution/plan/compute_generator.py", line 129, in invoke_compute_fn
    return fn(context, **args_to_pass) if context_arg_provided else fn(**args_to_pass)
                                                                    ^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/gitcoin-grants-data-portal/gitcoin-grants-data-portal/ggdp/assets/other.py", line 190, in raw_hypercert_claims
    result = get_hypercerts(HYPERCERTS_ENDPOINT, window_creators)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()

The above exception was caused by the following exception:
requests.exceptions.HTTPError: 410 Client Error: Gone for url: https://api.thegraph.com/subgraphs/name/hypercerts-admin/hypercerts-optimism-mainnet

Stack Trace:
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/gitcoin-grants-data-portal/gitcoin-grants-data-portal/ggdp/assets/other.py", line 171, in get_hypercerts
    response.raise_for_status()
  File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
davidgasquez commented 6 days ago

@DistributedDoge, seems the endpoint has been removed.

https://thegraph.com/hosted-service/subgraph/hypercerts-admin/hypercerts-optimism-mainnet

Getting this message from their GraphiQL.

This endpoint has been removed. If you are the owner of this subgraph you can upgrade it by going here: https://thegraph.com/hosted-service. If you have any questions, reach out to support@thegraph.zendesk.com. Learn more about why this happening here: https://thegraph.com/docs/en/sunrise/#upgrading-subgraphs-to-the-graph-network

Any ideas/thoughts?

DistributedDoge commented 6 days ago

Problem is The Graph rolled-out protocol wide upgrade where every subgraph either migrates to hosted service or stops working. I believe this migration has to be done by subgraph author, not by us as consumers.

https://thegraph.com/blog/sunbeam-upgrade-window/

The good news is that all hypercerts are created by a single contract which emits after every mint so using Covalent or RPC-node we can get the same information directly from on-chain events.

davidgasquez commented 5 days ago

Thanks for digging and sharing the details! :mag:

I wasn't aware at all of these changes going on at The Graph.

Relying on Covalent sounds good! For now, I've disabled them as they were blocking all other assets.