Closed DistributedDoge closed 7 months ago
What is needed for this PR to be ready? Happy to merge it even if not fully complete!
The only thing needed from your side:
If you want you can consider it ready to merge. Tested new version on the fork, and CI appears to be working, as long as COVALENT_KEY is set up first.
New behaviour:
make run
or fetching assets in make dev
.COVALENT_API_KEY
, CI expects COVALENT_KEY
dagster._config.errors.PostProcessingError: You have attempted to fetch the environment variable "COVALENT_API_KEY" which is not set. In order for this execution to succeed it must be set in this environment.
Problems:
[*]
like we do in make run
. I was hoping asset groups would be a solution, alas I don't think dagster asset selection language used by CLI and python is actually using asset groups at all.
@asset(
compute_kind="Covalent_API",
group_name="chain_data",
)
CI run is actually failing. Back to the drawing board, strange that I didn't see this error maybe your covalent key wasn't pasted right?
dagster._core.errors.DagsterExecutionStepExecutionError: Error occurred while executing op "ethereum_project_registry_tx"::
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.covalenthq.com/v1/eth-mainnet/address/0x03506eD3f57892C85DB20C36846e9c808aFe9ef4/transactions_v3/page/1/
strange that I didn't see this error maybe your covalent key wasn't pasted right?
I still haven't added the Covalent API key. Doing that right now!
Alright, COVALENT_API_KEY
is now on the CI environment! Sorry for the delay there!
If you could kindly squash before merging, or however Github calls it. I don't like leaving a mess in commit history.
Yes! In fact, let me make squashing the default in this repository.
Forked CI with my own API key inserted will succeed now, so the most likely cause of error here is environment mismatch between our versions.
Absolutely on rush to do so, but if you do happen to circle back here after the weekend:
secrets
tab not variables
COVALENT_API_KEY
cqt_something
without any " quotes or anything Weird, there is actually a COVALENT_API_KEY
in the repo since last week.
Going to merge and see if it needs to update the CI on main
for some reason.
Built an asset that fetches all mainnet project registry transactions. Appears to be working locally or inside reconfigured forked CI .
Manual config changes needed for new CI workflow:
COVALENT_API_KEY
set up in here.New asset fetches ~3k transactions, which should cost about 300 covalent credits out of 100k monthly free tier. Takes about 3 minutes to run, but in parallel with other assets doesn't affect overall speed to much.