amishas157
commented
2 months ago Steps outlined in the document added in this PR. I will add screenshots and links for confirming test results
Library in GCS bucket
JS UDF
Results confirmations:
Sample check against horizon (testnet) ✅
- transaction_hash: f5f5b0aaf758896ef8c5b4807f41c77d15c11977eecf2b0e4769d777324a2d11
js_udf_result: MCBD54KAHHA4AK4DOZWOSX5O5OZ4OI54N24QITDSFLPD7EG2WY2AMAAACYGGCDRL6UBUA
horizon_result: MCBD54KAHHA4AK4DOZWOSX5O5OZ4OI54N24QITDSFLPD7EG2WY2AMAAACYGGCDRL6UBUA
Next steps:
- [x] Discuss and get a review from the team
- [ ] Add js library to final bucket. Bucket to be decied.
- [x] Add SQL snippet to update column from the results
- [x] Run the query for historical data
PR Checklist
### PR Structure - [ ] This PR has reasonably narrow scope (if not, break it down into smaller PRs). - [ ] This PR avoids mixing refactoring changes with feature changes (split into two PRs otherwise). - [ ] This PR's title starts with the jira ticket associated with the PR. ### Thoroughness - [ ] This PR adds tests for the most critical parts of the new functionality or fixes. - [ ] I've updated the README with the added features, breaking changes, new instructions on how to use the repository. I updated the description of the fuction with the changes that were made. ### Release planning - [ ] I've decided if this PR requires a new major/minor/patch version accordingly to [semver](https://semver.org/), and I've changed the name of the BRANCH to release/_ , feature/_ or patch/\* .What
This PR add instructions on backfill historical data using tx_meta
Why
Airflow runs can be time and money consuming. To ease up the migrations, JS UDF can be convenient.
Known limitations
To be figures: Cost of UDF (they should be cheap most likely)