search

blockchain-etl / polygon-etl

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub
MIT License
101 stars 76 forks source link

set type to string for records with fields #137

Closed rstevens2022 closed 1 year ago

rstevens2022 commented 1 year ago

Hi! In the original Seaport tables for v1.1 on Polygon - for fields that were "RECORD" type data was coming back as NULL, in the Ethereum Seaport tables and also when I run the Seaport contract through a parser, I see those fields as "STRING" which I think might be contributing to the NULL values being seen

charlielewisme commented 1 year ago

I agree. Thank you for raising this.

Let me quote my colleague @TimNooren, as reference for yourself and others:

The problem is that we do not recursively parse trace inputs like we do for logs. You compare the javascript UDF for logs with the one for traces (note no recursion in the latter). This means that we return structs as actual objects for logs, but as plain (json?) strings for traces. The problem occurs when the BigQuery table schema for a trace definition does specify the whole struct schema, since it will just receive a string. I think this leads to all fields of the struct being null.