Add --ramp-up-blocks parameter to the stream command to deal with OOM

blockchain-etl / polygon-etl

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

MIT License

101 stars 76 forks source link

Closed medvedev1088 closed 1 year ago

medvedev1088 commented 1 year ago

Add --ramp-up-blocks parameter to the stream command to deal with OOM

ramp_up_blocks (int, optional): Specifies the count of initial blocks to be processed one by one before switching to batch processing. This approach is particularly beneficial for mitigating issues like Out-of-Memory (OOM) errors when dealing with large batches. Default is 0.