Open crystaldust opened 1 year ago
Some data, like github commits, pr, issues are fetched and inserted by pages. Data in one page may not be as large as a batch. Setup a batch and collect these data into batch, when batch full, insert the batch data into opensearch
crystaldust
commented
1 year ago
Some data, like github commits, pr, issues are fetched and inserted by pages. Data in one page may not be as large as a batch. Setup a batch and collect these data into batch, when batch full, insert the batch data into opensearch