batch processing project

[fronzec]

Create a project example of batch processing in go, similar capabilites provided by spring batch

Tools to consider

• https://github.com/chararch/gobatch
• https://github.com/MasterOfBinary/gobatch
• https://github.com/topics/batch-processing?l=go
• https://github.com/viney-shih/goroutines
• https://github.com/dhrp/moulin
• https://github.com/Deeptiman/go-batch
• https://github.com/elgopher/batch

In each tool consider

• latest update
• is still supported
• dependencies
• is secure code and doesnt have vulnerabilites

Jobs

Job 1

Step 1

• read from csv file
• process add new info
• write to db table1 Step 2
• read from table 1
• process adding
• write to table 2 Step 3
• read from table 2
• process to convert dto
• write dispatch in batch

Job 2

Step 1

• read from table 2, grouping by some condition
• write to table 3 grouped data

Step 2

• read from table 3 grouped data
• write send over rest api group generated

Step 3

• read group sent
• write update fk id in table 2 with the id in table 3 group id

[fronzec]

choosed https://github.com/chararch/gobatch