use goroutines to speed up bulk copy 2X (~5mins for me and i2.txt)
How to review
go test -count 1 -v -tags comptest ./dataingest/addtodb -run TestAddGeoGeoMetricData
(note this test uses manual rollback since I couldn't get the pgxpool rollback to fully work)
If you are keen create a database the old (no go routines) way as "census" and the new way as "censusgo2i" and run "dataingest/datasanity/import/data_test.go" a few times (with no caching) to check random rows
What
use goroutines to speed up bulk copy 2X (~5mins for me and i2.txt)
How to review
go test -count 1 -v -tags comptest ./dataingest/addtodb -run TestAddGeoGeoMetricData
(note this test uses manual rollback since I couldn't get the pgxpool rollback to fully work)
If you are keen create a database the old (no go routines) way as "census" and the new way as "censusgo2i" and run "dataingest/datasanity/import/data_test.go" a few times (with no caching) to check random rows
Who can review
Anyone