search

viktorminko / microservice-task

0 stars 0 forks source link

Parser fails with gRPC timeout exceeded when feeding big CSV files [#8 was not fixed] #10

Open tb0hdan opened 5 years ago

viktorminko commented 5 years ago

Not sure what is the error your see. How big your files are ? I've tested it on 1 million records and it worked well, the process took about 2 mins on my machine. I've added limit for a number of concurrent goroutines, this might help to resolve your issue. See https://github.com/viktorminko/microservice-task/blob/master/cmd/parser/main.go#L21 Maybe you can try to increase execution time for large files.

tb0hdan commented 5 years ago

Nevermind.

P.S. Test set contains 104 million records