Remove the files as the memory for reading csv's and instead use a reader type to capture the csv from the http response body.
We need to validate that we're not getting any kind of a memory leak here. The issue is the response body is a io.ReadCloser which is an interface that satisfy's both the io.Reader and io.Closer types. This allowed us to defer close the reader after reading the file. Right now, we're storing the contents of the response body (interface {reader, closer}) in a Reader type which satisfies the io.ReadCloser; the issue is we now read the file, but do not close it.
I'm not sure if this creates a memory leak or how it will interact with the code running once every day to fetch new csv's and parse them again. So we need to look into this.
closes #9
Remove the files as the memory for reading csv's and instead use a reader type to capture the csv from the http response body.
We need to validate that we're not getting any kind of a memory leak here. The issue is the response body is a io.ReadCloser which is an interface that satisfy's both the io.Reader and io.Closer types. This allowed us to defer close the reader after reading the file. Right now, we're storing the contents of the response body (interface {reader, closer}) in a Reader type which satisfies the io.ReadCloser; the issue is we now read the file, but do not close it.
I'm not sure if this creates a memory leak or how it will interact with the code running once every day to fetch new csv's and parse them again. So we need to look into this.