tag1consulting / goose

Load testing framework, inspired by Locust
https://tag1.com/goose
Apache License 2.0
813 stars 71 forks source link

Requests-log with csv format gives malformatted files #515

Closed mTsBucy1 closed 2 years ago

mTsBucy1 commented 2 years ago

The CSV request logs generated by goose include a RAW column that does not properly escape quotes causing many csv readers to confuse column headers.

In particluar this is the first two rows of the CSV requests log unittest. (Running cargo test --package goose --test logs -- test_requests_logs_csv --exact --nocapture after commenting out the cleanup of files https://github.com/tag1consulting/goose/blob/main/tests/logs.rs#L316:L322).

elapsed,raw,name,final_url,redirected,response_time,status_code,success,update,user,error,coordinated_omission_elapsed,user_cadence
1,"GooseRawRequest { method: Get, url: "http://127.0.0.1:65508/", headers: [], body: "" }","/","http://127.0.0.1:65508/",false,5,200,true,false,0,,0,0
7,"GooseRawRequest { method: Get, url: "http://127.0.0.1:65508/error", headers: [], body: "" }","/error","http://127.0.0.1:65508/error",false,0,503,false,false,0,503 Service Unavailable: /error,0,0

Which both Google Spreadsheet and Microsoft Excel interpret as (shortened for clarity)

+──────────+───────────────────────────────────────────+──────────────+──────────────+─────────────+
| elapsed  | raw                                       | name         | final_url    | redirected  |
+──────────+───────────────────────────────────────────+──────────────+──────────────+─────────────+
| 1        | GooseRawRequest { method: Get, url: ..."  | headers: []  | body: "" }"  | /           |
| 7        | GooseRawRequest { method: Get, url: ..."  | headers: []  | body: "" }"  | /error      |
+──────────+───────────────────────────────────────────+──────────────+──────────────+─────────────+

Which clearly is not desireable, since now the rest of the csv heads are shifted over by two columns.

The solution is to propertly escape quotes (see this stackoverflow answer) resulting in this csv output (notice the doubled up quotes in the raw column). This gets property interpreted by both Google Docs and Excel.

elapsed,raw,name,final_url,redirected,response_time,status_code,success,update,user,error,coordinated_omission_elapsed,user_cadence
1,"GooseRawRequest { method: Get, url: ""http://127.0.0.1:65508/"", headers: [], body: """" }","/","http://127.0.0.1:65508/",false,5,200,true,false,0,,0,0
7,"GooseRawRequest { method: Get, url: ""http://127.0.0.1:65508/error"", headers: [], body: """" }","/error","http://127.0.0.1:65508/error",false,0,503,false,false,0,503 Service Unavailable: /error,0,0
jeremyandrews commented 2 years ago

Thanks for the bug report. I imagine there's a library out there that would do a better job writing in CSV format, but as our needs are so minimal my goal has been to do so without adding yet another dependency. I'm open to fixing this manually the same as all CSV writing currently, or to adding a new dependency.