Semicolons aren’t escaped in CSV exports

[jsorva]

The reports of student submissions may contain semicolons, but those aren’t escaped in the CSV exports, which creates problems when trying to analyze the reports programmatically.

Example file from O1: https://plus.cs.aalto.fi/api/v2/courses/197/submissiondata/?exercise_id=31374&format=excel.csv

[markkuriekkinen]

I don't understand the problem. The CSV format may use different separators: comma ,, semicolon ; or something else. I didn't notice any escaping issues in the A+ output. That is to say, if the semicolon is used as the separator, then semicolons in the content are correctly escaped. If the comma is used as the separator, then commas in the content are correctly escaped.

The A+ Excel CSV output format (format=excel.csv) is our own customization to the CSVRenderer in the framework (Django REST framework). The framework certainly knows how to escape content correctly since it is a very basic requirement. The A+ Excel CSV format uses the semicolon as the separator by default. The A+ Excel CSV format also accepts an additional optional parameter that changes the separator character: for commas, use the GET query parameter sep=%2C (see below for full examples). This format also removes new lines from the values, which particularly affects the textual feedback fields in submissions.

The CSV format format=csv uses commas by default. This format comes directly from the framework without modifications in A+.

• https://plus.cs.aalto.fi/api/v2/courses/197/submissiondata/?exercise_id=31374&format=csv
• https://plus.cs.aalto.fi/api/v2/courses/197/submissiondata/?exercise_id=31374&format=excel.csv
• https://plus.cs.aalto.fi/api/v2/courses/197/submissiondata/?exercise_id=31374&format=excel.csv&sep=%2C (%2C is comma)

For developers

• submission data view and its renderers: CSVRenderer:
• https://github.com/apluslms/a-plus/blob/709cfb7302921af56e391afb4b162b0fb6094016/exercise/api/csv/views.py#L62-L65
• The A+ CSVExcelRenderer and the default separator:
• https://github.com/apluslms/a-plus/blob/709cfb7302921af56e391afb4b162b0fb6094016/lib/api/renderers.py#L9
• the Excel CSV default separator in the Django settings:
• https://github.com/apluslms/a-plus/blob/709cfb7302921af56e391afb4b162b0fb6094016/aplus/settings.py#L94

[jsorva]

OK, thanks for the clarifications! Maybe things are fine after all and it’s my mistake. I’ll report back if I cannot solve the matter; feel free to close this issue.

[markkuriekkinen]

Closing since there hasn't been new reports about issues.