search

tl-its-umich-edu / canvas-course-manager-next

Canvas Course Manager Next: A redesign of the existing CCM application. It extends Canvas features, makes cumbersome features easier to use, and adds new features.
9 stars 10 forks source link

Do we want to allow quotation marks within cells? #274

Open ssciolla opened 2 years ago

ssciolla commented 2 years ago

As it currently stands, our configuration of papaparse allows for the quoting of cells. However, this introduces some unexpected behavior with respect to quotation marks within a cell. The escapeChar has to be used to escape quotations, which would require some tricky pre-processing before passing to papaparse (unless it's just left to the user). I see two main options:

1) Disallow double quotes within cells. This is essentially what we're doing now. This would allow cell quoting to work, but not allow quotes in strings. This could be problematic for section (Create Sections) and assignment names (Format Third Party Gradebook), but they could use the escape character if they wanted to (we'd have to document it). 2) Disallow cell quoting. This would make it so all quotation marks are treated literally in text. This would be done by setting quoteChar to '' (empty string) in the papaparse config. This could be problematic for users that want to use commas inside of fields (not sure of the use case there).

See https://www.papaparse.com/docs#config and https://github.com/mholt/PapaParse/issues/430

ssciolla commented 2 years ago

This needs more research, testing.

lsloan commented 2 years ago

I think CCM should support whatever occurs most commonly in CSV files. MS Excel is probably the most common tool for producing CSVs. So, if MS Excel allows double quotes within cells, then our code should, too. Likewise, if it doesn't allow it, ours shouldn't either.

I hate to suggest we follow anything MS does, but I recognize their app is probably used by most people for this purpose. If there were a true standard for CSV file formats, then I'd suggest we follow that. Last time I checked, no such standard exists.

Update: CSV is not fully standardized, but some work has been done in that direction. Whether MS Excel follows the standard or not is another question. The informational RFC is available from the IETF at (https://datatracker.ietf.org/doc/html/rfc4180). That may help provide answers to some of the questions in this issue, but it's probably still necessary to cater to the pervasive MS format.

pushyamig commented 2 years ago

Will re-visit this issue if user complain about this. Will add some info in documentation about the use of double quotes.