search

GSS-Cogs / gss-data-docs

Documentation for GSS data project
0 stars 0 forks source link

CSVW Schema Validation webapp #5

Open ajtucker opened 5 years ago

ajtucker commented 5 years ago

A web form to allow developers to make use of ODI's csvlint without having to install it locally.

ajtucker commented 5 years ago

Initial stab is at https://git.floop.org.uk/ONS/csvlint-webapp.

Current issue is that when using csvlint on the commandline, we've managed to make it work in the forked version by providing only a schema and ensuring that the schema points to the CSV file to be checked, while with the library, there seems to be some logic that both schema and CSV file should be provided but if so, the schema is ignored.

The code at https://github.com/theodi/csvlint.rb#user-content-schema-validation doesn't work (one thing is that dialect needs to be {}, not nil) I think because https://github.com/ONS-OpenData/csvlint.rb/blob/master/lib/csvlint/validate.rb#L94 checks for a particular type of schema.

ajtucker commented 5 years ago

Initial stub deployed at https://csvlint.floop.org.uk

ajtucker commented 5 years ago

Got it working. Need to ensure that the uploaded files get put in the same (temp) directory so that the reference from the schema to the CSV file works.

Still need to:

  1. Prettify the output.
  2. Ensure that uploaded files get saved in the temp directory only.
  3. Validate form input.
ajtucker commented 5 years ago

Sanitized filenames of uploaded files, to avoid vulnerabilities.

ajtucker commented 5 years ago

Need to remove error messages about mimetypes.