Data-Liberation-Front / csvlint.rb

The gem behind http://csvlint.io
MIT License
286 stars 87 forks source link

Validation not working as expected. #260

Open RobertMcGinn opened 2 years ago

RobertMcGinn commented 2 years ago

I have a rails app where you upload a csv file. I want to use Csvlint to validate the format of the file.

In the app controller I have the following:

file_path(connect) -> the uploaded csv file, the contents are as follows:

Date,Time,Product,ISIN,Reference,Venue,Quantity,Price,,Local value,,Value,,Exchange rate,Transaction and/or third,,Total,,Order ID
11-02-2022,11:29,VANGUARD S&P500,IE00B3XXRP09,LSE,MSEU,43,62.7350,GBP,-2697.61,GBP,-2697.61,GBP,,-2.11,GBP,-2699.72,GBP,8b7ae994-e727-4fa2-bb1d-89f7bfc31e6e
14-12-2021,10:49,VANGUARD S&P500,IE00B3XXRP09,LSE,MSEU,17,67.1300,GBP,-1141.21,GBP,-1141.21,GBP,,-2.05,GBP,-1143.26,GBP,eb1f8861-43b9-4de2-8276-21a18f42eedb

In the lib/csv folder I have a file called degiro.csv which includes the following (i am using this for testing purposes)

{
  "@context": "http://www.w3.org/ns/csvw",
  "url": "http://example.com/example1.csv",
  "tableSchema": {
    "columns": [
      {
        "name": "Date",
        "required": true,
        "datatype": { "base": "integer" }
      },
      {
        "name": "Time",
        "required": true,
        "datatype": { "base": "string", "maxLength": 1 }
      },
      {
        "name": "Product",
        "required": true
      }
    ]
  }
}

When I run the following, I get the following error:

path = Rails.root.join('lib', 'csv', 'degiro.csv') schema = Csvlint::Schema.load_from_uri(path) validator = Csvlint::Validator.new(file_path(connect), nil, schema)

NoMethodError: undefined method []' for nil:NilClass from /Users/robert.mcginn/.rvm/gems/ruby-2.7.1/gems/csvlint-0.4.0/lib/csvlint/validate.rb:289:inset_dialect'

it appears that a dialect is required for the error not to be thrown.

If i supply an empty hash rather than nil i get a valid "Csvlint::Validator" object returned but without any errors found @errors=[]

# validator = Csvlint::Validator.new(file_path(connect), {} , schema)

Initially, I wanted to test that the schema validator was working appropriately, I am expecting at least the "Time" column which is accepting a string of maxLength 1 to return an error. But, that's not working.

Any help would be appreciated.

Expected Behaviour

What should happen?

I'm expecting validator.errors.any? to be true (errors to be found)

Desired Behaviour (for improvement suggestions only)

if relevant include images or hyperlinks to other resources that clarify the enhancement you're seeking

Current Behaviour (for problems)

validator.errors.any? = false

Steps to Reproduce (for problems)

Provide a link to a live example, or an unambiguous set of steps to reproduce this bug. Include code to reproduce, if relevant

As above.

Your Environment

Include as many relevant details about the environment you experienced the bug in - this will help us resolve the bug more expediently

local development environment

Rails 7.0.1 ruby "2.7.1" gem "csvlint" chrome = Version 101.0.4951.64