NREL / disco

DISCO
BSD 3-Clause "New" or "Revised" License
7 stars 5 forks source link

Improve Pydantic validation of upgrade cost database #147

Closed daniel-thom closed 2 years ago

daniel-thom commented 2 years ago

If a column in the upgrade cost database is misspelled or has the wrong case, there are so many errors that it is difficult to understand the problem. In this example a couple of columns were incorrectly capitalized and it printed this message for every line (and there were 279 of them):

File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 598 validation errors for UpgradeCostAnalysisSimulationModel
calculate_costs -> lines -> 0 -> description
  field required (type=value_error.missing)
calculate_costs -> lines -> 0 -> Description
  extra fields not permitted (type=value_error.extra)

It would be better if we could stop processing on the first error (I don't that think Pydantic allows this) or check the field names explicitly.