DATAUP-549: Update ADR with new template, best effort parsing, etc

[MrCreosote]

• Update and explain the template schema
• Describe the best effort parsing strategy
• Add an incorrect_column_count error
• Dial back on required logging for parse errors. Most of the stack traces will not be useful.

For xSV files, since pandas drops data if a row has more columns than the first row, manually parse the first line and then pass the remainder of the file handle to pandas.

Tested with excel files - in that case pandas just duplicates the first cell X times in the header structure where X is the maximum column count in the data.

[sonarcloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information