The validator currently throws errors when it encounters non-UTF-8 encoded files. This behavior interrupts the validation process and prevents the user from receiving a complete report of all issues with their dataset. Additionally, the validator is checking the encoding of all files, including those that are not directly relevant to the psych-DS schema.
Current Behavior
The validator throws an error and stops processing when it encounters a non-UTF-8 encoded file.
All files in the dataset are being checked for UTF-8 encoding, regardless of their relevance to the psych-DS schema.
Desired Behavior
When encountering a non-UTF-8 encoded file, the validator should:
Handle the error gracefully without interrupting the overall validation process.
Add an issue to the validation output indicating that the file is not UTF-8 encoded.
The validator should only check the encoding of relevant files:
Canonical datafiles (e.g., CSV files specified in the dataset's metadata).
JSON metadata files.
Other file types specified as required by the psych-DS schema.
Description
The validator currently throws errors when it encounters non-UTF-8 encoded files. This behavior interrupts the validation process and prevents the user from receiving a complete report of all issues with their dataset. Additionally, the validator is checking the encoding of all files, including those that are not directly relevant to the psych-DS schema.
Current Behavior
Desired Behavior