ohare93 / brain-brew

Automated Anki flashcard creation and extraction to/from Csv
The Unlicense
89 stars 5 forks source link

Checking the number of rows in all columns is same (or less-than-or-equals?) as number of headers (per file) (error-reporting) #52

Open aplaice opened 7 months ago

aplaice commented 7 months ago

The user should have CSV files which have all columns labeled (i.e. no row has more columns than there are headers in the given file). If this is the case, then the error described here won't occur. (i.e. I'm not describing a bug in BrainBrew — if the inputs are correct, then everything is fine — just sub-optimal error-reporting.)

Unfortunately, while this will usually be the case, it won't always be, as the user might add some text to the right of the rightmost column (as temporary annotation) or forget to add the relevant column header. If it isn't the case, then a rather confusing message is displayed (e.g. see here):

``` $ pipenv run build INFO:root:Builder file recipes/source_to_anki.yaml is ✔ good INFO:root:Attempting to generate Guids INFO:root:Generate guids complete Traceback (most recent call last): File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/bin/brain_brew", line 8, in sys.exit(main()) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/main.py", line 19, in main command.execute() File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/run_recipe.py", line 15, in execute recipe = TopLevelBuilder.parse_and_read(self.recipe_file_name, self.verify_only) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/top_level_builder.py", line 60, in parse_and_read return cls.from_list(recipe_data) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/recipe_builder.py", line 20, in from_list tasks = cls.read_tasks(data) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/recipe_builder.py", line 68, in read_tasks task_or_tasks = [matching_task.from_repr(task_arguments)] File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/parts_builder.py", line 40, in from_repr return cls.from_list(data) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/recipe_builder.py", line 20, in from_list tasks = cls.read_tasks(data) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/commands/run_recipe/recipe_builder.py", line 68, in read_tasks task_or_tasks = [matching_task.from_repr(task_arguments)] File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/build_tasks/csvs/notes_from_csvs.py", line 56, in from_repr file_mappings=rep.get_file_mappings(), File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/build_tasks/csvs/shared_base_csvs.py", line 22, in get_file_mappings return list(map(FileMapping.from_repr, self.file_mappings)) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/transformers/file_mapping.py", line 76, in from_repr derivatives=list(map(cls.from_repr, rep.derivatives)) if rep.derivatives is not None else [], File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/transformers/file_mapping.py", line 68, in from_repr csv.read_file() File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/representation/generic/csv_file.py", line 47, in read_file self._data.append({key.lower(): row[key] for key in row}) File "/home/munksgaard/.local/share/virtualenvs/ultimate-geography-wQmoXBY1/lib/python3.10/site-packages/brain_brew/representation/generic/csv_file.py", line 47, in self._data.append({key.lower(): row[key] for key in row}) AttributeError: 'NoneType' object has no attribute 'lower' ```

It's possible to track down the source of the error with a small amount of debugging, but it's not very user-friendly.

If BrainBrew checked that there are no rows with too many columns (i.e. more columns than there are headers in the given file), then I believe that this class of issues could be caught with nicer warning messages.

This is obviously not very urgent/important and I haven't really thought about whether there are better solutions/whether my proposed solution would not slow BB down etc. — I'm mainly writing this down here for reference! :)