mincong-h commented 2 years ago

When an error occurs, the exception contains more information for debugging, where we have the path of the CSV file, the expected columns and their types, and all the pandas-specific fields. Here is an example:

Traceback (most recent call last):
  File "/usr/local/bin/finance-toolkit", line 33, in <module>
    sys.exit(load_entry_point('finance-toolkit==0.1.0', 'console_scripts', 'finance-toolkit')())
  File "/usr/local/lib/python3.7/site-packages/finance_toolkit-0.1.0-py3.7.egg/finance_toolkit/__main__.py", line 71, in main
  File "/usr/local/lib/python3.7/site-packages/finance_toolkit-0.1.0-py3.7.egg/finance_toolkit/tx.py", line 237, in move
  File "/usr/local/lib/python3.7/site-packages/finance_toolkit-0.1.0-py3.7.egg/finance_toolkit/pipeline.py", line 30, in run
  File "/usr/local/lib/python3.7/site-packages/finance_toolkit-0.1.0-py3.7.egg/finance_toolkit/boursorama.py", line 115, in read_new_transactions
  File "/usr/local/lib/python3.7/site-packages/finance_toolkit-0.1.0-py3.7.egg/finance_toolkit/boursorama.py", line 64, in read_raw
finance_toolkit.pipeline.PipelineDataError: Failed to read new Boursorama data. Details:
  path=/data/source/export-operations-11-06-2022_09-52-55.csv
  headers=ï»¿dateOp;dateVal;label;category;categoryParent;amount;comment;accountNum;accountLabel;accountbalance
  pandas_kwargs={'decimal': ',', 'delimiter': ';', 'dtype': {'accountNum': 'str'}, 'encoding': 'ISO-8859-1', 'parse_dates': ['dateOp', 'dateVal'], 'skipinitialspace': True, 'thousands': ' '}
  pandas_error=Missing column provided to 'parse_dates': 'dateOp'

Thanks for this custom error, we know that we cannot trust pandas' error as we already provided "dateOp" to the "parse_dates".

codecov-commenter commented 2 years ago

Codecov Report

Merging #74 (5422bb2) into master (cd8de4a) will increase coverage by 0.14%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #74      +/-   ##
==========================================
+ Coverage   94.08%   94.23%   +0.14%     
==========================================
  Files          11       11              
  Lines         592      607      +15     
  Branches       97       97              
==========================================
+ Hits          557      572      +15     
  Misses         20       20              
  Partials       15       15

Impacted Files	Coverage Δ
finance_toolkit/boursorama.py	`94.20% <100.00%> (+0.55%)`	:arrow_up:
finance_toolkit/pipeline.py	`91.91% <100.00%> (+0.80%)`	:arrow_up:

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

mincong-h commented 2 years ago

mincong-h / finance-toolkit

Improve error handling for Boursorama #74

Codecov Report