Schwab conversion no longer working

[rvernica]

I started working on a Schwab mapping but I'm not getting a lot of progress. I also managed to crash csv2ofx in the middle of the file and I'm not sure what caused it. Here is how far I got.

The CSV file that you get from a Schwab Brokerage account looks like this:

"Transactions  for account Main XXXX-1234 as of 12/19/2020 22:47:12 ET"
"Date","Action","Symbol","Description","Quantity","Price","Fees & Comm","Amount",
"12/18/2020","Buy","FOO","FOO ETF","12","$12.34","","-$12.34",
"12/16/2020 as of 12/15/2020","Bank Interest","","BANK INT 111620-121520 SCHWAB BANK","","","","$0.01",
"12/15/2020","Cash Dividend","FOO","FOO ETF","","","","$12.34",
Transactions Total,"","","","","","",$12.34

Right of the bat I need to:

1. Get rid of the first and last line. I did this:

   > tail -n +2 Main_Transactions_20201219-224712.CSV | head -n -1 > Schwab.CSV

2. Fix the Date field for the Bank Interest transactions. I did this:

   > sed --in-place 's#^"\(../../....\) as of ../../...."#"\1"#g' Schwab.CSV

Regarding the mapping itself, this is what I have so far:

mapping = {
    'account': 'Brokerage',
    'amount': itemgetter('Amount'),
    'bank': 'Schwab',
    'date': itemgetter('Date'),
    'desc': itemgetter('Description'),
    # Action
    # Symbol
    # Quantity
    # Price
    # Fees & Comm
}

The comment lines still need some sort of mapping.

I ran this and it went fine for more than one year of transactions but it crashed suddenly. Here is the error:

                    <STMTTRN>
                        <TRNTYPE>DEBIT</TRNTYPE>
                        <DTPOSTED>20191002000000</DTPOSTED>
                        <TRNAMT>-12.34</TRNAMT>
    after_comma: None
after_decimal: None
Traceback (most recent call last):
  File "/.local/lib/python3.8/site-packages/meza/fntools.py", line 629, in byte
    bytes_ = content.encode(ENCODING)
AttributeError: 'itertools.islice' object has no attribute 'encode'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.local/lib/python3.8/site-packages/csv2ofx/main.py", line 183, in run
    res = write(dest, IterStringIO(content), **kwargs)
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 1453, in write
    return sum(read_any(filepath, writer, mode, content, **kwargs))
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 522, in read_any
    for line in _read_any(filepath, reader, args, **kwargs):
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 457, in _read_any
    for num, line in enumerate(reader(f, *args, **kwargs)):
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 1431, in writer
    for chunk in ft.chunk(content, chunksize):
  File "/.local/lib/python3.8/site-packages/meza/fntools.py", line 665, in <genexpr>
    generator = (content.read(chunksize) for _ in it.count())
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 145, in read
    return self._read(self.iter, num, False)
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 129, in _read
    byte = ft.byte(content)
  File "/.local/lib/python3.8/site-packages/meza/fntools.py", line 634, in byte
    bytes_ = b''.join(map(encode, content))
  File "/.local/lib/python3.8/site-packages/meza/io.py", line 66, in <genexpr>
    encode = lambda iterable: (s.encode(ENCODING) for s in iterable)
  File "/.local/lib/python3.8/site-packages/csv2ofx/ofx.py", line 451, in gen_body
    trxn_data = self.transaction_data(datum['trxn'])
  File "/.local/lib/python3.8/site-packages/csv2ofx/ofx.py", line 146, in transaction_data
    data = super(OFX, self).transaction_data(trxn)
  File "/.local/lib/python3.8/site-packages/csv2ofx/__init__.py", line 221, in transaction_data
    amount = self.convert_amount(trxn)
  File "/.local/lib/python3.8/site-packages/csv2ofx/__init__.py", line 181, in convert_amount
    return utils.convert_amount(self.get('amount', trxn))
  File "/.local/lib/python3.8/site-packages/csv2ofx/utils.py", line 102, in convert_amount
    return to_decimal(content, **get_separators(content))
  File "/.local/lib/python3.8/site-packages/meza/fntools.py", line 818, in get_separators
    raise ValueError('Invalid number format for `{}`.'.format(content))
ValueError: Invalid number format for ``.

And here is the current and after line in the CSV:

"10/02/2019","Buy","FOO","FOO ETF","12","$12.3456","","-$12.34",
"10/02/2019","Buy","BAR","BAR ETF","23","$23.4567","","-$23.45",

I'm not sure what is the issue here.

> python3 --version
Python 3.8.6
> csv2ofx --version
v0.27.0

[reubano]

About to update this program for new Python versions. Please update if the issue persists.

[reubano]

Also, what will help is if you can identify what differs between the files that work and the ones that don't.

[reubano]

If you are still having issues, add an example csv file to the issue.

[reubano]

A minimum working example below

schwab.csv

"Date","Action","Symbol","Description","Quantity","Price","Fees & Comm","Amount",
"12/18/2020","Buy","FOO","FOO ETF","12","$12.34","","-$12.34",
"12/16/2020","Bank Interest","","BANK INT 111620-121520 SCHWAB BANK","","","","$0.01",
"12/15/2020","Cash Dividend","FOO","FOO ETF","","","","$12.34",

schwab.py

from operator import itemgetter

mapping = {
    "has_header": True,
    "bank": "Schwab",
    "account": "Brokerage",
    "date": itemgetter("Date"),
    "amount": itemgetter("Amount"),
    "desc": itemgetter("Description"),
}

csv2ofx -x schwab.py schwab.csv

DATA:OFXSGML
ENCODING:UTF-8
<OFX>
    <SIGNONMSGSRSV1>
        <SONRS>
...

[reubano]

CR #32