Argparse - Custom type not working on addscript

[waltonereed]

I created a custom type for parser.add_argument, and this works fine when I run the my script directly through the console. However, when I try to upload the script through the command line manage.py addscript, I get a bunch of error messages, like below:

Converting D:\Users\sc82170\.ipython\datetest2.py
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
ement\__init__.py", line 338, in execute_from_comm
    utility.execute()
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
ement\__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(s
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
ement\base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
ement\base.py", line 441, in execute
    output = self.handle(*args, **options)
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
commands\addscript.py", line 73, in handle
    res = add_wooey_script(**add_kwargs)
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
ls.py", line 245, in add_wooey_script
    parser = Parser(script_name=filename, script_p
ile))
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
, line 19, in __init__
    parser_obj = [pc(script_path, script_source) f
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
, line 19, in <listcomp>
    parser_obj = [pc(script_path, script_source) f
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
se.py", line 43, in __init__
    self.process_parser()
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
gparse_.py", line 262, in process_parser
    node = ArgParseNode(action=action)
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
gparse_.py", line 140, in __init__
    self.node_attrs = dict([(i, field_type[i]) for
  File "D:\Users\sc82170\Anaconda3\envs\py35\lib\s
gparse_.py", line 140, in <listcomp>
    self.node_attrs = dict([(i, field_type[i]) for
TypeError: 'NoneType' object is not subscriptable

Here's my script--aacommis a custom package I wrote that basically accepts a YYYYMMDD input and reads a .csv file through Pandas. The remaining part of the script exports 10 records to a .csv file.

import argparse
import re
import aacomm

def valid_date_type(arg_date):
    """custom argparse *date* type for user dates values given from the command line"""
    try:
        return re.search("([0-9]{8})", arg_date)
    except ValueError:
        msg = "Given Date ({0}) not valid! Expected format, YYYYMMDD!".format(arg_date_str)
        raise argparse.ArgumentTypeError(msg)

parser = argparse.ArgumentParser(description='Export to .csv 10 records from AA_COMM file.')
parser.add_argument('start_date',
                    type=valid_date_type,
                    help='Use date in format YYYYMMDD (e.g. 20180131)')

def main():
    args = parser.parse_args()
    start_date_object = args.start_date

    aa = aacomm.import_aa('1', start_date_object)

    aa = aa.head(10)

    aa.to_csv(r'D:\Users\sc82170\output.csv', index=False)

if __name__ == '__main__':
    main()

When I change the type=int, the script uploads fine to Wooey. Is there something I'm missing?

[Chris7]

This is something I haven't considered, but is a bug. Wooey would need to represent the types as a widget element, but what element it should be is ambiguous if it is a function. My instinct is to declare it as a charfield and let the user manually update the widget to their specs. I'd appreciate any other opinions on this.

[jvshah74]

I'm facing the same issue (I have a custom type - to allow for both hex and decimal values as input). Is there any plan to add this support in near future?

[Chris7]

This is actually a problem with the parser in clinto. It's pretty straight forward to just handle this like a CharField, so I did that. You will need to install from master to use that version of clinto -- I'd love to know if it works as intended for you.

[jvshah74]

Thanks Chris. I did not really pull the master - but manually merged in your changes, and it does seem to work. Just to understand this better (I'm fairly new to Python itself) - this change of handling this as "CharField" will automatically call the respective function, or would the user still need to process the arguments in the script? I think it happens automatically based on my trial - but wanted to make sure.

Secondly, there is a related issue: I had another script which uses type = str.lower() in order for the command line to accept both upper case and lower case representations of the string input. However, importing that script throws error: Exception Type: | TypeError 'NoneType' object is not subscriptable

[Chris7]

That is correct -- CharField means it will pass a string to your function, that then processes it.

For the second, the type is not a function. If you set it like: type=lambda x: x.lower(), it would work.

[jvshah74]

Sorry - I think I made a typo above. The actual usage was: type = str.lower e.g. parser.add_argument('--magic', dest='magic', default=hex(0xcd), type = str.lower, choices = [hex(0xab), hex(0xcd)]) This works in the script (will allow any of the two values regardless of case - e.g. 0xAB, 0xaB, 0xAb, 0xab when using the command line) - but causes problem with Wooey import.

The alternate you suggested does work though.

[Chris7]

This is interesting. That is because str.lower is a method_descriptor and not actually a FunctionType. Calling str.lower('ABC'), you are passing 'ABC' as self to the lower method of the str object. It works, but it definitely isn't standard :)

[Chris7]

A follow up PR fixed the use of str.lower and any other generally callable action type, https://github.com/wooey/clinto/pull/33