memacs_csv: help parameter is not working

nbehrnd commented 2 years ago

Contrasting to the general README.org, module memacs_csv.py doesn't reply to --help (because it is not anticipated by the module), nor to -h when using a pristine virtual environment venv of Python 3.10 in Linux Debian 12/bookworm (branch testing).

The hypothesis the module's action were limited because it runs from a virtual environment was not supported because memacs_photos.py is working well. From my understanding of memacs_csv.py's documentation, it suffices if the first column in the .csv contains a timestamp; yet experience with a file organized as

2022-07-10,2
2022-07-11,4

and

python3 memacs_csv.py -f ~/Desktop/Book1.csv --delimiter "," --timestamp-field  "date" --output-format "{text}"

shows I err on this:

(probe_memacs) norwid@carnot2:~/Desktop/probe_memacs/lib/python3.10/site-packages/bin$ python3 memacs_csv.py -f ~/Desktop/Book1.csv --delimiter "," --timestamp-field  "date" --output-format "{text}"
## -*- coding: utf-8 mode: org -*-
 ## This file was generated by memacs_csv.py. Any modification will be overwritten upon next invocation.
 ## To add this file to your list of org-agenda files, open the stub file (file.org) not this file (file.org_archive) within emacs and do following: M-x org-agenda-file-to-front
 * Memacs for csv files          :Memacs:csv:
Traceback (most recent call last):
     File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/memacs/lib/memacs.py", line 188, in handle_main
       self._main()
     File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/memacs/csv.py", line 139, in _main
       self._handle_row(row)
     File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/memacs/csv.py", line 91, in _handle_row
       timestamp = datetime.datetime.fromtimestamp(int(row[self._args.timestamp_field]))
   KeyError: 'date'
 Traceback (most recent call last):
  File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/bin/memacs_csv.py", line 39, in <module>
    main()
  File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/bin/memacs_csv.py", line 35, in main
    memacs.handle_main()
  File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/memacs/lib/memacs.py", line 188, in handle_main
    self._main()
  File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/memacs/csv.py", line 139, in _main
    self._handle_row(row)
  File "/home/norwid/Desktop/probe_memacs/lib/python3.10/site-packages/memacs/csv.py", line 91, in _handle_row

The addition of a time stamp (date2name) to the file followed by starting the module does not work work either. Based on the experience with memacs_photos.py I thought memacs_csv.py would equally yield a .org file to use like the other agenda files.

excerpt.org.txt Book1.csv.txt

novoid commented 2 years ago

Hi,

Unfortunately, yet another module I'm not using myself at the moment.

From looking at the memacs/tests/data/example1.csv file and the parser options of csv.py, the default format is "DD.MM.YYYY" (German default). Not a good default choice, IMO. However, you can modify the expected format using a CLI parameter such as --timestamp-format "%Y.%m.%d".

Maybe this is the missing piece to make it work at your side. Please report back.

nbehrnd commented 2 years ago

After normal installation with pip in Linux Debian 12/bookwork (branch testing) (which is going to be a separate issue report), the module is accessible, though possibly not fully. In my case, the installation of the modules uses ~/.local/bin (which is added to the PATH variable).

My observations are conflicting with each other. For one, the direct call yields the reminder I anticipate for a Python script using argparse, i.e. to offer a brief list of the options available. Thus, no surprise with the synopsis

norwid@carnot2:~/.local/bin$ ./memacs_csv
usage: memacs_csv [-h] [--version] [-v] [-s] [-o FILE] [-a] [-t TAG]
                  [--autotagfile FILE] [--number-entries NUMBER_ENTRIES]
                  [--columns-header STRING] [--custom-header STRING]
                  [--add-to-time-stamps STRING] [--inactive-time-stamps] -f
                  CSVFILE [-d DELIMITER] [-e ENCODING] [-n FIELDNAMES]
                  [-p PROPERTIES] --timestamp-field TIMESTAMP_FIELD
                  [--timestamp-format TIMESTAMP_FORMAT] --output-format
                  OUTPUT_FORMAT [--skip-header]
memacs_csv: error: the following arguments are required: -f/--file, --timestamp-field, --output-format
norwid@carnot2:~/.local/bin$

Requesting the version information by --version equally works as anticipated:

norwid@carnot2:~/.local/bin$ ./memacs_csv --version
memacs_csv v0.1 from 2012-02-24
norwid@carnot2:~/.local/bin$

From the synopsis, I would anticipate a call with -h (mentioned) or --help (mentioned in the general README.org) to yield a more verbose description about the parameters. Hélas, neither one is working:

norwid@carnot2:~/.local/bin$ ./memacs_csv -h
ERROR:root:Traceback (most recent call last):
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 187, in handle_main
       self.__init()
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 87, in __init
       self._parser_parse_args()
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/csv.py", line 78, in _parser_parse_args
       Memacs._parser_parse_args(self)
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 168, in _parser_parse_args
       self._args = self._parser.parse_args(self.__argv)
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/argparser.py", line 140, in parse_args
       args = ArgumentParser.parse_args(self, args=args, namespace=namespace)
     File "/usr/lib/python3.10/argparse.py", line 1838, in parse_args
       args, argv = self.parse_known_args(args, namespace)
     File "/usr/lib/python3.10/argparse.py", line 1871, in parse_known_args
       namespace, args = self._parse_known_args(args, namespace)
     File "/usr/lib/python3.10/argparse.py", line 2080, in _parse_known_args
       start_index = consume_optional(start_index)
     File "/usr/lib/python3.10/argparse.py", line 2020, in consume_optional
       take_action(action, args, option_string)
     File "/usr/lib/python3.10/argparse.py", line 1948, in take_action
       action(self, namespace, argument_values, option_string)
     File "/usr/lib/python3.10/argparse.py", line 1111, in __call__
       parser.print_help()
     File "/usr/lib/python3.10/argparse.py", line 2568, in print_help
       self._print_message(self.format_help(), file)
     File "/usr/lib/python3.10/argparse.py", line 2552, in format_help
       return formatter.format_help()
     File "/usr/lib/python3.10/argparse.py", line 295, in format_help
       help = self._root_section.format_help()
     File "/usr/lib/python3.10/argparse.py", line 226, in format_help
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 226, in format_help
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 545, in _format_action
       help_text = self._expand_help(action)
     File "/usr/lib/python3.10/argparse.py", line 642, in _expand_help
       return self._get_help_string(action) % params
   TypeError: %d format: a real number is required, not dict
Traceback (most recent call last):
  File "/home/norwid/.local/bin/./memacs_csv", line 8, in <module>
    sys.exit(main())
  File "/home/norwid/.local/lib/python3.10/site-packages/bin/memacs_csv.py", line 35, in main
    memacs.handle_main()
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 187, in handle_main
    self.__init()
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 87, in __init
    self._parser_parse_args()
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/csv.py", line 78, in _parser_parse_args
    Memacs._parser_parse_args(self)
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 168, in _parser_parse_args
    self._args = self._parser.parse_args(self.__argv)
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/argparser.py", line 140, in parse_args
    args = ArgumentParser.parse_args(self, args=args, namespace=namespace)
  File "/usr/lib/python3.10/argparse.py", line 1838, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 1871, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 2080, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib/python3.10/argparse.py", line 2020, in consume_optional
    take_action(action, args, option_string)
  File "/usr/lib/python3.10/argparse.py", line 1948, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/lib/python3.10/argparse.py", line 1111, in __call__
    parser.print_help()
  File "/usr/lib/python3.10/argparse.py", line 2568, in print_help
    self._print_message(self.format_help(), file)
  File "/usr/lib/python3.10/argparse.py", line 2552, in format_help
    return formatter.format_help()
  File "/usr/lib/python3.10/argparse.py", line 295, in format_help
    help = self._root_section.format_help()
  File "/usr/lib/python3.10/argparse.py", line 226, in format_help
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 226, in format_help
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 545, in _format_action
    help_text = self._expand_help(action)
  File "/usr/lib/python3.10/argparse.py", line 642, in _expand_help
    return self._get_help_string(action) % params
TypeError: %d format: a real number is required, not dict
norwid@carnot2:~/.local/bin$

norwid@carnot2:~/.local/bin$ ./memacs_csv --help
ERROR:root:Traceback (most recent call last):
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 187, in handle_main
       self.__init()
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 87, in __init
       self._parser_parse_args()
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/csv.py", line 78, in _parser_parse_args
       Memacs._parser_parse_args(self)
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 168, in _parser_parse_args
       self._args = self._parser.parse_args(self.__argv)
     File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/argparser.py", line 140, in parse_args
       args = ArgumentParser.parse_args(self, args=args, namespace=namespace)
     File "/usr/lib/python3.10/argparse.py", line 1838, in parse_args
       args, argv = self.parse_known_args(args, namespace)
     File "/usr/lib/python3.10/argparse.py", line 1871, in parse_known_args
       namespace, args = self._parse_known_args(args, namespace)
     File "/usr/lib/python3.10/argparse.py", line 2080, in _parse_known_args
       start_index = consume_optional(start_index)
     File "/usr/lib/python3.10/argparse.py", line 2020, in consume_optional
       take_action(action, args, option_string)
     File "/usr/lib/python3.10/argparse.py", line 1948, in take_action
       action(self, namespace, argument_values, option_string)
     File "/usr/lib/python3.10/argparse.py", line 1111, in __call__
       parser.print_help()
     File "/usr/lib/python3.10/argparse.py", line 2568, in print_help
       self._print_message(self.format_help(), file)
     File "/usr/lib/python3.10/argparse.py", line 2552, in format_help
       return formatter.format_help()
     File "/usr/lib/python3.10/argparse.py", line 295, in format_help
       help = self._root_section.format_help()
     File "/usr/lib/python3.10/argparse.py", line 226, in format_help
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 226, in format_help
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
       item_help = join([func(*args) for func, args in self.items])
     File "/usr/lib/python3.10/argparse.py", line 545, in _format_action
       help_text = self._expand_help(action)
     File "/usr/lib/python3.10/argparse.py", line 642, in _expand_help
       return self._get_help_string(action) % params
   TypeError: %d format: a real number is required, not dict
Traceback (most recent call last):
  File "/home/norwid/.local/bin/./memacs_csv", line 8, in <module>
    sys.exit(main())
  File "/home/norwid/.local/lib/python3.10/site-packages/bin/memacs_csv.py", line 35, in main
    memacs.handle_main()
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 187, in handle_main
    self.__init()
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 87, in __init
    self._parser_parse_args()
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/csv.py", line 78, in _parser_parse_args
    Memacs._parser_parse_args(self)
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/memacs.py", line 168, in _parser_parse_args
    self._args = self._parser.parse_args(self.__argv)
  File "/home/norwid/.local/lib/python3.10/site-packages/memacs/lib/argparser.py", line 140, in parse_args
    args = ArgumentParser.parse_args(self, args=args, namespace=namespace)
  File "/usr/lib/python3.10/argparse.py", line 1838, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 1871, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 2080, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib/python3.10/argparse.py", line 2020, in consume_optional
    take_action(action, args, option_string)
  File "/usr/lib/python3.10/argparse.py", line 1948, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/lib/python3.10/argparse.py", line 1111, in __call__
    parser.print_help()
  File "/usr/lib/python3.10/argparse.py", line 2568, in print_help
    self._print_message(self.format_help(), file)
  File "/usr/lib/python3.10/argparse.py", line 2552, in format_help
    return formatter.format_help()
  File "/usr/lib/python3.10/argparse.py", line 295, in format_help
    help = self._root_section.format_help()
  File "/usr/lib/python3.10/argparse.py", line 226, in format_help
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 226, in format_help
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 226, in <listcomp>
    item_help = join([func(*args) for func, args in self.items])
  File "/usr/lib/python3.10/argparse.py", line 545, in _format_action
    help_text = self._expand_help(action)
  File "/usr/lib/python3.10/argparse.py", line 642, in _expand_help
    return self._get_help_string(action) % params
TypeError: %d format: a real number is required, not dict
norwid@carnot2:~/.local/bin$

These observations are puzzling, because extraction/reformat of the relevant data from a .csv file became successful:

norwid@carnot2:~/.local/bin$ ./memacs_csv -f ~/Desktop/koffer_memacs/Memacs/memacs/tests/data/example1_isodate.csv --fieldnames "date,text,value,currency," --timestamp-field "date" --timestamp-format "%Y.%m.%d" --output-format "{text}" --properties "currency,value"
## -*- coding: utf-8 mode: org -*-
 ## This file was generated by ./memacs_csv. Any modification will be overwritten upon next invocation.
 ## To add this file to your list of org-agenda files, open the stub file (file.org) not this file (file.org_archive) within emacs and do following: M-x org-agenda-file-to-front
 * Memacs for csv files          :Memacs:csv:
 ** <2012-03-14 Wed> Amazon
    :PROPERTIES:
   :CURRENCY:   EUR
   :VALUE:      100,00
   :ID:         052af01160fb5643d7195ebaa78153e144f31449
   :END:

 * successfully parsed 1 entries by ./memacs_csv at [2022-07-13 Wed 13:19] in ~0.002283s.

novoid commented 2 years ago

I've found the issue with the help parameter (missing escaping of the % character) and pushed the change to GitHub.

novoid / Memacs

memacs_csv: help parameter is not working #113