TypeError: vars() argument must have __dict__ attribute

[kkrizka]

I'm getting hit by the following error again when submitting jobs to the grid using prun.

Traceback (most recent call last):
  File "/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/pyAmi/pyAMI-5.0.7.1/lib/argparse/__init__.py", line 1200, in __eq__
    return vars(self) == vars(other)
TypeError: vars() argument must have __dict__ attribute

This has been seen in the past, but without any resolution: https://github.com/UCATLAS/xAODAnaHelpers/issues/837#issuecomment-286581638 https://groups.cern.ch/group/hn-atlas-PATHelp/Lists/Archive/DispForm.aspx?ID=16349

I am on the xAHrc branch, so maybe PyROOT does not like some sequence of commands introduced there. I will look into this further, but I wonder if someone here has any other ideas.

-- Karol Krizka

[kratsg]

Ok, so I consider myself an argparse expert (having played with the internals a lot) and can tell you that this situation occurs when you're trying to check if two namespaces are equal to each other. The only time you ever get a Namespace object out of argparse is when you parse_args / parse_known_args or make your own custom one.

In /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/pyAmi/pyAMI-5.0.7.1/lib/pyAMI/app.py:148, we have:

    def run(self):

        if sys.version_info[0] == 3:
            args = dict(self._parser.parse_args([                               (sys.argv[i]) for i in range(1, len(sys.argv))])._get_kwargs())
        else:
            args = dict(self._parser.parse_args([pyAMI.utils.safe_decoded_string(sys.argv[i]) for i in range(1, len(sys.argv))])._get_kwargs())

And somewhere along the line, args is None because of this:

>>> vars(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: vars() argument must have __dict__ attribute

see the report here: https://bugs.python.org/issue21481

[kratsg]

(where is Namespace equality happening though?!)

[kratsg]

And it looks like it might be because pyAMI is catching an error and ignoring it -- because that might be the only way parse_args returns None:

    # =====================================
    # Command line argument parsing methods
    # =====================================
    def parse_args(self, args=None, namespace=None):
        args, argv = self.parse_known_args(args, namespace)
        if argv:
            msg = _('unrecognized arguments: %s')
            self.error(msg % ' '.join(argv))
        return args 

    def parse_known_args(self, args=None, namespace=None):
        # args default to the system args
        if args is None:
            args = _sys.argv[1:]

        # default Namespace built from parser defaults
        if namespace is None:
            namespace = Namespace()

        # add any action defaults that aren't present
        for action in self._actions:
            if action.dest is not SUPPRESS:
                if not hasattr(namespace, action.dest):
                    if action.default is not SUPPRESS:
                        setattr(namespace, action.dest, action.default)

        # add any parser defaults that aren't present
        for dest in self._defaults:
            if not hasattr(namespace, dest):
                setattr(namespace, dest, self._defaults[dest])

        # parse the arguments and exit if there are any errors
        try: 
            namespace, args = self._parse_known_args(args, namespace)
            if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):
                args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))
                delattr(namespace, _UNRECOGNIZED_ARGS_ATTR)
            return namespace, args 
        except ArgumentError:
            err = _sys.exc_info()[1]
            self.error(str(err))

[kkrizka]

This happens when --optBatchWait is specified with the prun driver. For some reason the PrunDriver does not like running with driver.submit().