crashes on __DATE__, packed, aligned

[blueforesticarus]

~/test $ clang2py --version
clang2py version 2.2.3

~/test $ cat test.h 
#define DATE __DATE__

~/test $ clang2py -k m test.h 
Traceback (most recent call last):
  File "/usr/bin/clang2py", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.8/site-packages/ctypeslib/clang2py.py", line 291, in main
    generate_code(files, stream,
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/codegenerator.py", line 926, in generate_code
    parser.parse(srcfile)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 132, in parse
    self.startElement(node)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 174, in startElement
    stop_recurse = self.parse_cursor(node)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 304, in parse_cursor
    return self.cursorkind_handler.parse_cursor(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 38, in parse_cursor
    return mth(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 1048, in MACRO_DEFINITION
    tokens = self._literal_handling(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 532, in _literal_handling
    value = self.get_registered(value).body
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/handler.py", line 54, in get_registered
    return self.parser.get_registered(name)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 218, in get_registered
    return self.all[name]
KeyError: '__DATE__'

~/test $ cat test2.h 
#define PACK __attribute__((aligned(2)))

~/test $ clang2py -k m test2.h 
Traceback (most recent call last):
  File "/usr/bin/clang2py", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.8/site-packages/ctypeslib/clang2py.py", line 291, in main
    generate_code(files, stream,
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/codegenerator.py", line 926, in generate_code
    parser.parse(srcfile)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 132, in parse
    self.startElement(node)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 174, in startElement
    stop_recurse = self.parse_cursor(node)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 304, in parse_cursor
    return self.cursorkind_handler.parse_cursor(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 38, in parse_cursor
    return mth(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 1048, in MACRO_DEFINITION
    tokens = self._literal_handling(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 532, in _literal_handling
    value = self.get_registered(value).body
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/handler.py", line 54, in get_registered
    return self.parser.get_registered(name)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 218, in get_registered
    return self.all[name]
KeyError: 'aligned'

~/test $ cat test3.h 
#define PACK __attribute__((packed))

~/test $ clang2py -k m test3.h 
Traceback (most recent call last):
  File "/usr/bin/clang2py", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.8/site-packages/ctypeslib/clang2py.py", line 291, in main
    generate_code(files, stream,
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/codegenerator.py", line 926, in generate_code
    parser.parse(srcfile)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 132, in parse
    self.startElement(node)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 174, in startElement
    stop_recurse = self.parse_cursor(node)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 304, in parse_cursor
    return self.cursorkind_handler.parse_cursor(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 38, in parse_cursor
    return mth(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 1048, in MACRO_DEFINITION
    tokens = self._literal_handling(cursor)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/util.py", line 119, in fn
    return func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/cursorhandler.py", line 532, in _literal_handling
    value = self.get_registered(value).body
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/handler.py", line 54, in get_registered
    return self.parser.get_registered(name)
  File "/usr/lib/python3.8/site-packages/ctypeslib/codegen/clangparser.py", line 218, in get_registered
    return self.all[name]
KeyError: 'packed'

[blueforesticarus]

replicated with clang 11

$ sudo python -m pip show ctypeslib2
Name: ctypeslib2
Version: 2.2.3
Summary: ctypeslib2 - FFI toolkit, relies on clang
Home-page: https://github.com/trolldbois/ctypeslib
Author: Loic Jaquemet
Author-email: loic.jaquemet+python@gmail.com
License: License :: OSI Approved :: MIT License
Location: /usr/lib/python3.9/site-packages
Requires: 
Required-by: 

$ sudo python -m pip show clang
Name: clang
Version: 11.0
Summary: libclang python bindings
Home-page: http://clang.llvm.org/
Author: LLVM team - pypi upload by Loic Jaquemet
Author-email: UNKNOWN
License: License :: OSI Approved :: University of Illinois/NCSA Open Source License
Location: /usr/lib/python3.9/site-packages
Requires: 
Required-by: 

$ clang2py --version
clang2py version 2.2.3

$ clang --version
clang version 11.0.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

[blueforesticarus]

workaround, patch codegen/cursorhandler.py

532c532,533
<                     value = self.get_registered(value).body
---
>                     if self.is_registered(value):
>                         value = self.get_registered(value).body

[blueforesticarus]

actually, that produces bad output

~ $ cat test.h 
#define PACK __attribute__((packed,aligned(2)))

~ $ clang2py -k m test.h
# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes

PACK = __attribute__((packed,aligned(2))) # macro
__all__ = \
    ['PACK']

[blueforesticarus]

another workaround:

--- codegen/codegenerator.py    2021-02-24 11:10:03.927921957 -0500
+++ codegen/codegenerator.py    2021-02-24 12:43:51.284608507 -0500
@@ -194,7 +194,18 @@
             print("# %s:%s" % (macro.location), file=self.stream)
         if self.generate_comments:
             self.print_comment(macro)
-        print("%s = %s # macro" % (macro.name, macro.body), file=self.stream)
+
+        txt = macro.body
+        try:
+            eval("%s" % txt)
+        except:
+            txt = "'%s'" % txt.replace("'", "\\'")
+            try:
+                eval(txt)
+            except:
+                txt = "None # " + macro.body
+
+        print("%s = %s # macro" % (macro.name, txt), file=self.stream)
         self.macros += 1
         self.names.add(macro.name)
         return

[trolldbois]

I'll add test cases. Can you try the HEAD master here ? I fixed a bunch of macro related issues this week after the 2.2.3 release

[blueforesticarus]

It appears to work. Cool