Cython emits "None" into C code

[robertwb]

Reported by bdarnell on 9 Aug 2015 16:11 UTC Cython produces C code that fails to compile (because it contains the identifier None) for the following snippet:

def f(*args, **kwargs):
    for k, v in dict(*args, **kwargs).items():
        pass

Output of cythonize -i foo.py:

Compiling /private/tmp/cythontest/foo.py because it changed.
[Cythonizing /private/tmp/cythontest/foo.py
running build_ext
building 'foo' extension
creating /private/tmp/cythontest/tmpf3710600/private
creating /private/tmp/cythontest/tmpf3710600/private/tmp
creating /private/tmp/cythontest/tmpf3710600/private/tmp/cythontest
/usr/bin/clang -Wno-unused-result -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -pipe -Os -I/opt/local/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c /private/tmp/cythontest/foo.c -o /private/tmp/cythontest/tmpf3710600/private/tmp/cythontest/foo.o
/private/tmp/cythontest/foo.c:732:41: error: use of undeclared identifier 'None'
  __pyx_t_2 = __Pyx_PyObject_GetAttrStr(None, __pyx_n_s_items); if (unli...
                                        ^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1

A few lines of context from the generated code:

  __Pyx_RefNannySetupContext("f", 0);

  /* "foo.py":2
 * def f(*args, **kwargs):
 *     for k, v in dict(*args, **kwargs).items():             # <<<<<<<<<<<<<<
 *         pass
 */
  __pyx_t_2 = __Pyx_PyObject_GetAttrStr(None, __pyx_n_s_items); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0](1/1]); __pyx_lineno = 2; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF(__pyx_t_2);
  __pyx_t_3 = NULL;
  if (CYTHON_COMPILING_IN_CPYTHON && likely(PyMethod_Check(__pyx_t_2))) {

This occurs in cython 0.22 and 0.23, but not 0.21. Tested on OSX with cpython 3.4 and 2.7.

Migrated-From: http://trac.cython.org/ticket/856

[robertwb]

Comment by scoder on 14 Aug 2015 19:35 UTC A work-around (and actually an improvement) is to just use iteritems() instead.

[robertwb]

Comment by scoder on 14 Aug 2015 19:47 UTC This was hiding another bug that prevented iteration over dict(..., **kwargs).items() from being optimised.

https://github.com/cython/cython/commit/591199d1307b79112fee38b1a746484529ebb17d

[robertwb]

Comment by scoder on 14 Aug 2015 19:55 UTC The test can be reduced to dict(*args, **kwargs).items() generating broken C code.

[robertwb]

Comment by scoder on 15 Aug 2015 20:02 UTC Fixed here:

https://github.com/cython/cython/commit/00bd02fdf8a6d5d1d5ba80b660f62d68bce42680

[robertwb]

Modified by scoder on 15 Aug 2015 20:26 UTC

[robertwb]

Comment by bdarnell on 13 Sep 2015 15:59 UTC The bug is still present in cython 0.23.2.

The reduced version without the loop is now fine:

 def good(*args, **kwargs):
     dict(*args, **kwargs).items()

But the original loop still fails in the same way:

 def bad(*args, **kwargs):
     for k, v in dict(*args, **kwargs).items():
         pass

/tmp/cythontest.c:732:41: error: use of undeclared identifier 'None'
  __pyx_t_2 = __Pyx_PyObject_GetAttrStr(None, __pyx_n_s_items); if ...

[robertwb]

Comment by scoder on 15 Sep 2015 18:22 UTC Ok, but this should fix it: https://github.com/cython/cython/commit/8710e415b8f0a9be0ee7cd4bf48e453507ec9f6f

Thanks for insisting.