Outputting Cython code

[GoogleCodeExporter]

Shedskin can't output code that depends on a library without the C code, but it 
can run type analysis on the same library if a Python prototype exists.

Many of the built-in libraries are pure-python and would require minimal, if 
any, changes to be analyzed with shedskin (not turned into C). The few that are 
wrappers around .so files would simply be re-written along the lines of 
lib/builtin.py (headers and returns).

My proposal to fix the limitations to use shedskin's great static type analysis 
abilities for a greater range of programs is to add a new mode of operation 
that uses Cython's C generation abilities and instead adds cdefs to a .pyx 
version of the input file(s). This operation would, instead of erroring out on 
unknown libraries, either use the standard library's definitions if it can 
understand them or simply ignore them and treat any variables going in or out 
as completely unknown.

Then, instead of outputting warnings about unknown types, it would just not add 
cdef lines to them in the output file. Many of shedskin's features map to 
Cython's syntax, for instance:

__ss_int i ===> cdef int i 
(http://docs.cython.org/src/userguide/language_basics.html)
FAST_FOR(...) ===> for i from 0 <= i < n by s 
(http://docs.cython.org/src/userguide/language_basics.html#integer-for-loops)

Basically, this mode would replace shedskin's C generation with Cython's and 
instead output a .pyx file that is similar to the .ss.py annotation file but 
with more than just the types.

Original issue reported on code.google.com by fah...@recreclabs.com on 19 Apr 2011 at 11:43

• Merged into: #122

[GoogleCodeExporter]

thanks for the suggestion! I've considered outputting cython declarations 
before, though not because of library support directly, but because cython 
compiled code integrates better with other python implementations. for example, 
it would completely take away the need to convert builtin objects to/from 
shedskin. it would also remove quite a bit of code from shedskin (the whole of 
lib?!). I guess in general, it seems good to have a separation between type 
inferencers/profilers and the actual compilation system.

I don't think fully automatic integration with arbitrary libraries will work 
very well, though. we don't know in advance which parts will be compatible or 
analyzed easily, and scalability will suffer from having to analyze huge swaths 
of code. a more realistic approach might be to only require a manual type model 
(as in lib/*.py). that could still be a big improvement, since a type model is 
usually very easy to make, compared to implementing things.

I'm slightly worried about how performance would suffer versus C++, since in 
the C++ version everything is exactly known to the C++ compiler, whereas in 
cython, I don't think it can understand everything as well with just a few 
manual declarations.

I probably won't work on this, in any case. I like the idea though, and it 
might be a very useful and interesting project for some other developer. I'd be 
happy to assist of course.  

Original comment by mark.duf...@gmail.com on 23 Apr 2011 at 9:10

[GoogleCodeExporter]

Original comment by mark.duf...@gmail.com on 23 Apr 2011 at 9:11

• Changed title: Outputting Cython code

[GoogleCodeExporter]

Original comment by mark.duf...@gmail.com on 25 Jun 2011 at 8:13

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Original comment by mark.duf...@gmail.com on 16 Jul 2011 at 9:54

• Changed state: Duplicate