Move Cython-language utility code inclusion to code generation phase

[robertwb]

Currently, utility code written in Cython is injected into the main module tree right prior to code generation, so that the code generation phase cannot add new utility code (written in Cython). Instead of merging the trees, Code.GlobalState should do the merge at the target source file block level. (Might require chaining initialization functions though.)

Migrated from http://trac.cython.org/ticket/341

[robertwb]

scoder changed description from

Currently, utility code written in Cython is injected into the main module tree right prior to code generation, so that the code generation phase cannot add new utility code (written in Cython). Instead of merging the trees, Code.GlobalState should do the merge at the target source file block level. (Might require chaining initialization functions though.)

to

Currently, utility code written in Cython is injected into the main module tree right prior to code generation, so that the code generation phase cannot add new utility code (written in Cython). Instead of merging the trees, Code.GlobalState should do the merge at the target source file block level. (Might require chaining initialization functions though.) commented

Given that the code needs to get generated into a separate namespace anyway, why not do the complete Cython utility code compilation in a separate step after the normal module code generation, and then just generate the utility code into the same code writer?

The only real problem that I see is that utility types need to be generated before the module declared types. Maybe we could distinguish between utility types and utility functions, and generate types before the module code generation and utility functions after it? It should be acceptable to require the need for utility types to be known before code generation. Utility types are a rare requirement anyway that should usually depend on the existence of tree nodes rather than special cases that pop up during code generation.

Or maybe utility types really need to become part of the module source tree and symbol tables, just with a separate C code namespace, so that methods and attributes can be properly looked up also during analysis...