Create and use a unique cog module for each file processed.

[dmurdin]

Change-Id: Icd9f0aa7ff1f0714322ed90c9db16f0dffbc3a50

I use SCons for building, and as SCons is python I, just as Cog, have a "module" that can be imported into the Cog-files with various data. I then run Cog on the files by calling Cog.main() directly. However since Cog internally sets up a "cog" module that sets up output to correct files and this is imported by Cog-files, parallell building is not supported. I thus modified cogapp.py to create and use a unique "cog module" for each file that can be imported safely. This should have no effect on the nominal way with running cog.py from commandline, but enables parallell (threaded) building within one python interpreter instance.

cogapp automatically imports cog, so this does not need to be done by user. Documentation examples (and legacy code I assume), might explicitly "import cog", so I also added "cog" as a valid name for the "cog module", note this usage will then still cause clashes with threaded code (no worse than current master though).

[nedbat]

Thanks for repeating this here on GitHub, and for accounting for the explicit import case.

Help me understand how we can test this.

[dmurdin]

Help me understand how we can test this.

Running cogapp as previously should work as before of course.

We run it by doing something like:

import cogapp
arguments = ['cog.py', '-o', 'cogappfile.c.src, '-d', '-D', 'TAG=w']
cogapp.Cog().main(arguments)

Running cogapp.Cog().main(arguments) from multiple threads (with different files) is what can cause problems. Some tests with threads pausing/starting and checking that they get "their" correct cog could be done if we want tests explicitly for problem this patch solves.

[nedbat]

Thanks, I have a test I will add after merging.