pauses in processing

[GoogleCodeExporter]

Hi,

I see random pauses in processing.

You can see animation stutter with this example:
python -m pygame.examples.testsprite

Mostly(or entirely) within the first few seconds, and then it stabilizes.

cheers,

Original issue reported on code.google.com by ren...@gmail.com on 10 Jan 2010 at 12:49

• Merged into: #40

[GoogleCodeExporter]

That makes sense, that's the JIT compiling hot functions.  :)  The garbage 
collector
should be causing similar problems, unless you've disabled it or you're not 
making
cycles.

Our plan for reducing the pause lengths is to offload the JIT compilation into a
secondary thread.  However, that thread needs to hold the GIL while it 
generates LLVM
IR from pyc bytecode, so there will still be short execution pauses until the 
GIL
situation is improved.  Fortunately, the majority of the compilation time is 
spent on
LLVM optimizations and code generation.

Another alternative if the pauses are unacceptably long is to disable JIT 
compilation
during the timing sensitive portion of your program.  The following code will 
disable
the JIT on unladen and have no effect on CPython:

try:
    import _llvm
    _llvm.set_jit_control("never")
except:
    pass

Just be aware that if you disable the JIT, performance will suffer.

Original comment by reid.kle...@gmail.com on 10 Jan 2010 at 8:59

• Changed state: Duplicate

[GoogleCodeExporter]

Actually, I've been experimenting with background bytecode manipulation - all 
you
need is to hold the GIL while incref-ing all the code objects you'll need, then,
since code and string objects are immutable (and you'll only be dealing with 
those),
you can manipulate them in the background safely.

You must then hold the GIL while committing the change (the generated IR), which
again is a short time.

Original comment by klaussfr...@gmail.com on 10 Jan 2010 at 9:20

[GoogleCodeExporter]

Not quite.  So code objects are *mostly* immutable, but they now point to some 
stuff
which isn't, like the feedback data.  Furthermore, we want to be able to do 
things
like reach into the globals and builtins dicts during compilation and cache 
pointers
to things like the "len" function or other global constants.  With the LOAD_ATTR
optimization, we now look at the type objects found in the feedback and their
descriptors.  Eventually, I think we're going to want to do even more.

At first I tried to only acquire the GIL for these feedback optimizations, but I
didn't want to create a GIL battle situation like the one David Beazley 
described. 
It also makes maintenance more complex, because you have to verify that all the 
Py*
functions you're calling are thread-safe.

In the end, the approach that I'm using now is to just acquire the GIL for the
bytecode translation, since I *believe* the majority of compilation time is in 
the
code generation.  There is instrumentation to measure this, but I haven't 
looked at
it in awhile.

Original comment by reid.kle...@gmail.com on 10 Jan 2010 at 10:08