No errors or warnings, but can't compile

[GoogleCodeExporter]

What steps will reproduce the problem?
1. make

This is the output on Fedora 11 x86_64:
g++  -O2 -pipe -Wno-deprecated  -I.
-I/home/frafra/Scaricati/shedskin-0.1.1/lib
/home/frafra/Scaricati/shedskin-0.1.1/lib/sys.cpp
/home/frafra/Scaricati/shedskin-0.1.1/lib/builtin.cpp life_shed.cpp
/home/frafra/Scaricati/shedskin-0.1.1/lib/re.cpp -lgc -lpcre  -o life_shed
/home/frafra/Scaricati/shedskin-0.1.1/lib/builtin.cpp:688: error:
‘uint32_t’ does not name a type
/home/frafra/Scaricati/shedskin-0.1.1/lib/builtin.cpp: In member function
‘virtual int __shedskin__::str::__hash__()’:
/home/frafra/Scaricati/shedskin-0.1.1/lib/builtin.cpp:738: error:
‘SuperFastHash’ was not declared in this scope
life_shed.cpp: In member function ‘virtual __shedskin__::tuple2<int, int>*
__life_shed__::__gen_product::next()’:
life_shed.cpp:228: error: cannot convert ‘__shedskin__::tuple2<void*,
void*>*’ to ‘__shedskin__::tuple2<int, int>*’ in return
make: *** [life_shed] Error 1

This is the output on Ubuntu Jaunty i386:
g++  -O2 -pipe -Wno-deprecated  -I. -I/usr/lib/shedskin/lib
/usr/lib/shedskin/lib/sys.cpp /usr/lib/shedskin/lib/builtin.cpp life.cpp
/usr/lib/shedskin/lib/re.cpp -lgc -lpcre  -o life
life.cpp: In member function ‘virtual __shedskin__::tuple2<int, int>*
__life__::__gen_product::next()’:
life.cpp:228: error: cannot convert ‘__shedskin__::tuple2<void*, void*>*’
to ‘__shedskin__::tuple2<int, int>*’ in return
make: *** [life] Errore 1

Here's my code. The first function (product) is a reimplementation for
shedskin of itertools.product, fully working.

I'm using shedskin 0.1.1 and no errors or warnings are visible during:
$ shedskin life_shed.py

Original issue reported on code.google.com by frap...@gmail.com on 16 Jun 2009 at 6:31

Attachments:

• life_shed.py

[GoogleCodeExporter]

thanks for reporting!

the x86_64 specific issue is fixed in SVN (some old types are no longer 
supported in
newer versions of GCC):

http://code.google.com/p/shedskin/source/detail?r=751

the other issue is caused by a missing cast (tuple<void *..> should be cast to
tuple<int..> here).. I will look into that. in the meantime, if you'd like to 
compile
the code anyway, you can just change 'void *' with 'int'. 

Original comment by mark.duf...@gmail.com on 16 Jun 2009 at 7:49

• Changed state: Accepted

[GoogleCodeExporter]

Thank you :)

Now it's fully working for me. Actually, it's 3.3 time faster, good :)
And with another program that I've wrote, it's 10 time faster :D

Thank you for your work ;)

Original comment by frap...@gmail.com on 17 Jun 2009 at 6:26

[GoogleCodeExporter]

thanks for the feedback!!

I had a quick look at the problem, and apparently I forgot to add casts for 
yield
statements. so that should be easy to fix.. 

Original comment by mark.duf...@gmail.com on 17 Jun 2009 at 7:20

[GoogleCodeExporter]

okay, I updated SVN with a fix for this.. thanks again for reporting, and 
please let
me know if you run into anything else.

Original comment by mark.duf...@gmail.com on 18 Jun 2009 at 10:22

• Changed state: Fixed

[GoogleCodeExporter]

btw, if you replace this:

for a,b in enumerate(c):

with this:

for a in range(len(c)):
    b = c[a]

you will get a large speedup. this is because with enumerate, you are implicitly
generating lots of tuple objects, which slows down C++ a lot.

also, you might want to use shedskin -bw for a secondary speedup (but note your 
code
for parsing arguments won't raise an IndexError anymore now, but this is easy 
to work
around).

Original comment by mark.duf...@gmail.com on 21 Jun 2009 at 9:13

[GoogleCodeExporter]

after profiling life_shed.cpp a bit, I decided to optimize list slicing 
(something I
should have done ages ago). this makes the program a bit faster still.. now it's
about 30 times faster here (after replacing the enumerate statements). if you'd 
be
interested in trying, please pull from SVN.

Original comment by mark.duf...@gmail.com on 21 Jun 2009 at 11:00

[GoogleCodeExporter]

Woow! It's so fast! It's 10 times faster of the last shed_skin version, so 
about 30
times faster of the original version. Thanks :)

Original comment by frap...@gmail.com on 22 Jun 2009 at 8:32

[GoogleCodeExporter]

I could also have a look at the other program you mentioned, if you like? 

btw, I'm working on a fix so that 'for a,b in enumerate(some_sequence)' will 
become
as fast as manually replacing it.. thanks for reminding me of this useful 
optimization.

Original comment by mark.duf...@gmail.com on 22 Jun 2009 at 7:49

[GoogleCodeExporter]

Sure ;) This is only a class with a little test, nothing more.
I thinked: it could be possible to use shed for qt programs? c and python use 
the
same methods for qt :)

Original comment by frap...@gmail.com on 22 Jun 2009 at 9:13

Attachments:

• idee.py

[GoogleCodeExporter]

some tips for better performance:

-use sets for 'in' checks and 'index'. for lists, these imply walking over the 
list
every time (which isn't faster in C++), while for sets, this takes only 
constant time
(one step).

-shedskin -r boosts performance a bit more (shedskin now uses C random numbers
instead of Python compatible ones)

-there is still one problematic line: del self.uids[self.uids.index(uid)]. 
because of
index, but also because 'del' on a list means shifting on average half of the 
list.
you might be able to find a faster solution usings sets or possible a linked 
list, so
half the list doesn't have to be shifted each time.

see attachment for a slightly modified version.

note that you really don't want to compile qt code. because a) interface glue 
won't
get any faster, and b) this would restrict your code too much. a better 
solution is
to compile an extension module (shedskin -e, see the tutorial), and import this 
in
some 'main' program that uses Qt.

Original comment by mark.duf...@gmail.com on 23 Jun 2009 at 9:06

Attachments:

• idee.py

[GoogleCodeExporter]

Thanks so much, I'll try the extension module approach.

Original comment by frap...@gmail.com on 23 Jun 2009 at 5:47

[GoogleCodeExporter]

I've changed a bit the test part, when it tries to remove some random items. So 
I can
use only sets. The performance of this python file are shown here:

[frafra@rocketman idee]$ time python idee_shed.py
[...]
real    0m3.777s
user    0m3.676s
sys 0m0.064s
[frafra@rocketman idee]$ ../../shedskin -r idee_shed.py
[...]
[frafra@rocketman idee]$ time ./idee_shed
[...]
real    0m0.209s
user    0m0.162s
sys 0m0.042s

So, about 18 times faster :)
Yes, the test code makes the performances a bit more different for every run, 
but it
doesn't matter :)

Original comment by frap...@gmail.com on 23 Jun 2009 at 6:39

Attachments:

• idee_shed.py

[GoogleCodeExporter]

I finally sat down and optimized the following in SVN:

.. for var, .. in enumerate(some list/tuple) ..

so the hack with 'for .. in range(len(..))' is not necessary anymore for 
life_shed.

I'm hoping to do something similar for 'zip', before the next release:

.. for .., .. in zip(list/tuple args) ..

both patterns are quite common, so I should have done this a long time ago..

btw, did you try to build an extension module yet?

Original comment by mark.duf...@gmail.com on 25 Jun 2009 at 5:43

[GoogleCodeExporter]

Actually I'm doing an important exam, please excuse me :) I'll try as soon as I 
can ;)

Original comment by frap...@gmail.com on 25 Jun 2009 at 8:48