Closed GoogleCodeExporter closed 9 years ago
not sure why I wrote this in such a stupid way before, but this is much better
obviously, and the problem is gone now:
http://gitorious.org/shedskin/mainline/commit/49c126bd74bdda3d55311a5c80fa709fe5
627716
thanks a lot for tracking this down! :D should be another nice fix for 0.8..
Original comment by mark.duf...@gmail.com
on 15 May 2011 at 3:07
Thanks for the fix :D
Yeah, I'll try to get International Karate 2 running :D
And Giana Sisters
And Bubble Bobble :D
And Microprose Soccer... I remember the hard physics in that game, it was
almost impossible to score a goal in the rain :D
http://channelf.se/c64fav.html aha :D
As for how far I am: well, the only major things missing are disk drive and
sound chip and exotic graphics modes. There a small things missing too - like
VIC collission detection. First have to read up on when exactly it counted as a
collision (after all if you had a checkered background over the entire screen
that didn't count as collision for the sprites on the screen, did it? O_o)
Btw, I found a small bug in the struct module: it ignored the endianness; patch
with the fix is attached.
Original comment by danny.m...@gmail.com
on 15 May 2011 at 4:15
Attachments:
pushed, thanks a lot! :D
it sounds like you are making good progress!
how large has your program become by now (using sloccount)? did you run into a
case yet where shedskin did not terminate, or aborted after the max number of
iterations?
Original comment by mark.duf...@gmail.com
on 15 May 2011 at 4:42
[deleted comment]
SLOC Directory SLOC-by-Language (Sorted)
14987 c64 cpp=12121,python=2866
701 top_dir python=701
409 lib cpp=409
49 tool python=49
8 pygtk_s python=8
0 data (none)
0 doc (none)
0 image (none)
0 tapclean (none)
Totals grouped by language (dominant language first):
cpp: 12530 (77.57%)
python: 3624 (22.43%)
No, as far as I remember, shedskin always terminated so far :-)
Hm, would it be possible to make the AssertionError a little bit more helpful?
Right now sometimes I'm getting...
Traceback (most recent call last):
File "/home/dannym/source/playground/trunk/emu/C64/gmonitor.py", line 92, in fire_timer
self.C64.fire_timer()
AssertionError
... from my own asserts in the "shedskin -e"'d module. But which assert was it?
Even just printing the assert condition on stderr would help a lot; the way it
is now I have to guess which it was... (Even abort() would be better, then I
could run "gdb --args python2 c64_main.py" and on the abort, read the backtrace
to see where it was...)
btw, International Karate crashes with an exotic instruction after a LOT of
fine ones somewhere (usually a sign that I lost track somewhere along the way).
Original comment by danny.m...@gmail.com
on 16 May 2011 at 7:35
thanks! so do you compile all those 2866 lines with shedskin..? and is it
getting much slower over time as the program grows? I'm interested, because
it's the largest program to compile with shedskin so far, that I know of at
least.. :)
showing a backtrace in C++ can be done I think, but we would probably have to
add -fno-omit-framepointer to FLAGS at least, which can really slow things
down. alternatively we could add line numbers to assertion exceptions, but then
this doesn't seem very elegant.
an easy workaround for now is to just add messages to the assert statements,
indicating each statement. for example, this:
assert 1==0, '1==0, check 3'
will give you this:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError: 1==0, check 3
Original comment by mark.duf...@gmail.com
on 17 May 2011 at 6:00
Yes, all those 2866 lines are compiled with shedskin.
It takes pretty long so I usually don't wait in front of the computer for it to
complete - I go eat something, for a walk in the park, do some maths, whatever.
So I don't really know. But it feels as it takes as long as ever - though
that's just bias from me preferring never memories probably :-)
As for the backtrace, given that assert()s are "should never ever happen" marks
(or at least I use them as such), it would be overkill to have stack frames for
the sole purpose of locating where one was in the unlikely event one should
fail.
I didn't know it was possible to pass a second argument to assert, so since
that works, it's fine the way it is. Thanks :-)
Original comment by danny.m...@gmail.com
on 21 May 2011 at 9:35
okay, so no need to worry about this further then. I wish I had time to look
into everything, but unfortunately I'm not getting any development help at the
moment.. I guess most good programmers are busy on their own projects, such as,
oh, c64 emulators.. :P
btw, note that you can turn off assertions completely with 'shedskin -o'
(similar to 'python -O'). this might give a speed boost, and encourage you to
put in lots more assertions without slowing the compiled code down.
2,866 lines, not bad :D when I started shedskin, I mentioned reaching 3,000
lines as one of the goals for shedskin 1.0 in the tutorial.. so you have to
work a bit harder :P
struct.pack/unpack/calcsize are almost finished here. running thousands of
randomized tests now to work out the final bugs.. thinking about one more
feature for 0.8, perhaps some better integration with numpy.
Original comment by mark.duf...@gmail.com
on 21 May 2011 at 9:54
hey, I'm thinking about releasing shedskin 0.8 within two weeks or so.. do you
think it might be timely to upgrade examples/c64 for this release? do you have
a game running yet? :) I'd be interested in showing off a larger program that
compiles in any case of course, especially if it runs giana sisters huhu :P.
but I'm not sure how close your current setup is to the one in examples..? I'd
be happy to rework your code to fit in the out-of-the-box experience again if
necessary, but it would be nicer I guess if I could convince upstream to make
such changes.. ;-)
Original comment by mark.duf...@gmail.com
on 25 May 2011 at 8:30
[deleted comment]
Hi,
cool, good to know about the option "-o"...
T64 tape loading does work now (every time I tried, yessss), and I can see
something happening after I start International Karate by "RUN". I think I
(barely with a lot of imagination) see someone fighting. The emulation of the
graphics mode it is using is buggy. I'll try to get it to run until that time -
after all, I'm sooo close.
As for the current state, what is in SVN now is the best state yet (good time
to try it, too). However, graphics bugs make it hard to see that the game is
actually running fine (as in control flow). It probably would be a simple
one-line fix per mode, but which line? ;-)
Virtual Joystick is cursor keys and right shift (where the heck is the second
button in the memory map :-) there was a second button, right?). It's currently
hardcoded to supply the first joystick.
The current setup should be very close to the one in examples:
c64/ is a shedskin -e extension module
the other stuff is CPython 2.6 code with pygtk, as usual - see README.
I think I did reorganize it to be the way you laid it out some months ago; if I
missed something that's annoying for you to redo all the time, please tell me
again :-)
Giana Sisters doesn't work yet because they use some silly (back-in-the-day
undocumented) instructions in the trainer (just for the heck of it as far as I
can see).
Original comment by danny.m...@gmail.com
on 27 May 2011 at 5:32
awesome, sounds like you're making solid progress :) great to hear also you are
using an extension module now, that should make it a lot easier for me to sync.
please keep me updated on your progress!
Original comment by mark.duf...@gmail.com
on 27 May 2011 at 6:22
hmm, this looks like something I might be able to fix..? :) let's have a look..
edit "timer.hpp" and remove '#include "c64.hpp"'
edit "timer.cpp" and add '#include "c64.hpp"'
Original comment by mark.duf...@gmail.com
on 27 May 2011 at 6:25
I think you didn't commit joysticks.py yet..?
Original comment by mark.duf...@gmail.com
on 27 May 2011 at 6:29
added a fake joysticks.py, and compiled the whole thing.. 3,000 lines, great
for me to see that work ^^ compilation took about 25 minutes, which is getting
a bit long. looks like at 5,000 lines you may have to wait 2 hours already..
great to see the cursor work better and of course the tape support! I tried to
load intkarat.t64, but ran into this:
File "/home/srepmub/C64/gdisplay.py", line 140, in repaint_sprite
GC.set_clip_mask(None)
TypeError: Gdk.GC.set_clip_mask() argument 1 must be gtk.gdk.Pixmap, not None
uncommenting this line gives me some messed up sprites, but I can't say they
look like they are fighting, more like randomly broken sprites.. ;-)
Original comment by mark.duf...@gmail.com
on 27 May 2011 at 9:47
[deleted comment]
Oops, committed joysticks.py
Ah, yes, that set_clip_mask was a pygtk bug which is only fixed in versions >
2.22.0 (not just >= 2.22.0 as their changelog said...). If that call doesn't
work, everything is messed up since then once you use any sprites you have a
sprite "stamp" used everywhere you draw anything :P
However, I'm replacing the graphics part by one that builds a pixbuf in memory
(from within the shedskin module) since International Karate changes the
graphics mode 3 times per screen (depending on which scanline we are at). Hence
the graphics screen has to manually build it up anyhow - and the call to
set_clip_mask will be gone, and so will the pixmaps and masks.
About the includes, yeah, that would be nice.
The reason why it is that way (I think) is because shedskin finds that the call
to "timeout_add" gets a C64 object. So it includes the c64 header file (naively
;-) )
Of course the c64.py module will call timeout_add (as shedskin did just figure
out ^^), so it better include timer.hpp ... But that includes the c64 header
file... which includes the timer header file... which... ooooops
Original comment by danny.m...@gmail.com
on 28 May 2011 at 6:39
yes, this circular include stuff is fun :) I thought I had solved the problem
by forward declaring all used classes inside each module, but apparently
there's no way to make this work when inheriting from them, because in this
case the C++ needs more information.. :S
so this means that if you avoid inheritance from TimingOut (which doesn't do
anything at the moment anyway), the problem will probably disappear. this might
be a useful workaround for now, so you don't have to edit the result after each
compilation.
so I thought a bit about this, and I think the problem can be solved, as long
as the inheritance relations between modules don't become circular. because it
looks like this is impossible to support in C++, and probably not very nice
design anyway.. :-)
building the whole screen inside the extmod sounds like the way to go, also
because it probably minimizes communication with the extmod and only the screen
has to be copied (which should be very fast if only done 60 times per second or
so).
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 10:35
Yeah, I noticed. Commented out the TimingOut thing and it doesn't need manual
editing now.
As for the whole-screen-inside the extmod, I've almost got this now, however it
seems there is a length limit in what size strings shedskin supports (?). That,
or I still have a bug somewhere... doesn't seem so, though...
Original comment by danny.m...@gmail.com
on 28 May 2011 at 12:06
[deleted comment]
[deleted comment]
I've created a patch
http://scratchpost.dreamhosters.com/software/patch/C64_screen-extmod.patch
which does the screen-inside thing for now. I'm still debugging the issue but
maybe it's something obvious that I can't see :-)
c64/screens.py flip() contains the code that's actually updating the pixbuf
data.
gdisplay.py repaint() contains the code that converts that to a gdk pixbuf by
calling gtk.gdk.pixbuf_new_from_data(). While the length of the data seems to
be the same, it doesn't seem to be able to read it all out. Strange...
Original comment by danny.m...@gmail.com
on 28 May 2011 at 12:35
biting my teeth on the circular problem first.. looks like I'm almost there,
but I want to test lots of things before committing. it may be there's a buggy
limit somewhere in the shedskin string code. perhaps it's possible to minimize
the problematic code to a simple test case..?
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 1:33
okay, committed the fix. please let me know when you run into similar issues!
I'm off for a long walk now (borrowed my colleague's dog), but when I get back
I can probably help you debug things.
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 2:07
I see :-)
Did, and found the bug in my code by it. Works now: ultra-slow but correct
graphics :-)
Original comment by danny.m...@gmail.com
on 28 May 2011 at 2:13
hah, congratulations :D I will sync and see for myself.. wondering why things
are ultra-slow though..?
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 3:08
getting different errors at the moment running c64_main.py from SVN.. please
let me know which revision I can try here ;)
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 3:41
btw, would it be useful for debugging to have some kind of freezing/unfreezing
operation, where you dump/load the whole c64 state? it seems awful to have to
debug some game using just cpython, or to have to wait for shedskin to finish
all the time..
shedskin could probably also dump/load its final analysis state, so a new
analysis takes perhaps only 1 minute or so, but that probably won't be
implemented anytime soon..
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 3:57
hmm I see, calling struct.pack 5 million times per second is a bit slow.. :) I
will test a bit to see if I can find something faster..
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 4:56
it's a bit ugly, but using the following in gdisplay.py, it looks like we can
update the screen about 100 times per second from an extension module on my PC:
s = struct.Struct('>'+'I'*360*300)
data = screen.get_rendered_image() # this one just does 'return data' (list of
ints)
pixbuf_data = s.pack(*data)
I have to go again, but I will think about this a bit..
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 5:04
this one seems more elegant, and a lot faster :-)
def render_to_image():
a = array.array('I', data)
return a.tostring()
only thing is, endianness is usually little here, so the result is actually
different.. perhaps it's possible to have data be a list of bytes instead, or
perhaps even a byte array..? :) then only a tostring would remain, plus
conversion, in total 2 memcpy's, which should take almost no time at all.
anyway, this one already clocks in at 300 screens per second here.. and I think
I can make it go a _lot_ faster even, by optimizing arrays internally (right
now it's quite dumb and slow).
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 6:01
alright, optimized just the array('I', list) case for now. now this one goes at
about 800 screens per second. time to walk the dog again ^^
Original comment by mark.duf...@gmail.com
on 28 May 2011 at 7:42
[deleted comment]
Hehe, nice :-)
Updated shedskin from git and tried it: works fine...
>btw, would it be useful for debugging to have some kind of freezing/unfreezing
operation, where you dump/load the whole c64 state?
Yeah, restarting from the beginning is starting to get old. Freeze/unfreeze
shouldn't be too hard to do either, just dump guest registers and guest memory.
If we're lucky, we don't even need to dump the other VIC state since it's
generated from what is visible in memory anyway (as opposed to the SID which
does not store state in a visible place - and is not implemented yet anyway...).
>shedskin could probably also dump/load its final analysis state, so a new
analysis takes perhaps only 1 minute or so, but that probably won't be
implemented anytime soon..
Yeah, would be nice - and very hard to do, probably :-)
>def render_to_image():
> a = array.array('I', data)
> return a.tostring()
Is that in the extmod or in the main python? :-)
As for the endianness, the pixel values ultimately come from a fixed palette of
16 colors (palette.py). We can just change it there in order to support any
endianness we want. The way it is now is just how pygtk's pixbuf.fill() wants
the color, i.e. how it actually is in the pixbuf buffer: 0xRRGGBBAA, no need to
be hell-bent on keeping it that way :-)
I thought about using a byte array; it would make things a little bit uglier
since then the array index isn't just "which pixel" any more but I guess it
doesn't make it any slower since finding the beginning of a row by WIDTH *
row_index is not that different to (WIDTH * 4) * row_index... where WIDTH is
constant anyway.
Somehow, calling gdk_pixbuf_new 60 times per second bothers me.
If would be nicer doing gtk.gdk.Pixbuf() just once and just giving a reference
to its buffer to shedskin somehow:
>>> import gtk
>>> p = gtk.gdk.Pixbuf(gtk.gdk.COLORSPACE_RGB, True, 8, 366, 300)
>>> p.get_pixels_array().__class__
<type 'numpy.ndarray'>
>>> p.get_pixels()
=> very long string
Is is possible to give that get_pixels() or that get_pixels_array() result to
shedskin in a mutable-there way?
So far I've broken the C64 emu trunk a bit to show you what I mean:
http://svn.nomike.com/playground/trunk/emu/C64/c64/pixbufs.py is a dummy
interface to make shedskin understand what I'd like to be able to do (i.e. for
type inference to work).
I think it's better than having the screen wander over the entire host ram
before the garbage collection kicks in (if ever).
For the native_pixbuf attribute I'm using __ss_int right now which is bad (on
amd64 for some reason the compiler uses 32 bits (!) for int and 64 bits for
void*).
Original comment by danny.m...@gmail.com
on 29 May 2011 at 11:05
>>def render_to_image():
>> a = array.array('I', data)
>> return a.tostring()
>
>Is that in the extmod or in the main python? :-)
this is in the extmod.. you could try this first, because I think it will be
really fast enough.. :-)
having cpython objects mutate shedskin objects and vice versa would be nice and
useful (especially if things get even larger, numpy arrays and such), but I
probably won't look into this for the foreseeable future..
Original comment by mark.duf...@gmail.com
on 29 May 2011 at 11:13
>shedskin could probably also dump/load its final analysis state, so a new
analysis >takes perhaps only 1 minute or so, but that probably won't be
implemented anytime >soon..
>
>Yeah, would be nice - and very hard to do, probably :-)
actually, not really.. but I think CPython is fast enough for most debugging,
with a bit of work perhaps (that's why I suggested the (un)freezing stuff..
:-)). I will probably get to this once I know of several programs of this scale
that compile (at all)..
Original comment by mark.duf...@gmail.com
on 29 May 2011 at 11:20
I followed the README in SVN now, and get the flashy colors on the side for
intkarat.t64.. seems to get stuck after that though. but it looks cool anyway
:) it's a bit slow, but cpu usage only seems to be 5% here..? perhaps just a
timing issue?
once I'm able to see the game, I'd love to get this version in
shedskin/examples. adding overrides is not an option there, but I'd be happy to
try and get something like the array solution working.
Original comment by mark.duf...@gmail.com
on 29 May 2011 at 3:31
I'm dying for updates, since international karate seems to be almost running!
:D (or perhaps it already runs for you??). once that works I'd love to get it
to work at full speed as part of examples/ without any overrides.
Original comment by mark.duf...@gmail.com
on 31 May 2011 at 1:17
Ah, I'm still going through VIC documentation to find out what I'm doing wrong
(probably one of these darn unfindable one-off-bugs)... there's also a lot of
things that I'm still using that doesn't make all that much sense now (like
sprite_datas which contains copies of the "current" sprite data part of guest
memory. I'm trying to get rid of these indirections now, i.e. maybe I should
just stop caching so much)...
As for the array solution, I've changed pixbufs.py now to make it possible to
provide for that and added a check to find out which it is supposed to be using
in the main python - i.e. it should work with and without override.
long story short: no, it doesn't work for me yet... but it will, even if it's
the last thing I do *muhahaha*</gargamel>
Original comment by danny.m...@gmail.com
on 2 Jun 2011 at 2:32
[deleted comment]
And about the 8% CPU, maybe you have more cores? It is at 90% cpu here just
flashing the cursor after the ready prompt :-)
Original comment by danny.m...@gmail.com
on 2 Jun 2011 at 2:50
it looks like you already solved the overflow problem? this happens when
array('I') receives negative values.
well, I have 4 cores, but only one was working at 5%.. anyway, with this change:
- self.timer = gobject.timeout_add(90, self.fire_timer)
+ self.timer = gobject.timeout_add(20, self.fire_timer)
it looks like I'm getting 50 screen updates per second, very fluently, with
only 18% cpu usage. so that's using the shedskin-compiled extension module,
with this inside it:
+ def get_rendered_image(self):
+ data = [(2**32+item) if item < 0 else item for item in self.data]
+ a = array.array('I', data)
+ return a.tostring()
Original comment by mark.duf...@gmail.com
on 2 Jun 2011 at 8:24
I was wondering if you are using a real C64 on the side, to help debug things,
perhaps with some power cartridge of sorts..? I imagine it can be very useful
to compare PC traces or selected portions of memory..
Original comment by mark.duf...@gmail.com
on 4 Jun 2011 at 11:44
well, another emu might be simpler.. what really helped me debug this kind of
situation before was to compare memory checksums. so you can see exactly when
there is even a minor difference, and then it's often easy to figure out.
Original comment by mark.duf...@gmail.com
on 4 Jun 2011 at 12:34
Yeah, but my C64 broke some years ago when I tried to hook a homebuilt
contraption to the user port :-(
But good idea to use another emulator.
I've used the monitor of VICE to find out tricky things yesterday...
As for the international karate problem: found a few things and fixed them:
- in order to acknowledge a raster interrupt, you have to write the same (!)
value back you just read. So
LDA $D019
STA $D019
(yes, in that order, without anything in-between).
The documentation states you have to set the bits whose interrupt you want to
acknowledge to 0, which is obviously wrong: it just writes it back unchanged :P
I've now changed it to do actual_value = actual_value & (arg ^ 0xFF) in that
case which seems to work better.
- the raster interrupt handler of international karate enables interrupts again
before it's done (?!!!?!?!?!). I've added a non-standard "Am I in a interrupt
handler currently?" flag to the emulator to compensate, i.e. not
endlessly-recurse on the interrupt enabler instruction.
I've fixed the override and it's now much faster.
And I just got an IndexError by the shedskin module... within screens.py
somewhere. Can I set a breakpoint with gdb somewhere so I can find out where
that was?
Original comment by danny.m...@gmail.com
on 4 Jun 2011 at 1:54
Tried builtin.hpp __throw_index_out_of_range and the IndexError constructor, to
no avail... strange...
Original comment by danny.m...@gmail.com
on 4 Jun 2011 at 2:03
turns out there's a copy of shedskin in "dist-packages" and one in
"site-packages"... I used the wrong one when finding the line number for the
breaekpoint (updated the computer to ubuntu only a week ago, sorry :-) ).
Original comment by danny.m...@gmail.com
on 4 Jun 2011 at 2:07
I'm looking into this, because it sounds very useful to be able to do. for now,
if you cause a real hard crash in __throw_index_out_of_range, gdb will stop and
give you a nice backtrace..
Original comment by mark.duf...@gmail.com
on 4 Jun 2011 at 2:16
btw, do you really still need the override..? is it noticably faster than
returning array.tostring()..? ;)
btw2, I think commodore 64's are still dirt cheap on those second hand
websites.. :-)
Original comment by mark.duf...@gmail.com
on 4 Jun 2011 at 2:19
so __throw_index_of_out_range was defined in the header file, causing it to be
inlined perhaps and falling out of gdb's scope. perhaps at -O0 we could break
here, but they really should be in builtin.cpp, because we never want to inline
these in fact (rare occasions, lots of instructions in them). so I moved them
there, and now you can set a future breakpoint as follows:
b __shedskin__::__throw_index_out_of_range
Original comment by mark.duf...@gmail.com
on 4 Jun 2011 at 2:48
Original issue reported on code.google.com by
danny.m...@gmail.com
on 15 Oct 2010 at 9:34Attachments: