Automated testing support

[GoogleCodeExporter]

A lot of tests have been created in the pspautotests repo on GitHub.

https://github.com/hrydgard/pspautotests

While interactive demo apps are very useful, the benefit of these automated 
tests is that they can be run quickly and without any UI, and generate a clear 
pass/fail result.  The PPSSPP buildbot runs all tests each commit.

Sometimes when implementing features or creating tests, I've checked JPCSP's 
code, and noticed PSP behavior that didn't seem to be implemented.  Being able 
to easily run these tests would show for sure, and could help improve 
compatibility (although, e.g. the LwMutex tests are very detailed, and I doubt 
any game actually cares about the accuracy there, although PPSSPP does 
implement it without problems or real speed impact.)

Supporting these tests is fairly easy.  The tests send commands to the 
"emulator:" device using sceIoDevCtl() function, as follows:
 * cmd 3: IS_EMULATOR, writes a 1 to the output ptr.
 * cmd 2: SEND_OUTPUT, the input data (of input length) indicates data to be compared with the expected data (it can be written to a file or stdout as in PPSSPP first.)
 * cmd 32: EMIT_SCREENSHOT, the current display buffer should be compared to the left 480 rows of the 512x272 expected.bmp file.

Note that the tests, while comprehensive in some areas, are still not complete 
of course.  For example, there is little to no testing of syscalls within 
interrupts.

-[Unknown]

Original issue reported on code.google.com by unknownb...@gmail.com on 19 Aug 2013 at 2:33

[GoogleCodeExporter]

Just to give an example, I recently did tests on FPLs on a PSP.

At least in 6.60, blocks in an FPL are allocated sequentially, even if blocks 
are freed.  In other words, if A and B are allocated, then A is freed, and C is 
allocated, the filled positions will be 2 and 3, and 1 will still be free.

This can be seen pretty clearly here (generated by a PSP-3000 running 6.60):
https://github.com/hrydgard/pspautotests/blob/2121ec9e74438c66bf8d2ef9e28274b86d
dc1496/tests/threads/fpl/allocate.expected#L12

And I don't think JPCSP does it properly:
http://code.google.com/p/jpcsp/source/browse/trunk/src/jpcsp/HLE/kernel/types/Sc
eKernelFplInfo.java?r=3019#128

There are other behaviors, such as an error when the alignment value is wrong, 
that I didn't check whether JPCSP implements.  Some aren't likely to affect 
compatibility, but you never know.

-[Unknown]

Original comment by unknownb...@gmail.com on 26 Aug 2013 at 12:42

[GoogleCodeExporter]

Thank you for the information!
I did not check your updates on pspautotests... this is an amazing progress!

Original comment by gi...@web.de on 18 Sep 2013 at 11:27

• Changed state: Started

[GoogleCodeExporter]

Oh, so it already had some support - I didn't realize.  I could swear I looked 
at the --help and didn't see it, my bad.

Glad it's useful.

-[Unknown]

Original comment by unknownb...@gmail.com on 20 Sep 2013 at 2:55