search

Abstrct / Schemaverse

The Schemaverse is a space-based strategy game implemented entirely within a PostgreSQL database. Compete against other players using raw SQL commands to command your fleet. Or, if your PL/pgSQL-foo is strong, wield it to write AI and have your fleet command itself!
schemaverse.com
PostgreSQL License
356 stars 39 forks source link

Suggestion: Add in a regression test of some sort #15

Open cbbrowne opened 12 years ago

cbbrowne commented 12 years ago

It would be a good idea to add in a regression test that:

Several salutory effects: a) Means that it doesn't take much work to see if "that dumb patch someone contributed" breaks stuff b) If done carefully enough, allows using "git bisect" to figure exactly which, of 45 "dumb patches someone contributed" was to blame c) Increases confidence in proposed changes. "Yes, the system survives that surgery!" d) Provides examples of how to do stuff to would-be players.

cbbrowne commented 12 years ago

Trivial thing; just loads the schema. (Possibly badly.) https://github.com/cbbrowne/Schemaverse/commit/fc9bfac15bf1a6b58996715819ed3de96adcbe46

Abstrct commented 12 years ago

Sorry, I keep forgetting to reply to this thread. I think starting to develop a testing framework for the game is probably a great idea.

This looks like a great start so far. We just need to work in the perl scripts.

I have been working on a GettingStarted.sql script for a while (although it kind of sucks currently). It cannot be that difficult to run it a couple times to understand where it should be at certain portions of the round, then use these values to validate the changes. The only thing that should change would be the amount of fuel being returned during mine (although this can be forced with setseet() for testing purposes).

cbbrowne commented 12 years ago

Yeah, running the perl script is an apropos addition.

I'm not sure it's necessary to force completely deterministic results (e.g. - make the random bits repeatable).

The usual approach to "smoke tests", a good example being the regression test that's in the PostgreSQL build, is NOT to try to exercise every single bit of logic. Rather, to make sure that enough interesting bits are exercised that you know when things break.

For Schemaverse, it's important to know that: a) The schema still loads properly, start to end b) The ticker tics c) All the functions that players use are functioning d) A fleet script can be run

Super-comprehensive isn't necessary. Really, it would be mighty nice if the whole test runs in a few seconds. That would make it plausible to use "git bisect" to search for the stupid change that busted things. If it takes 20 minutes for the game to "warm up" before you know if the test worked, that won't work out terribly well ;-).

So, ideally, I think you'd want to: a) Load schema b) Add 2 players c) Player 1 has 1 script d) Player 2 has 2 scripts e) Fudge parameters so that they have enough money to build 50 ships, and enhance speed so that they can start attacking each other after a single turn f) Set up 3 fleets across the players g) Make sure a planet has >30 ships, so some mining is expected to fail h) Move close enough to attack, and do so i) Repair some damage j) Look at the events table to make sure that everything expected occurred

In effect, a successful run gives a known output that's basically a COUNT(*) query against my_events.