Vlad-Shcherbina / icfpc2015-tbd

2 stars 0 forks source link

Interface to their server #12

Open Vlad-Shcherbina opened 9 years ago

Vlad-Shcherbina commented 9 years ago

(I apologize for not opening this issue from the beginning, so lots of stuff went into #6 which is a different thing)

As I understand it, this system allows to send solutions (integration with interactive UI is pending), and to look at recent submissions and compare score computed by our simulator with theirs.

It seems to be done, with usage instructions are in #6 (I haven't checked them yet).

manpages commented 9 years ago

Usage instructions are scattered around #6 and api.py.

Also, for protocol, we went with my sshfs mounting sqlite idea, which was implemented by @graphite.

So we run all the Python code locally and side effects of storage io happen over network affecting centralized database.

peluche commented 9 years ago

The "Contradictions" part of the webpage doesn't seem to work.

We were lucky enough to find a power phrase randomly so our locally calculated score was 1425, but we actually scored 1731 on the remote server and the contradictions still shows ∅

manpages commented 9 years ago

@peluche First of all, lol. Second of all, I bet the idiot who wrote SQL for contradictions forgot to account for poweScore deviance. Is the problem still actual?

Vlad-Shcherbina commented 9 years ago

I managed to use submission system successfully with the following steps:

First, make sure that your public key is submitted in public-keys/. If you did it recently, ping @manpages or @graphite . Then run

production/golden/port-tunnel.sh

Keep terminal open.

Now run

python -m production.ui --prompt_for_submit

Play the game interactively (see #4 for details). At the end it will ask if you want to submit final solution. Confirm.

Then go to http://localhost:55315/ . You will see recent submissions, including yours. Its status will be in progress. It will take them about 10 minutes to test it.

If your submission is in table scoring submissions, it means our simulator produces the same score as theirs. If your solution was interesting, consider adding it to golden tests #6.

If it's in contradictions, that means there is a divergence between our simulator and theirs.

manpages commented 9 years ago

A way to replay a game —

python3 -m production.ui --problem qualifier/problem_6.json \
                         --prompt_for_submit                \
                         --seed 0                           \
                         --moves 'ei! ei! ei! ei!'          \
                         --delay=0.001
manpages commented 9 years ago
manpages commented 9 years ago

Vlad's solver makes now requests.

I moved http- functions into api! Use only those functions from now on. The one in utils is left just for backward-compatibility!

There are two extremely important things to do —

  1. Fix contradiction detection! Expected semantics of powerScore of our implementation is different from organizers' (see current contradictions, for instance №1176e3baec81db482e4af05e21881b3f 23:314:337)
  2. Make tournament API and make use of it. name will be name of implementation then.

There is a minor concern —

  1. Current SQL implementation is inefficient when we search for contradictions (and maybe somewhere else), so we sometimes still get error 500 (even though I moved database into ramfs). If you have nothing better to do, you can do something about it (write code that clears old results, optimize queries). I strongly advice against doing this though, for now it's far from being critical.