microsoft / jericho

A learning environment for man-made Interactive Fiction games.
GNU General Public License v2.0
253 stars 42 forks source link

Verify quality of valid action detection #33

Open mhauskn opened 4 years ago

mhauskn commented 4 years ago
mhauskn commented 4 years ago

Library step 35. gold_act: Ask technician about security gates-(((64, 16), (67, 61)), ((20, 8), (64, 8)), (), ()) not in valids. Reason: security gates not present at current location (i.e. no Interactive Object). ztuu

  1. gold_act: take blue-(((105, 20),), ((105, 12),), ((105, 2),), ()) not in valids. 'blue lens' is lumped with green, black, and red lenses due to same examine response. We choose 'red lens' and cannot get blue back.
  2. gold_act: put blue in glasses-(((105, 50),), ((50, 3),), (), ()) not in valids; Same story.
  3. gold_act: shake lamp-((), ((28, 9),), (), ()) not in valids; Player in darkness - all objects give same examine output "Darkness, noun. An absence of light to see by."
  4. gold_act: drop zm$100000-(((59, 128),), (), (), ()) not in valids. Obs: Dropped.; Dictionary contains 'zm$1' but not 'zm$100000'. However it's definitely recognized since 'x zm$100000' gives you a legit response. Loose
  5. gold_act: knock knock-((), ((99, 8),), (), ()) not in valids. Obs: "Who's there?" a voice inquires from inside the bungalow.; knock OBJ is a template, but knock is likely never detected as an interactive object.
  6. NoWorldChange gold_act: boo, obs: The door opens a crack. "Boo who?"
  7. gold_act: boo-((), (), (), ()) not in valids. Obs: The door opens a crack. "Boo who?"
  8. NoWorldChange gold_act: ask mary for ladder, obs: Emulator halted due to runtime error.
  9. gold_act: ask mary for ladder-((), (), (), ()) not in valids. Obs: Emulator halted due to runtime error.; Seems the emulator starts experiencing errors from step 13 and onwards. Reverb
  10. gold_act: get spray-(((49, 20),), ((49, 18),), ((49, 23),), ()) not in valids. Obs: Taken. The building you are in shakes ominously under the force of the earthquake. You lean up against a wall until it subsides. "SO, ya wanted to be a hero, kid? I'd say your future as the Lone Ranger isn't looking too promising."; Hairspary/spray is lumped with many other objects due to the same examine response. Afflicted
  11. gold_act: take tongs-(((171, 57),), (), ((171, 3), (176, 3), (64, 3), (192, 3), (62, 3), (73, 3), (58, 3), (59, 3), (60, 3)), ()) not in valids. Obs: Taken.; Tongs are not in the location description. Also, world_object tree names are missing.
  12. gold_act: take eyes with tongs-(((77, 57),), ((77, 39),), ((77, 3), (171, 3), (176, 3), (64, 3), (192, 3), (62, 3), (73, 3), (58, 3), (59, 3), (60, 3)), ()) not in valids. Obs: The eyes bob on the surface of the dark pool each time you stab them with the tongs, but after a bit of practice you succeed in removing them from the sewer.; Eyes aren't in the look description since they are hidden in sewer and world_objs don't have names.
  13. gold_act: take opener-(((113, 57),), (), ((113, 3), (180, 3), (59, 3), (60, 3)), ()) not in valids. Obs: Taken.; Letter opener not in look desc or inv. Acorncourt 5-8. gold_act: fix bucket with acorn-(((30, 25),), (), (), ()) not in valids. Obs: You plug one of the holes in the tin bucket with a green acorn. There are now only 3 holes left.; 'acorn' not in interactive objects (only plural acorns in description) and the game doesn't allow 'fix bucket with acorns'. Furthermore world objects list them as ACORN2, ACORN3, etc which don't respond to examine. Infidel
  14. gold_act: roll statue sw-(((223, 127), (128, 129), (126, 127), (133, 127)), ((127, 29),), ((128, 25), (126, 25)), ()) not in valids.; Statue is not in desc and thus not detected.
  15. gold_act: put diamond cluster in first hole-(((104, 181), (143, 175)), ((143, 18),), (), ()) not in valids.; Hole is not in loc desc and thus not detected.
  16. gold_act: open door-(((104, 195),), ((194, 25),), (), ()) not in valids.; Door not in loc desc and thus not detected. Inhumane
  17. gold_act: push coffin n-(((15, 104), (103, 104), (108, 104)), (), (), ()) not in valids. Obs: You lean on the coffin, and it scrapes along the floor.
  18. gold_act: push coffin w-(((15, 95), (108, 95)), (), (), ()) not in valids. Obs: You lean on the coffin, and it scrapes along the floor. We have a template for push OBJ OBJ and detect the coffin object, but north and west are not objects - thus these actions aren't generated. 905
  19. gold_act: no-((), (), (), ()) not in valids. Obs: It turns out that the ramp was blocked anyway - some sort of maintenance work. You keep driving.
  20. gold_act: no-((), (), (), ()) not in valids. Obs: Soon the Loungent building is in your rear-view mirror. You keep driving. It seems the decisions to exit the freeway aren't encoded in a ram address or any of the world objects. This is a bit of a mystery. Murdac
  21. gold_act: howl arac-(((12, 61),), (), (), ()) not in valids. Obs:
  22. gold_act: take peridots-(((45, 7),), (), (), ()) not in valids. Obs:
  23. gold_act: drop peridots-(((45, 66),), (), (), ()) not in valids. Obs:
  24. gold_act: gnoeval-(((42, 116),), (), (), ()) not in valids. Obs:
  25. gold_act: oozelumny-(((7, 70),), (), (), ()) not in valids. Obs:
  26. gold_act: agganrw-(((30, 97),), (), (), ()) not in valids. Obs:
  27. gold_act: prick dummy-((), (), (), ((3365, 1),)) not in valids. Obs:
  28. gold_act: exodus-((), (), ((46, 5),), ((3365, 2),)) not in valids. Obs:
  29. gold_act: sword-(((12, 7),), (), (), ()) not in valids. Obs: Darkness descends on the keep. There are sounds the like Lots of special non-english word actions and specialized keywords. Periodots not detected as noun. Ludicorp
  30. gold_act: push button-((), ((158, 8),), (), ()) not in valids. Obs: The gate slides open, making a nasty grinding noise. It doesn't sound like it'll close again.
  31. gold_act: push button-(((55, 20),), ((55, 12),), (), ()) not in valids. Obs: The machine dispenses a blank card, which you pick up. Button isn't is obs desc (62) and 170 - the world object is named 'Card Dispenser Button'. Enter
  32. gold_act: open door-((), ((179, 14), (173, 9)), (), ()) not in valids. Door not in obs desc.
  33. gold_act: say -120 to garrulous-((), (), (), ((9667, 1),)) not in valids. Obs: "That's it, Doug," says Ms. Garrulous. "I knew you you could do it!"
  34. gold_act: knock on door-(((78, 136),), (), (), ()) not in valids. Door not in obs desc.
  35. gold_act: say rat to Picasso-(((121, 20),), ((121, 12),), (), ()) not in valids. Obs: "That's it, Doug," says Ms. Picasso. "Nice work. Thanks for some real thinking! Here's a little reward for you." Wishbringer
  36. gold_act: OPEN DOOR-((), ((238, 23), (77, 25)), ((238, 23),), ()) not in valids. Obs: Door not in obs
  37. gold_act: SAY KALUZE-((), ((208, 30), (208, 25)), (), ()) not in valids. Obs: KALUZE not in obs.
  38. gold_act: OPEN DOOR-(((10, 165),), ((238, 23), (199, 25)), ((238, 23),), ()) not in valids. Door not obs.
  39. gold_act: UNLOCK DOOR WITH STEEL KEY-(((82, 220), (10, 116)), ((238, 23), (238, 23), (154, 20), (156, 25)), ((238, 23), (238, 23), (156, 14), (156, 30), (82, 30)), ()) not in valids. Door not in obs
  40. gold_act: KNOCK ON DOOR-(((238, 63),), ((238, 23), (77, 25)), ((238, 23),), ((9678, 77),)) not in valids. Obs: Yomomma
  41. gold_act: norbert
  42. gold_act: point at cola
  43. gold_act: point at vodka
  44. gold_act: look under table
  45. gold_act: look at tag through lens
  46. gold_act: sleaze-((), ((80, 44),), (), ()) not in valids
  47. gold_act: scrape gum with card
  48. gold_act: open satchel Jewel
  49. gold_act: ask allarah about white
  50. gold_act: ask allarah about black
  51. gold_act: ask allarah about red
  52. gold_act: ask allarah about jewel
  53. gold_act: ask dragon about jewel Colors white/black/red are not detected as interactive objects. Plundered
  54. gold_act: SMASH WINDOW WITH COFFER - Proper noun "window sill" prioritizes 'sill' over 'window', and 'smash sill with coffer' doesn't work.
  55. gold_act: CLIMB ROPE - Rope not in obs desc
  56. gold_act: REMOVE CLOTHES - Clothes not in obs desc
  57. gold_act: GET TREATISE - Treatise not in obs desc (but in examines)
  58. gold_act: PRESS ISLAND - Island not in obs desc Seastalker Much of the dialog has no world change. Game runs too slowly to complete this process. Zork2 Actions to command the robot are not easily generated. Balances Learning spells is general not detected as valid because the name of the spell is almost never in the observation. Theatre -- Note: there is a crash at the end of this game.
  59. gold_act: get popcorn-(((75, 15),), ((75, 14),), (), ()) not in valids. Obs: You scoop up the popcorn (there is plenty more in the barrels).
  60. gold_act: drop all but pager-(((78, 49), (77, 49), (76, 49), (75, 49), (35, 49)), (), (), ()) not in valids. Obs: old stale popcorn: Dropped. old stale popcorn: Dropped. old stale popcorn: Dropped. old stale popcorn: Dropped. loose page: Dropped. What now
  61. gold_act: push piano east-(((323, 15), (15, 287), (26, 287), (285, 287), (323, 287)), ((323, 14),), (), ()) not in valids. Obs: The piano moves easily as you push it. South End of Long Corridor This is a long wood panelled corridor that goes north from here. Doors lead south and west. The piano rolls to a stop. What now
  62. gold_act: drop all but lens-(((124, 49), (31, 49), (37, 49), (117, 49), (33, 49), (42, 49), (28, 49)), (), (), ()) not in valids. Obs: old ticket: Dropped. loose page: Dropped. loose page: Dropped. Rienhart's corpse: Dropped. loose page: Dropped. loose page: Dropped. loose page: Dropped. What now
  63. gold_act: open cabinet-((), ((290, 16),), (), ()) not in valids. Obs: You open the medical cabinet, revealing a loose page, some pain killer tablets and a stethoscope. What now
  64. gold_act: pull dial-((), ((112, 10),), (), ()) not in valids. Obs: You pull the little dial out of the watch, a little bit. What now Lurking
  65. gold_act: PRESS 4-((), (), (), ((11251, 4),)) not in valids. Obs: The timer display now reads 0:04.
  66. gold_act: PRESS 5-((), (), (), ((11251, 5),)) not in valids. Obs: The timer display now reads 4:05. 12. gold_act: PRESS MED-((), (), (), ((961, 3),)) not in valids. Obs: The bottom of the display now reads "medium."
  67. gold_act: READ PAGE
  68. gold_act: LIGHT FLASHLIGHT Karn Dialog and commands given to k9 aren't generated. Hollywood Suffers from a segmentation fault after step 65: GET PILLAR, when a subsequent LOOK command is executed. The segfault involves decoding text from dumb_output. Partyfoul Encounters a runtime error. Skipping for now. Sherlock Too slow, skipping for now. Trinity Too slow, skipping for now.
MarcCote commented 2 years ago

Regarding the segfault in Hollywood, see patch here: https://github.com/microsoft/jericho/pull/54/files#r797250250

MarcCote commented 2 years ago

Regarding the stack overflow in Loose, see patch: https://github.com/microsoft/jericho/pull/54/files#r799577224