Arity and key errors in update_with_supervised.

[CatherineWong]

Several errors are being thrown, using PyCCG at master: - Within enumeration, on reach the horse****

 raise InconsistentTypeHierarchyException("Function %s appears with the wrong arity" % fn_name)
nltk.sem.logic.InconsistentTypeHierarchyException: The variable 'Function F000 appears with the wrong arity' was found in multiple places with different types.

- During update_with_supervised:

Parse failed for sentence 'reach blue circle'
No successful parses computed.
Parse failed for sentence 'reach blue circle'
0.001 Token(reach => (S/N) {\x.move(x)})
0.001 Token(reach => (S/N) {\x.move(unique(x))})
0.001 Token(reach => N {\z2.z2(\z1.circle(z1))})
0.001 Token(reach => N {unique})
0.001 Token(reach => N {circle})
0.001 Token(reach => N {\z2.unique(\z1.z2(z1))})
0.001 Token(reach => N {move})
0.001 Token(below => (S/N) {\x.move(unique(\y.relate(y,x,down)))})
0.001 Token(below => N {down})
0.001 Token(above => (S/N) {\x.move(unique(\y.relate(y,x,up)))})
0.001 Token(above => N {up})
0.001 Token(, => ((S\S)/S) {\a b.a})
0.001 Token(, => ((S\S)/S) {\a b.b})
0.001 Token(of => ((N\N)/N) {\x d y.relate(x,y,d)})
0.001 Token(of => ((N\N)/N) {\x d y.relate(unique(x),d,y)})
0.001 Token(to => ((N\N)/N) {\x y.x})
0.001 Token(one => ((S/N)/N) {\d x.move(unique(\y.relate(y,x,d)))})
0.001 Token(one => ((S/N)/N) {\d x.move(unique(\y.relate_n(y,x,d,1)))})
0.001 Token(right => (N/N) {\f x.and_(apply(f,x),in_half(x,right))})
0.001 Token(right => N {right})
0.001 Token(most => ((N\N)/N) {\x d.max_in_dir(x,d)})
0.001 Token(the => (N/N) {\x.unique(x)})
0.001 Token(left => N {left})
0.001 Token(horse => N {\x.horse(x)})
0.001 Token(rock => N {\x.rock(x)})
0.001 Token(rock => N {unique(\x.rock(x))})
0.001 Token(cell => N {\x.true})
0.001 Token(spade => N {\x.spade(x)})
0.001 Token(spade => N {unique(\x.spade(x))})
0.001 Token(heart => N {\x.heart(x)})
0.001 Token(heart => N {unique(\x.heart(x))})
0.001 Token(circle => N {\x.circle(x)})
3.000 Token(triangle => N {unique(triangle)})
0.001 Token(diamond => N {diamond})
0.001 Token(house => N {house})
0.001 Token(blue => ((N\N)/N) {\z3 z2.z2(unique(\z1.z3(z1)))})
0.001 Token(blue => ((N\N)/N) {\z3 z2.move(z2(\z1.z3(z1)))})
Updating PyCCG with supervised on move(unique(\z1.rock(z1))) 
Parse failed for sentence 'reach cell rock'
No successful parses computed.
Parse failed for sentence 'reach cell rock'
Traceback (most recent call last):
  File "/om2/user/zyzzyva/pyccg/pyccg/word_learner.py", line 300, in update_with_supervision
    self.lexicon, sentence, model, learning_rate=self.learning_rate, lf=lf)
  File "/om2/user/zyzzyva/pyccg/pyccg/perceptron.py", line 143, in update_perceptron_supervised
    **update_perceptron_kwargs)
  File "/om2/user/zyzzyva/pyccg/pyccg/perceptron.py", line 67, in update_perceptron
    raise NoParsesError("No successful parses computed.", sentence)
pyccg.util.NoParsesError: No successful parses computed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "ecLearner.py", line 542, in <module>
    **args)
  File "/om2/user/zyzzyva/ec/dreamcoder/ec.py", line 130, in explorationCompression
    for r in ecIterator(*arguments, **keywords):
  File "/om2/user/zyzzyva/ec/dreamcoder/ec.py", line 411, in ecIterator
    result=result)
  File "ecLearner.py", line 337, in wake_generative_with_pyccg
    if self.use_pyccg_enum: self._update_pyccg_with_supervised_batch(fallback_frontiers) 
  File "ecLearner.py", line 247, in _update_pyccg_with_supervised_batch
    self.pyccg_learner.update_with_supervision(instruction, model, converted_pyccg)
  File "/om2/user/zyzzyva/pyccg/pyccg/word_learner.py", line 307, in update_with_supervision
    self.lexicon = augment_lexicon_unification(self.lexicon, sentence, self.ontology, lf)
  File "/om2/user/zyzzyva/pyccg/pyccg/lexicon.py", line 843, in augment_lexicon_unification
    candidate_parses = list(candidate_parses)
  File "/om2/user/zyzzyva/pyccg/pyccg/lexicon.py", line 719, in attempt_candidate_parse
    category_sem_arities = lexicon.category_semantic_arities()
  File "/om2/user/zyzzyva/pyccg/pyccg/lexicon.py", line 312, in category_semantic_arities
    ret[category] = set(get_arity(entry.semantics()) for entry in entries)
  File "/om2/user/zyzzyva/pyccg/pyccg/lexicon.py", line 312, in <genexpr>
    ret[category] = set(get_arity(entry.semantics()) for entry in entries)
  File "/om2/user/zyzzyva/pyccg/pyccg/logic.py", line 2667, in get_expr_arity
    return 1 + self.get_expr_arity(expr.term)
  File "/om2/user/zyzzyva/pyccg/pyccg/logic.py", line 2669, in get_expr_arity
    function = self.functions_dict[expr.pred.variable.name]
KeyError: 'z2'

Complete log file at: om2/user/zyzzyva/puddle-world/jobs/pyccg_debug

Steps to replicate: You can run ./launchers/ec_learner test 30 directly from /om2/user/zyzzyva/puddle-world to see the results, and ./launchers/ec_learner is a good example file for running. --mlu_cap $MLU sets the maximum length of the utterances.

[hans]

Isolating these bugs is going to be very tough .. was hoping to get at least the arity checks with https://github.com/hans/pyccg/commit/1b1864810334404ce3bfc07f4074d1ba7f0c89b7 , but it looks like it's more complicated than it appears on the surface.

Once I can get my own environment running (early next week I hope?) I will squash these bugs! Neither looks especially difficult to solve once I can isolate the cause.

[hans]

https://github.com/hans/pyccg/commit/be7dbfe542bbd7799a925c866a0fff69798f11cf may fix the latter arity error, though I still need to run the whole EC flow to verify this.

[hans]

Running the full flow now with debug output so that I can squash the other bug — stay posted!