Currently binary questions do not answer yes or no, but a word. For example if question was: "Was King Khufu one of the Pharaohs that built the Giza pyramids?" and sentence was: "Sneferu was succeeded by his son, Khufu (2589–2566 BC), who built the Great Pyramid of Giza." BERT returns "Khufu" when it should be "yes". Also if the sentence was instead "Sneferu was succeeded by his son, Khufu (2589–2566 BC), who had not built the Great Pyramid of Giza." it returns "not".
Perhaps to fix this we can check if BERT returns [CLS], if it does not and actually returns a word, we can scan sentence for negative words and return yes or no from there. Further testing is required to see if this idea works in practice.
Currently binary questions do not answer yes or no, but a word. For example if question was: "Was King Khufu one of the Pharaohs that built the Giza pyramids?" and sentence was: "Sneferu was succeeded by his son, Khufu (2589–2566 BC), who built the Great Pyramid of Giza." BERT returns "Khufu" when it should be "yes". Also if the sentence was instead "Sneferu was succeeded by his son, Khufu (2589–2566 BC), who had not built the Great Pyramid of Giza." it returns "not".
Perhaps to fix this we can check if BERT returns [CLS], if it does not and actually returns a word, we can scan sentence for negative words and return yes or no from there. Further testing is required to see if this idea works in practice.