I was able to get LLAMA to run on LMQL, however it is unable to answer multiple choice questions where we constrain the answer choices if the answer choice it selects is two words. This is a problem since for sentences with multiple fill in the blanks the answer is encoded as a string with multiple words and spaces between them. I asked on the LMQL discord for help regarding the issue, and it seems like it is a known problem.
I plan to submit the github issue on lmql github tomorrow morning.
I was able to get LLAMA to run on LMQL, however it is unable to answer multiple choice questions where we constrain the answer choices if the answer choice it selects is two words. This is a problem since for sentences with multiple fill in the blanks the answer is encoded as a string with multiple words and spaces between them. I asked on the LMQL discord for help regarding the issue, and it seems like it is a known problem.
I plan to submit the github issue on lmql github tomorrow morning.