Error with Factool Installation and Execution

[kentrachmat]

Hello,

I'm experiencing an error after following the installation guidelines for Factool and using the code from example.py. Despite trying both recommended installation methods and ensuring all three API keys are correctly defined, the tool is not functioning as expected. Could you please assist in resolving this issue?

[EthanC111]

it might be related to this issue: https://github.com/GAIR-NLP/factool/issues/33

[kentrachmat]

Hello Ethan,

Thank you for your reply. It works now but when I tried to run response_list, it gave me <coroutine object Factool.run at 0x7edc56c414d0> . Could you please give me more guidance to get the same output as in example?

NOTES:

• I am using your Scientific Literature Review example
• My modified repo: https://github.com/kentrachmat/factool-modified/blob/main/factool/factool.py

Thank you in advance

[EthanC111]

I see. maybe change response_list = factool_instance.run(inputs) to response_list = asyncio.run(factool_instance.run(inputs)). Remember to import asyncio.

Also, for Scientific Literature Review, the scholarly API may have some bugs (please refer to this issue).

[kentrachmat]

Hello,

I am trying to run my prompt but I got a NoneType error, do you know why it might happen? Also, I used await because asyncio.run will give me the same error as before

inputs = [ {
            "prompt": 'Write a systematic survey or overview about the advancements in self-supervised learning for robotic grasping, including the use of large-scale datasets and extensive training hours to improve performance and generalization. Discuss the challenges, successes, and future directions for this approach in the field of robotics and artificial intelligence.',
            "category": "scientific",
            "response": 'Current model free learning-based ......'
        },
] 
response_list = await factool_instance.run(inputs)

print(response_list)```

Thanks

[EthanC111]

I see. Just curious, does this issue happen in other tasks (KBQA, math, code)?

It is possible that the response doesn't contain any valid paper so the claim extraction process extracts nothing. The exception handling process for this part might not be robust.

Also, as I mentioned earlier, there are bugs in the current scholarly code, so you might want to change to this line into self._rows = self._soup.find_all('div', class_='gs_r gs_or gs_scl gs_fmar') + self._soup.find_all('div', class_='gsc_mpat_ttl')

[kentrachmat]

Hello, The issue happened only in the scientific task. The response corresponds to a valid paper's abstract but paraphrased so the claim should have extracted something. However, you're right I will check regarding the exception handling.

Thank you for the guidance.

[EthanC111]

Thanks for the feedback! So I think the current version of FacTool inherently assumes that the given response should include at least one claim (i.e., the response contains the name of at least one paper, no matter it is a hallucinated paper or not.). It is possible that the abstract itself doesn't mention any paper, so no claims will be extracted, causing the NoneType error. Maybe you can try something like:

try:
  response_list = await factool_instance.run(inputs)
except TypeError:
  print("no claims in the response!")