AttributeError: 'NoneType' object has no attribute 'config'

[Austin-Routt]

Hi,

Thank you for your work on this project. I really need Guidance support.

I'm having issues with loading your extension, I installed the requirements, but I'm getting the following error:

2023-09-03 13:24:32 INFO:Loading the extension "guidance_api"...
starting guidance server
2023-09-03 13:24:32 ERROR:Failed to load the extension "guidance_api".
Traceback (most recent call last):
  File "E:\Apps\oobabooga_windows\oobabooga_windows\text-generation-webui\modules\extensions.py", line 40, in load_extensions
    extension.setup()
  File "E:\Apps\oobabooga_windows\oobabooga_windows\text-generation-webui\extensions\guidance_api\script.py", line 134, in setup
    gen =GuidanceGenerator()
  File "E:\Apps\oobabooga_windows\oobabooga_windows\text-generation-webui\extensions\guidance_api\guidance_gen.py", line 36, in __init__
    self.data= setup_model_data()
  File "E:\Apps\oobabooga_windows\oobabooga_windows\text-generation-webui\extensions\guidance_api\model_info.py", line 13, in setup_model_data
    config_dict = vars(shared.model.config)
AttributeError: 'NoneType' object has no attribute 'config'

[Austin-Routt]

I solved this by disabling all other extensions. I believe the conflict is with the gallery extension.

That being said, I still can't use guidance. I tried to run the example code, but all I get is:

Traceback (most recent call last):
  File "C:\Users\polym\AppData\Local\Temp\temp_1693770167340.py", line 9, in <module>
    guidance.llm = guidance.llms.TGWUI("http://127.0.0.1:9555")
AttributeError: module 'guidance.llms' has no attribute 'TGWUI'

[danikhan632]

thanks you for finding that,

Already have an update just need to push

[Austin-Routt]

Great!

I've been trying to whittle through all the issues.

I managed to solve AttributeError: module 'guidance.llms' has no attribute 'TGWUI' by pip uninstall guidance and then installing your unmerged version via pip install git+https://github.com/danikhan632/guidance.

After that, I got the KeyError: 'results'. I then used the following code to examine the API response:

import requests

def test_api_response(api_url):
    try:
        response = requests.get(api_url)
        if response.status_code == 200:
            print("API Response Status: 200 OK")
            print("API Response JSON:")
            print(response.json())
        else:
            print(f"API Request failed with status code {response.status_code}")
    except requests.exceptions.RequestException as e:
        print(f"An error occurred during the API request: {e}")

if __name__ == "__main__":
    # Replace 'http://localhost:5000/api/v1/model' with your actual API endpoint URL
    api_url = 'http://localhost:5000/api/v1/model'

    print(f"Testing API Response for URL: {api_url}\n")
    test_api_response(api_url)

That gave me this output:

Testing API Response for URL: http://localhost:5000/api/v1/model

API Response Status: 200 OK
API Response JSON:
{'result': '13b'}

So, in your guidance code, I changed the results key to result. However, this doesn't work because you are expecting a dictionary to be returned instead of just a string for the model name:

Traceback (most recent call last):
  File "C:\Users\polym\AppData\Local\Temp\temp_1693772917767.py", line 15, in <module>
    guidance.llm = guidance.llms.TGWUI(URI)
  File "C:\Users\polym\AppData\Local\Programs\Python\Python39\lib\site-packages\guidance\llms\_tgwui.py", line 26, in __init__
    self.model_name = self.model_info["model_name"]
TypeError: string indices must be integers

Does your update fix this as well? If it doesn't can I help you, because I really need to get something working relatively soon?

[dazpants1]

Hi, I was wondering if this issue had been resolved as i am seeing the same error message.