search

dusty-nv / NanoLLM

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
https://dusty-nv.github.io/NanoLLM/
MIT License
196 stars 31 forks source link

AttributeError: type object 'ChatModel' has no attribute 'OutputEmbed' #18

Closed TadayukiOkada closed 5 months ago

TadayukiOkada commented 5 months ago

I built nano_llm docker image from the latest jetson-containers. When I run Agent Studio, I get the error in the title. It seems OutputEmbed should be replaced with OutputHistory(?):

https://github.com/dusty-nv/NanoLLM/blob/0d04e08892dd24c98470821aaaf361df8450e507/nano_llm/plugins/llm/chat_model.py#L212

Traceback (most recent call last):
  File "/opt/NanoLLM/nano_llm/plugin.py", line 254, in run
    self.dispatch(input, **kwargs)
  File "/opt/NanoLLM/nano_llm/plugin.py", line 271, in dispatch
    outputs = self.process(input, **kwargs)
  File "/opt/NanoLLM/nano_llm/plugins/llm/chat_model.py", line 163, in process
    self.process(x, **kwargs)
  File "/opt/NanoLLM/nano_llm/plugins/llm/chat_model.py", line 212, in process
    self.output(chat_history.image_embedding, ChatModel.OutputEmbed)
AttributeError: type object 'ChatModel' has no attribute 'OutputEmbed'
TadayukiOkada commented 5 months ago

I guess it's not the right fix to simply replace it with OutputHistory.

dusty-nv commented 5 months ago

Impressed you dug in and built it Tadayuki! I am getting some final things buttoned up here (like the function calling) before doing an initial beta release of Agent Studio this week. I think this OutputEmbeds thing is fixed in my local branch, I will push the fixes tonight 👍

From: TadayukiOkada @.> Sent: Sunday, June 16, 2024 11:21:03 AM To: dusty-nv/NanoLLM @.> Cc: Subscribed @.***> Subject: Re: [dusty-nv/NanoLLM] AttributeError: type object 'ChatModel' has no attribute 'OutputEmbed' (Issue #18)

I guess it's not the right fix to simply replace it with OutputHistory.

— Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/NanoLLM/issues/18#issuecomment-2171740554, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK7Z76KJHXWBYWMNYYTZHWUN7AVCNFSM6AAAAABJMMFIY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZRG42DANJVGQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

TadayukiOkada commented 5 months ago

checked the code and I see it's gone. thanks.

Impressed you dug in and built it Tadayuki!

I just built it with 'jetson-containers build ...', it's so easy. :) I had some issues with Agent Studio when I used the image I built incrementally using some docker images built previously. so I deleted all the images and built it from scratch then it worked mostly fine. (sorry I can't tell you what was wrong with the incremental build.) I'm building the new one now. I'll let you know if I see any issue.

dusty-nv commented 5 months ago

Thanks @TadayukiOkada ! I rebuilt and pushed dustynv/nano_llm:r36.2.0 container image from the latest source, which now includes Agent Studio. For others who may read this, you can run that like python3 -m nano_llm.studio (will add docs for it this week)

TadayukiOkada commented 5 months ago

@dusty-nv great! I tried to build it from scratch again, but it seems to fail due to NumPy 2.0 release. I might look into it later, but I'll use your image for now.

dusty-nv commented 5 months ago

Yes, numpy 2.0 dropped yesterday and blew up the builds lol...

Pinned numpy<2 in https://github.com/dusty-nv/jetson-containers/commit/b19854a9f02ad4e2658679475b5c3c9fec41a375