search

lastmile-ai / aiconfig

AIConfig is a config-based framework to build generative AI applications.
https://aiconfig.lastmileai.dev
MIT License
945 stars 79 forks source link

Move `should_stream` logic into a helper function #861

Open rossdanlm opened 9 months ago

rossdanlm commented 9 months ago

Comment from https://github.com/lastmile-ai/aiconfig/pull/851#discussion_r1447487539

Make this into a separate helper function making it super clear what the two checks are:

  1. Check that inference options is set to true when calling config.run() --> ie: config.run(prompt_name, options=InferenceOptions(stream=True, stream_callback=my_stream_callback)
  2. Check that the stream option in the completion params (which we get from prompt settings) is not explicitly set to false

Testing

You will first need to pip install to this local directory (go to /aiconfig repo and type pip install -e ., and then when you do pip list | grep aiconfig make sure that whatever python-aiconfig links to your local path) Please follow instructions similar to https://github.com/lastmile-ai/aiconfig/pull/851 to test

deepanshu-byte commented 9 months ago

Hi @rossdanlm, I'm interested in tackling this issue. I have a couple of questions:

  1. Where would you recommend defining the new function, and what should be its scope within the project?
  2. Are there specific test cases expected for the new function I'll be creating?
rossdanlm commented 7 months ago

Hey @deepanshu-byte , I'm super sorry for the late reply, I just missed this and going through all the tasks now. Pls feel free to message me on "Rossdan Craig" on messenger if I miss any messages from you.

  1. Where would you recommend defining the new function, and what should be its scope within the project?

Great question! It's acceptable to simply make a utils file within the file itself, but personally I would prefer having this defined in the base class of "ModelParser" itself (put it around here: https://github.com/lastmile-ai/aiconfig/blob/main/python/src/aiconfig/model_parser.py#L198) and then we simply call self.should_stream(options, completion_params)

  1. Are there specific test cases expected for the new function I'll be creating?

I think simply manually testing the flow that I outlined in https://github.com/lastmile-ai/aiconfig/pull/851 should be sufficient! However please note that you will need to run pip3 install -r <filename> where filename is the path to this file: https://github.com/lastmile-ai/aiconfig/blob/main/extensions/HuggingFace/python/requirements.txt, and also that we've made a few modifications to the editor since then so you should follow the test plan from here instead: https://github.com/lastmile-ai/aiconfig/pull/1245 (make sure to run yarn && yarn build from the aiconfig/python/src/editor/client file first!)

One final note is that I'd like us to update EVERY relevant callsite that checks for this! There are a lot, so we can do this in a followup PR after the first refactoring is complete!

Once again I'm super sorry for the late reply. If you became busy since then no worries, pls feel free to message me on Messenger if you have any other questions!