Closed Webifi closed 1 year ago
(Something went very strange when I tried to pull the merge from your remote branch, hence all the repeated changes.)
@Webifi Great job! I'm gonna test it just in case and merge it soon.
@borzunov Anything terribly wrong with this PR?
@Webifi Sorry for the slow response, I'm on vacation right now :)
Are you okay with removing steps
from the response? I feel like this field is a bit confusing, since people may think that it's the overall number of steps from the beginning of the session, etc.
Other than that, I think it's good to go!
Yeah, I agree. I questioned if I should have even put steps in.
I'm having issues pushing changes to my fork again. I'll try to get this sorted out when I can find some time.
My chat.petals.dev colab isn't working any longer for some reason, even without these changes applied, so I can't fully test the latest change I just pushed until I'm back in the office in a few weeks.
@Webifi Seems to work, merging it. Thanks for your contribution!
Added
token_count
of delta for each respose, since if max_new_tokens > 1 steps would be be more difficult to use for speed calculation.Fixes #33.