set model_snapshot_path to None to prevent unbound local error

johnathanchiu commented 1 month ago

Description

Adding a variable model_snapshot_path to None should resolve the error introduced in the following issue.

Fixes #(3335)

Type of change

Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced. Please also list any relevant details for your test configuration.

[ ] Test A Logs for Test A
[ ] Test B Logs for Test B

Checklist:

[x] Did you have fun?
[ ] Have you added tests that prove your fix is effective or that this feature works?
[x] Has code been commented, particularly in hard-to-understand areas?
[ ] Have you made corresponding changes to the documentation?

agunapal commented 1 month ago

Thanks @johnathanchiu If you could please refer to my PR and make this pythonic and try out with llama 3.2 and update the README, that would be great. I can approve yours and close mine

johnathanchiu commented 1 month ago

just made the requested changes! I I also tested locally as well. It's working.

pytorch / serve