Lightning-AI / LitServe

Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.
https://lightning.ai/docs/litserve
Apache License 2.0
2.5k stars 158 forks source link

fix(Logger): handle unpickleable Logger #319

Closed aniketmaurya closed 1 month ago

aniketmaurya commented 1 month ago
Before submitting - [ ] Was this discussed/agreed via a Github issue? (no need for typos and docs improvements) - [ ] Did you read the [contributor guideline](https://github.com/Lightning-AI/pytorch-lightning/blob/main/.github/CONTRIBUTING.md), Pull Request section? - [ ] Did you make sure to update the docs? - [ ] Did you write any new necessary tests?

What does this PR do?

Unpickleable Loggers throw error. This PR handles such objects by recreating the unpickleable objects. https://github.com/Lightning-AI/LitServe/pull/284#issuecomment-2387499979

Output before and after this PR for a non-pickleable Logger: ```python import threading import litserve as ls class NonPicklableLogger(ls.Logger): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._lock = threading.Lock() # Non-picklable resource def process(self, key, value): with self._lock: print(f"Logged {key}: {value}") class TestAPI(ls.test_examples.SimpleLitAPI): def predict(self, x): self.log("predict", x) return super().predict(x) if __name__ == '__main__': lit_api = TestAPI() server = ls.LitServer(lit_api, loggers=NonPicklableLogger()) server.run() ```

Before

  File "/Users/aniket/miniconda3/envs/am/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/aniket/miniconda3/envs/am/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/aniket/miniconda3/envs/am/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object

After

Setup complete for worker 0.

PR review

Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95%. Comparing base (86e00bc) to head (73ffde5). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #319 +/- ## =================================== Coverage 95% 95% =================================== Files 21 21 Lines 1323 1343 +20 =================================== + Hits 1259 1279 +20 Misses 64 64 ```