search

fal-ai / fal

⚡ Fastest way to serve open source ML models to millions
https://fal.ai/docs
Apache License 2.0
549 stars 47 forks source link

feat: replace dill with cloudpickle #155

Closed efiop closed 7 months ago

efiop commented 7 months ago

This removes a lot of headache when working on fal, as you don't have to face obscure serialization errors and put @mainify everywhere, as cloudpickle provides a proper mechanism to tell it to serialize things from a particular module by value. This also results in exceptions and values returned by the user code to be loaded by reference, which allows for a normal workflow instead of trying to match class names.

Though there are some things that dill can serialize out of the box which cloudpickle can't and vice versa, as this is still serialization that we are talking about, so I had to adapt a few dill's picklers for misc types. There might be more (including other problems), but I'll solve them as they arise.

Overall everything should keep working as it was, with one exception of nested isolated functions, see xfail in tests.

Note that our prod workers already come with cloudpickle as well as dill, so you can use this already.

Fixes https://github.com/fal-ai/fal/issues/142 Pre-requisite for https://github.com/fal-ai/fal/issues/29

linear[bot] commented 7 months ago

FEA-2189 migrate to cloudpickle from dill

efiop commented 7 months ago

For the record: app tests are failing with 503 for unrelated reasons.

squat commented 7 months ago

For the record: app tests are failing with 503 for unrelated reasons.

That's what they all say!!!!!

efiop commented 7 months ago

Great points! I also went to try models from the registry, but then decided to start with trying to migrate to pydantic v2 just to see how it will go IRL https://github.com/fal-ai/fal/pull/167 , which resulted in me working more on pickling misc locks and stuff that were getting in the way in those two xfail's that I have here (was hoping to get away without , which means that I might fix them along the way.

Marked as WIP for now again.

efiop commented 7 months ago

or the obscure ones that we suspect like creative upscaler & fast sdxl etc) (at least doign fal fn run, not re-serving them

Tried them and it worked no problem. Would love to test out all in registry, but it it super labour intensive and slow, I'm trying to help out with fal.yaml to have a spec on serving all apps in our registry. Pydantic v2 was also working out based on initial test, there will be a PR for v1&v2 support separately after this one.