Allows defining custom pickling method.

Trung0246 commented 3 months ago

Things to check first

[X] I have searched the existing issues and didn't find my feature already requested there

Feature description

Currently for APScheduler 4 it is trivial to implement a custom class to handle serialization. Particular use case would be to support dill and similar library to do things such as lambda serialization. However callable_to_ref check prevented this.

Is it possible to disable this check?

Use case

Allows for custom serialization.

lekhanhtoan37 commented 3 months ago

Bug with https://github.com/confluentinc/confluent-kafka-python

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.11/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/anyio/from_thread.py", line 217, in _call_func
    retval = await retval_or_awaitable
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/_schedulers/async_.py", line 427, in add_schedule
    await self.data_store.add_schedule(schedule, conflict_policy)
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/datastores/sqlalchemy.py", line 496, in add_schedule
    schedule.marshal(self.serializer)
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/_structures.py", line 143, in marshal
    marshalled["args"] = serializer.serialize(self.args)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/serializers/pickle.py", line 29, in serialize
    return dumps(obj, self.protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot pickle 'cimpl.Producer' object

lekhanhtoan37 commented 3 months ago

Any update?

agronholm commented 3 months ago

Bug with https://github.com/confluentinc/confluent-kafka-python

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.11/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/anyio/from_thread.py", line 217, in _call_func
    retval = await retval_or_awaitable
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/_schedulers/async_.py", line 427, in add_schedule
    await self.data_store.add_schedule(schedule, conflict_policy)
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/datastores/sqlalchemy.py", line 496, in add_schedule
    schedule.marshal(self.serializer)
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/_structures.py", line 143, in marshal
    marshalled["args"] = serializer.serialize(self.args)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/toanlk/company-project/36hc/bet/.venv/lib/python3.11/site-packages/apscheduler/serializers/pickle.py", line 29, in serialize
    return dumps(obj, self.protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot pickle 'cimpl.Producer' object

How is this an APScheduler bug if pickle can't serialize an unserializable object?

agronholm commented 2 months ago

So can you explain what the problem is with callable_to_ref()? The traceback you posted offers no hints on that.

agronholm commented 1 month ago

Closing due to lack of responses.

agronholm / apscheduler