add escape hatch for custom JSON serialization

[vlindhol]

Summary

As reported in multiple issues (#1841, #1821, #1787, #1749), it was a surprising breaking change when some Python types became wrapped in a { "__type__": ..., "__value__": ...} envelope in 5.3.0 and subsequently deserialized back into the native types.

Using kombu.util.json.register_type() one could at least avoid e.g. datetime becoming deserialized into a datetime and keep it as a str as in <5.3.0, but it's currently impossible to get rid of the envelope itself.

This PR changes register_type() to accept marker=None, which signals that the JSON encoder should only transform the value using the encoder arg, but should not stick the result in an envelope.

A side-effect of this is that the decoder thus becomes unnecessary to specify (since there is no __type__ in the resulting JSON that would activate it). If decoder is omitted, default to a simple identity function.

This way one can replicate the behavior of pre-5.3.0 fairly well by overriding the default-registered types as follows:

def _serialize_datelike(datetime_or_date):
    if not isinstance(datetime_or_date, datetime.datetime):
        datetime_or_date = datetime.datetime(
            datetime_or_date.year,
            datetime_or_date.month,
            datetime_or_date.day,
            0,
            0,
            0,
            0,
        )
    r = datetime_or_date.isoformat()
    if r.endswith("+00:00"):
        r = r[:-6] + "Z"
    return r

register_type(datetime.datetime, None, _serialize_datelike)
register_type(datetime.date, None, _serialize_datelike)
register_type(datetime.time, None, lambda o: o.isoformat())
register_type(uuid.UUID, None, lambda o: str(o))
register_type(Decimal, None, lambda o: str(o))

Tests

Added some unit tests around register_type(), which were completely missing before. Also tests the new functionality.

[Nusnus]

@vlindhol see lint errors please

[vlindhol]

https://github.com/celery/kombu/actions/runs/8262054534/job/22602777695#step:7:1

Sorry, didn't see that one! I thought CI looked green 😅 Manually ran all CI commands in ci.yaml now, and pushed fixes.

[Nusnus]

Thank you!