error when pickle ServiceBusReceivedMessage object

[IsmaelJS]

• Package Name: azure-servicebus
• Package Version: 7.8.1
• Operating System: alpine 3.12
• Python Version: 3.8.6

Describe the bug There is an error when trying to pickle the ServiceBusReceivedMessage object. The "prefork" pool (multiprocessing) in celery uses pickle for serialization, and from v5.2 it is not working, see details in https://github.com/celery/celery/issues/7237

To Reproduce It can be reproduced by using the following code:

import os
import json
import pickle
from azure.servicebus import ServiceBusClient, ServiceBusMessage

CONNECTION_STR = "XXX"
QUEUE_NAME = "XXX"

msg = {
   "body":"W1tdLCB7ImlucHV0X2lkIjogNH0sIHsiY2FsbGJhY2tzIjogbnVsbCwgImVycmJhY2tzIjogbnVsbCwgImNoYWluIjogbnVsbCwgImNob3JkIjogbnVsbH1d",
   "content-encoding":"utf-8",
   "content-type":"application/json",
   "headers":{
      "lang":"py",
      "task":"tasks.example_task",
      "id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "shadow":None,
      "eta":"2021-10-07T02:30:23.764066+00:00",
      "expires":None,
      "group":None,
      "group_index":None,
      "retries":1,
      "timelimit":[
         None,
         None
      ],
      "root_id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "parent_id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "argsrepr":"()",
      "kwargsrepr":"{'input_id': 4}",
      "origin":"gen36@94713e01a9c0",
      "ignore_result":1,
      "x_correlator":"44a1978d-c869-4173-afe4-da741f0edfb9"
   },
   "properties":{
      "correlation_id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "reply_to":"7b9a3672-2fed-3e9b-8bfd-23ae2397d9ad",
      "origin":"gen68@c33d4eef123a",
      "delivery_mode":2,
      "delivery_info":{
         "exchange":"",
         "routing_key":"celery_task_queue"
      },
      "priority":0,
      "body_encoding":"base64",
      "delivery_tag":"dc83ddb6-8cdc-4413-b88a-06c56cbde90d"
   }
}

servicebus_client = ServiceBusClient.from_connection_string(conn_str="XXX")

with servicebus_client:
    with servicebus_client.get_queue_sender(QUEUE_NAME) as sender:
        # Sending a single message
        single_message = ServiceBusMessage(json.dumps(msg))
        sender.send_messages(single_message)

with servicebus_client:
    receiver = servicebus_client.get_queue_receiver(queue_name=QUEUE_NAME)
    with receiver:
        received_msgs = receiver.receive_messages(max_message_count=10, max_wait_time=5)
        for msg in received_msgs:
            print(str(msg))
            pickle.dumps(msg)
            receiver.complete_message(msg)

print("Receive is done.")

The result is:

Traceback (most recent call last):
  File "test_pickle.py", line 40, in <module>
    pickle.dumps(msg)
  File "stringsource", line 2, in uamqp.c_uamqp.CompositeValue.__reduce_cython__
TypeError: no default __reduce__ due to non-trivial __cinit__

Expected behavior It should works.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context There are several bugs opened in celery and billiard which might be related to this issue: https://github.com/celery/celery/issues/7237 https://github.com/celery/billiard/issues/356

[kashifkhan]

Thank you the feedback @IsmaelJS. We will investigate and get back to you asap

[swathipil]

Hi @IsmaelJS - I've just tried running the provided sample and am not seeing this error. We've had pickling issues in the past due to some objects in the underlying uamqp library being incompatible. This has been fixed in uamqp>=1.3.0.

Would you be able to provide the uamqp package version that you're using? Thanks!

[IsmaelJS]

Hi @swathipil , I have updated the serialization error because the old one corresponds to azure-servicebus==7.9.0a1.

The error raised is the following:

Traceback (most recent call last):
  File "test_pickle.py", line 40, in <module>
    pickle.dumps(msg)
  File "stringsource", line 2, in uamqp.c_uamqp.CompositeValue.__reduce_cython__
TypeError: no default __reduce__ due to non-trivial __cinit__

The lib version are:

azure-servicebus==7.8.1
uamqp==1.6.3

Thank you so much.

[swathipil]

Hi @IsmaelJS - Can you provide an example of data sent here that results in this error?

I believe I'm not able to reproduce using this sample b/c the data I'm sending and receiving doesn't result in the creation of the uamqp.c_uamqp.CompositeValue object and is therefore picklable.

[IsmaelJS]

Hi @swathipil , I have updated the code within "To Reproduce" section with the data used. Thank you so much.

[suranshu-engg-segg]

is there any update about it ?

[swathipil]

Hi @IsmaelJS - Thanks for the update and your patience! We were able to reproduce this issue.

Releasing uamqp with the bug fix for this will take some time. However, we are currently directing our efforts toward moving azure-servicebus away from uamqp and to an internal Pure Python-based AMQP stack. Pickling should no longer be an issue once we've released this Pure Python-based azure-servicebus. We plan to release a beta of this in the upcoming month and are working steadily towards a stable release.

We will keep you updated as we move forward!

[kashifkhan]

Hi @IsmaelJS , @suranshu-engg-segg we will be releasing a new version of the service bus client this week that uses the pure python based AMQP library which should address the pickling issues. Ill update this issue once its on pypi

[swathipil]

Hi @IsmaelJS, @suranshu-engg-segg - We just released azure-servicebus 7.10.0, which includes the fix for pickling ServiceBusReceivedMessage. https://pypi.org/project/azure-servicebus/7.10.0/

Thanks!

[github-actions[bot]]

Hi @IsmaelJS. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.