mvantellingen / python-zeep

A Python SOAP client
http://docs.python-zeep.org
Other
1.88k stars 586 forks source link

multiprocessing PicklingError #421

Closed iamtennislover closed 7 years ago

iamtennislover commented 7 years ago

Can we run zeep with multiprocessing? I am getting PickleError

  1. The version of zeep '0.23.0'
  2. Python: 2.7
  3. The WSDL you are using: https://login.swiftkanban.com/axis2/services/KanbanCardService?wsdl
  4. An example script
    ############### passing zeep client object into each process ###############
    import multiprocessing

    from zeep import Client, Transport
    from zeep.cache import InMemoryCache
    from zeep.wsse.username import UsernameToken
    from pyodinhttp import odin_retrieve_pair

    def _setup_connection(wsdl_url, username, password):
        return Client(wsdl=wsdl_url,
                                   wsse=UsernameToken(username, password),
                                   transport=Transport(verify=False, cache=InMemoryCache()))

    def make_request(projectId, soap_client):
        # make api call and get some output (this is just a dummy call)
        lanes = str(soap_client.service.getSmartLaneDetails(projectId=projectId))
        return {'lanes': lanes}

    def main():
        wsdl_url = 'https://login.swiftkanban.com/axis2/services/KanbanCardService?wsdl'
        soap_client = _setup_connection(wsdl_url, 'uname', 'pwd')
        pool = multiprocessing.Pool(multiprocessing.cpu_count())
        jobs = [(3, soap_client), (9, soap_client)]
        for i in pool.map(make_request, jobs):
            print i

    main()

    Traceback (most recent call last):
      File "/tmp/zeep_with_mp.py", line 31, in <module>
        main()
      File "/tmp/zeep_with_mp.py", line 28, in main
        for i in pool.map(make_request, jobs):
      File "/.../lib/python2.7/multiprocessing/pool.py", line 251, in map
        return self.map_async(func, iterable, chunksize).get()
      File "/.../lib/python2.7/multiprocessing/pool.py", line 558, in get
        raise self._value
    cPickle.PicklingError: Can't pickle <class 'zeep.xsd.dynamic_types.moveMultipleCardToReadyInput'>: import of module zeep.xsd.dynamic_types failed
mvantellingen commented 7 years ago

This seems to be a duplicate of #417