umr-lops / xsar

Synthetic Aperture Radar (SAR) Level-1 GRD python mapper for efficient xarray/dask based processing
https://cyclobs.ifremer.fr/static/sarwing_datarmor/xsar/
MIT License
24 stars 8 forks source link

When using dask distributed : AttributeError: type object 'Sentinel1Meta' has no attribute 'reader' #176

Open Skealz opened 11 months ago

Skealz commented 11 months ago

I followed this guide : https://cyclobs.ifremer.fr/static/sarwing_datarmor/xsar/examples/xsar_batch_datarmor.html but without using a notebook. (basically, using : res = ddf_l1.map_partitions(batch_processing, meta=('foo', str)) res.persist() )

I end up getting this error : AttributeError: type object 'Sentinel1Meta' has no attribute 'reader'

It seems that the BlockingActorProxy() class doesn't work correctly. In BlockingActorProxy init function, when I change a part of the code to :

        if False: #self._dask_client is not None:
            logger.debug('submit new actor')
            self._actor_future = self._dask_client.submit(self._cls, *args, **kwargs, actors=True)
            self._actor = self._actor_future.result()
        elif self._actor is None:
            # transparent proxy: no future
            self._actor = self._cls(*args, **kwargs)

I no longer get the error and the code works as expected.

agrouaze commented 1 month ago

I have to dig in the code to understood why we have this BlockingActorProxy. If it appears that it is no more needed I would be happy to remove it. @Skealz Do you want to propose a pull request?