Warnings about s3aio

[Kirill888]

When calling dc.load for the first time warnings are printed "Failed to resolve driver datacube.plugins.io.read::s3aio"

import datacube.drivers.readers
datacube.drivers.readers.reader_drivers()

Error is due to missing sharedarray dependency in the s3 drivers or something, but really datacube should not be installed with s3 option.

[omad]

The error is due to boto3 being the wrong version.

s3.load()
---------------------------------------------------------------------------
VersionConflict                           Traceback (most recent call last)
<ipython-input-4-564f3bc316dc> in <module>()
----> 1 s3.load()

/g/data/v10/public/modules/dea-env/20180405/lib/python3.6/site-packages/pkg_resources/__init__.py in load(self, require, *args, **kwargs)
   2321             )
   2322         if require:
-> 2323             self.require(*args, **kwargs)
   2324         return self.resolve()
   2325

/g/data/v10/public/modules/dea-env/20180405/lib/python3.6/site-packages/pkg_resources/__init__.py in require(self, env, installer)
   2344         # requirements for that extra are purely optional and skip over them.
   2345         reqs = self.dist.requires(self.extras)
-> 2346         items = working_set.resolve(reqs, env, installer, extras=self.extras)
   2347         list(map(working_set.add, items))
   2348

/g/data/v10/public/modules/dea-env/20180405/lib/python3.6/site-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
    781                 # Oops, the "best" so far conflicts with a dependency
    782                 dependent_req = required_by[req]
--> 783                 raise VersionConflict(dist, req).with_context(dependent_req)
    784
    785             # push the new requirements onto the stack

VersionConflict: (boto3 1.6.23 (/g/data2/v10/public/modules/dea-env/20180405/lib/python3.6/site-packages), Requirement.parse('boto3==1.4.3; extra == "s3"'))

I'm not sure why the s3 drivers are being installed. I think it must be to do with whichever package format is being used not supporting the requirement restrictions on the entry_points and installing them all by default...

[Kirill888]

but when you get pass that you'll get

import datacube.drivers.s3.driver
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-36341123188e> in <module>()
----> 1 import datacube.drivers.s3.driver

/g/data/v10/public/modules/dea/20180515/lib/python3.6/site-packages/datacube/drivers/s3/driver.py in <module>()
      8 
      9 from datacube.drivers.s3.datasource import S3DataSource
---> 10 from datacube.drivers.s3.storage.s3aio.s3lio import S3LIO
     11 from .utils import DriverUtils
     12 from datacube.utils import DatacubeException

/g/data/v10/public/modules/dea/20180515/lib/python3.6/site-packages/datacube/drivers/s3/storage/s3aio/__init__.py in <module>()
      5 from __future__ import absolute_import
      6 
----> 7 from .s3lio import S3LIO
      8 from .s3aio import S3AIO
      9 from .s3io import S3IO

/g/data/v10/public/modules/dea/20180515/lib/python3.6/site-packages/datacube/drivers/s3/storage/s3aio/s3lio.py in <module>()
      7 from __future__ import absolute_import, division
      8 
----> 9 import SharedArray as sa
     10 import hashlib
     11 import sys

ModuleNotFoundError: No module named 'SharedArray'

[Kirill888]

Is this where datacube is installed from?

https://github.com/GeoscienceAustralia/digitalearthau/blob/2a0bc7db4f8b1cd113b7d06c04b3d7a6c0e1b5c1/modules/dea/requirements.txt#L1

you can append [celery,test,replicas,distributed] to that and see if it helps