gallantlab / cottoncandy

sugar for s3
http://gallantlab.github.io/cottoncandy/
BSD 2-Clause "Simplified" License
33 stars 17 forks source link

Fix encryption for Python <= 3.7 #93

Open kroq-gar78 opened 4 days ago

kroq-gar78 commented 4 days ago

While working on #91 I was unable to upload or download anything with an encrypted interface. This code should at least allow the test code below to run.

Given encryption has been broken I'm not sure if anyone has actually used it recently. This might break compatibility with existing encrypted objects, so I would appreciate if someone could test if you could read existing objects.

Note that these changes are only for Python <= 3.7, since PyCrypto depends on time.clock , which was removed in Python 3.8. Both this PR and #91 are necessary for supporting >= 3.8.

Test code

I also did not see any automated tests for this, so here's a short snippet that should work:

import os; import cottoncandy as cc; import numpy as np
cci = cc.get_encrypted_interface(os.environ['DL_BUCKET'], verbose=False, ACCESS_KEY=os.environ['DL_ACCESS_KEY'], SECRET_KEY=os.environ['DL_SECRET_KEY'], endpoint_url=os.environ['DL_URL'])
a = np.arange(10)
cci.upload_raw_array('test_array', a)
assert np.allclose(a, cci.download_raw_array(a))

I can enable all existing tests to also run for encrypted interfaces, but this doubles testing time so I haven't included it here (but I can if that's not an issue).