gallantlab / cottoncandy

sugar for s3
http://gallantlab.github.io/cottoncandy/
BSD 2-Clause "Simplified" License
33 stars 17 forks source link

copy array to have it's own data if it doesn't already with upload vi… #12

Closed r-b-g-b closed 8 years ago

r-b-g-b commented 8 years ago

…a dict2cloud When uploading a dict containing an array arr with arr.flags.owndata=False (like if arr is a slice of a larger array) I got an error


AttributeError Traceback (most recent call last)

in () ----> 1 cci.dict2cloud('tmp2', {'x':np.random.randn(10,10)[::3,::2]}) /auto/k1/robertg/code/cottoncandy/cottoncandy/utils.pyc in iremove_root(self, object_name, _args, *_kwargs) 128 elif object_name[0] == SEPARATOR: 129 object_name = object_name[1:] --> 130 return input_function(self, object_name, _args, *_kwargs) 131 return iremove_root 132 /auto/k1/robertg/code/cottoncandy/cottoncandy/interfaces.py in dict2cloud(self, object_name, array_dict, acl, verbose, *_metadata) 618 elif isinstance(v, np.ndarray): 619 # v = v if v.flags.owndata else np.array(v) --> 620 _ = self.upload_raw_array(name, v, acl=acl, *_metadata) 621 else: # try converting to array 622 _ = self.upload_raw_array(name, np.asarray(v), acl=acl) /auto/k1/robertg/code/cottoncandy/cottoncandy/utils.pyc in iremove_root(self, object_name, _args, *_kwargs) 128 elif object_name[0] == SEPARATOR: 129 object_name = object_name[1:] --> 130 return input_function(self, object_name, _args, *_kwargs) 131 return iremove_root 132 /auto/k1/robertg/code/cottoncandy/cottoncandy/interfaces.py in upload_raw_array(self, object_name, array, gzip, acl, **metadata) 543 zipdata = StringIO() 544 gz = GzipFile(mode='wb', fileobj=zipdata) --> 545 gz.write(array.data) 546 gz.close() 547 zipdata.seek(0) AttributeError: cannot get single-segment buffer for discontiguous array

This fix eliminated that error, and I tested that the data is reliably saved to the destination.