search

janelia-flyem / dvid

Distributed, Versioned, Image-oriented Dataservice
http://dvid.io
Other
195 stars 33 forks source link

GET /subvolblocks does not respect compression option #347

Closed stuarteberg closed 4 years ago

stuarteberg commented 4 years ago

The documentation for the /subvolblocks endpoint of uint8blk instances includes an option for compression:

Query-string Options:

compression   Allows retrieval of block data in "jpeg" (default) or "uncompressed".

But as far as I can tell, adding compression=uncompressed to my query string makes no difference -- the data comes back in jpeg form anyway.

Here's an example. Notice that the header (which includes the size of the block data) does not change after adding compression=uncompressed to the query string:

In [132]: url = 'http://emdata4:8200/api/node/7415a1c824ee4e3a90da8eceaed9bffd/grayscalejpeg/subvolblocks/64_64_64/22848_11904_20032'

In [133]: r = requests.get(url)

In [134]: np.frombuffer(r.content[:16], dtype=np.int32)
Out[134]: array([  357,   186,   313, 87407], dtype=int32)

In [135]: r = requests.get(url + '?compression=uncompressed')

In [136]: np.frombuffer(r.content[:16], dtype=np.int32)
Out[136]: array([  357,   186,   313, 87407], dtype=int32)

For reference, here's the full documentation for /subvolblocks:

/subvolblocks documentation ``` GET /node///subvolblocks//[?queryopts] Retrieves blocks corresponding to the extents specified by the size and offset. The subvolume request must be block aligned. This is the most server-efficient way of retrieving imagelblk data, where data read from the underlying storage engine is written directly to the HTTP connection. Example: GET /node/3f8c/segmentation/subvolblocks/64_64_64/0_0_0 If block size is 32x32x32, this call retrieves up to 8 blocks where the first potential block is at 0, 0, 0. The returned byte stream has a list of blocks with a leading block coordinate (3 x int32) plus int32 giving the # of bytes in this block, and then the bytes for the value. If blocks are unset within the span, they will not appear in the stream, so the returned data will be equal to or less than spanX blocks worth of data. The returned data format has the following format where int32 is in little endian and the bytes of block data have been compressed in JPEG format. int32 Block 1 coordinate X (Note that this may not be starting block coordinate if it is unset.) int32 Block 1 coordinate Y int32 Block 1 coordinate Z int32 # bytes for first block (N1) byte0 Bytes of block data in jpeg-compressed format. byte1 ... byteN1 int32 Block 2 coordinate X int32 Block 2 coordinate Y int32 Block 2 coordinate Z int32 # bytes for second block (N2) byte0 Bytes of block data in jpeg-compressed format. byte1 ... byteN2 ... If no data is available for given block span, nothing is returned. Arguments: UUID Hexadecimal string with enough characters to uniquely identify a version node. data name Name of data to add. size Size in voxels along each dimension specified in . offset Gives coordinate of first voxel using dimensionality of data. Query-string Options: compression Allows retrieval of block data in "jpeg" (default) or "uncompressed". throttle If "true", makes sure only N compute-intense operation (all API calls that can be throttled) are handled. If the server can't initiate the API call right away, a 503 (Service Unavailable) status code is returned. ```