Faster Neural Networks Straight from JPEG: jpeg2dct subroutines

This repository contains source code useful for reproducing results presented in the paper Faster Neural Networks Straight from JPEG (ICLR workshop 2018):

@inproceedings{gueguen_2018_ICLR
  title={Faster Neural Networks Straight from JPEG},
  author={Lionel Gueguen and Alex Sergeev and Ben Kadlec and Rosanne Liu and Jason Yosinski},
  booktitle={International Conference on Learning Representations},
  year={2018}
}

jpeg2dct subroutines

The jpeg2dct library provides native Python functions and a TensorFlow Operators to read the Discrete Cosine Transform coefficients from image encoded in JPEG format. The I/O operation leverages standard JPEG libraries (libjpeg or libjpeg-turbo) to perform the Huffman decoding and obtain the DCT coefficients.

Usage

Read into numpy array

from jpeg2dct.numpy import load, loads

#read from a file
jpeg_file = '/<jpeg2dct dir>/test/data/DCT_16_16.jpg'
dct_y, dct_cb, dct_cr = load(jpeg_file)
print ("Y component DCT shape {} and type {}".format(dct_y.shape, dct_y.dtype))
print ("Cb component DCT shape {} and type {}".format(dct_cb.shape, dct_cb.dtype))
print ("Cr component DCT shape {} and type {}".format(dct_cr.shape, dct_cr.dtype))

#read from in memory buffer
with open(jpeg_file, 'rb') as src:
    buffer = src.read()
dct_y, dct_cb, dct_cr = loads(buffer)

Read into Tensorflow Op

Example 1

import tensorflow as tf
from jpeg2dct.tensorflow import batch_decode

# assemble the graph
batch_size = 4
images_byte_tensor = tf.placeholder(shape=(batch_size,), dtype=tf.string)
dcty_batched, dctc_batched, dctr_batched = batch_decode(images_byte_tensor)

# assemble the data to be fed to the graph
jpeg_file = '/<jpeg2dct dir>/test/data/DCT_16_16.jpg'
with open(jpeg_file) as src:
    byt = src.read()
images_byte_values = [byt for i in range(batch_size)]

with tf.Session() as sess:
    y, c, r = sess.run([dcty_batched, dctc_batched, dctr_batched],
                        feed_dict={images_byte_tensor:images_byte_values})

Example 2

import tensorflow as tf
from jpeg2dct.tensorflow import decode

jpeg_file = '/<jpeg2dct dir>/test/data/DCT_16_16.jpg'
with tf.Session() as sess:
    jpegbytes = tf.read_file(jpeg_file)
    dct_y_tf, dct_c_tf, dct_r_tf = decode(jpegbytes)
    print ("Y component DCT shape {} and type {}".format(dct_y_tf.eval().shape, dct_y_tf.dtype))

Installation

Requirements

Numpy>=1.14.0
libjpeg or libjpeg-turbo
(Optional) Tensorflow>=1.5.0

Pip

pip install jpeg2dct

On macOS 10.13, with default Python, the compiler has troubles. In Conda, the following is unnecessary.

mv /usr/local/include /usr/local/include_old
brew reinstall llvm libjpeg
pip install jpeg2dct

From source

git clone https://github.com/uber-research/jpeg2dct.git
cd jpeg2dct
python setup.py install

On Mac run the following, before python setup.py ...

export MACOSX_DEPLOYMENT_TARGET=10.10
# or
conda install --channel https://conda.anaconda.org/anaconda clangxx_osx-64

Test the installation

python setup.py test
# or
python setup.py develop
pytest

uber-research / jpeg2dct

readme