Improvements to data provider

[damro]

1. dict of dtypes and shapes as input or via convention out of saved metadata in a pkl file (or whatever); specified one has priority

2. imagelist -> shapedict and apply to all keys, not just "image" keys, or combine sourcedict with shapes and dtypes
3. make a general ParallByFile class independent of a data reader that takes the reader as input (TFRecords provider, HDF5 provider, LMDB provider etc.)
4. add a postprocessing argument to the ParallelByFile base class on strings top level provider on arrays (if specified)
5. instead of separate py_postprocess let user do the work or provide a convenience method that modifies postprocess dict and is called in postprocess_many
6. multiple filename queues that are joined upon after reading one batch
7. allow shuffle filename_queue as option and then allow passing a seed (or default to seed=0)
8. Make old tests run and create new tests
9. Update sonify in base.py to accept tf.dtypes
10. Think about how to restore data reading state
11. Parallelize data provider on attribute.

[nhaber]

On #8, need to think about how to deal with the case where filenames provided do not line up w corresponding data -- unclear whether there is a check.

-check that all filenames lists are the same length -check that corresponding elements have the same number of records

[qbilius]

How is this issue progressing?

I'm thinking of implementing a benchmark test comparing TFRecords / HDF5 reading from the filename queue, varying the number of reading and preprocessing threads and whether data is fed into a placeholder or directly. Seems like it would be generally useful to see how fast each combination is.

[yamins81]

We're nearing completion on this. It turned out to have some nontrivial technical challenges. I should have something by later today or tomorrow.