Test data run crashes

[enormandeau]

I followed the exact installation instructions (including the new conda activate popvae part) and then tested popvae with the test data with:

popvae.py --infile data/pabu/pabu_test_genotypes.vcf --out out/pabu_test --seed 42

And got the following run and error message:

Using TensorFlow backend.
WARNING:tensorflow:From /home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/popvae-0.1-py3.7.egg/EGG-INFO/scripts/popvae.py:126: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

loading genotypes
[read_vcf] 7386 rows in 0.07s; chunk in 0.07s (113138 rows/s); 1685:69
[read_vcf] all done (113048 rows/s)
counting alleles
dropping non-biallelic sites
dropping singletons
filling missing data with rbinom(2,derived_allele_frequency)
100%|█████████████████████████████████████████████████████████████| 95/95 [00:00<00:00, 8151.78it/s]
running train/test splits
['validation samples:ARK_CAH156' 'validation samples:GUA_DHB4507'
 'validation samples:KNS_SAR7995' 'validation samples:LOU_CAH140'
 'validation samples:OAX_DHB5582' 'validation samples:SCL_PB65556'
 'validation samples:SIN_CSW7732' 'validation samples:TEX_CAH087'
 'validation samples:TEX_CAH149' 'validation samples:YUC_BRB875']
running on 3887 SNPs
WARNING:tensorflow:From /home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/tensorflow-1.15.2-py3.7-linux-x86_64.egg/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/tensorflow-1.15.2-py3.7-linux-x86_64.egg/tensorflow_core/python/ops/nn_impl.py:183: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/training_utils.py:819: UserWarning: Output decoder missing from loss dictionary. We assume this was done on purpose. The fit and evaluate APIs will not be expecting any data to be passed to decoder.
  'be expecting any data to be passed to {0}.'.format(name))
2020-10-21 10:25:19.612072: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-21 10:25:19.649267: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2195055000 Hz
2020-10-21 10:25:19.652350: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5570827b5d90 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-21 10:25:19.652387: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Train on 85 samples, validate on 10 samples
Epoch 1/500
85/85 [==============================] - 1s 8ms/step - loss: 2664.8421 - val_loss: 2437.6572

Epoch 00001: val_loss improved from inf to 2437.65723, saving model to out/pabu_test_weights.hdf5
Traceback (most recent call last):
  File "/home/eric/miniconda3/envs/popvae/bin/popvae.py", line 4, in <module>
    __import__('pkg_resources').run_script('popvae==0.1', 'popvae.py')
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/pkg_resources/__init__.py", line 650, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1446, in run_script
    exec(code, namespace, namespace)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/popvae-0.1-py3.7.egg/EGG-INFO/scripts/popvae.py", line 415, in <module>
    batch_size=batch_size)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/training.py", line 1239, in fit
    validation_freq=validation_freq)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/training_arrays.py", line 216, in fit_loop
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/callbacks/callbacks.py", line 152, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/callbacks/callbacks.py", line 719, in on_epoch_end
    self.model.save(filepath, overwrite=True)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/network.py", line 1152, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/saving.py", line 449, in save_wrapper
    save_function(obj, filepath, overwrite, *args, **kwargs)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/saving.py", line 541, in save_model
    _serialize_model(model, h5dict, include_optimizer)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/engine/saving.py", line 161, in _serialize_model
    layer_group[name] = val
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/Keras-2.3.1-py3.7.egg/keras/utils/io_utils.py", line 233, in __setitem__
    dataset = self.data.create_dataset(attr, val.shape, dtype=val.dtype)
  File "/home/eric/miniconda3/envs/popvae/lib/python3.7/site-packages/h5py-3.0.0rc1-py3.7-linux-x86_64.egg/h5py/_hl/group.py", line 143, in create_dataset
    if '/' in name:
TypeError: a bytes-like object is required, not 'str'

[cjbattey]

Thanks for reporting this. H5py had been updated since I last tested things, and the latest version doesn't work with some of the tensorflow1.15 callbacks I'm using. Either cloning the latest version and rerunning setup.py, or running pip install h5py==2.10.0 in your popvae conda environment should fix it.

[enormandeau]

Thank you. I could make the test work after removing my popvar conda environment and reinstalling it from a new cloned repo.