AttributeError: 'str' object has no attribute 'decode' error in bpnet contrib

[np519]

Hello, I've been having some issues trying to get bpnet to run on an aws ec2 instance.

After installing and setting up bpnet using the conda-env.yml file, I started following the google colab tutorial using the example chip-nexus data.

I ran bpnet using the following command:

bpnet train /home/ec2-user/bpnet/examples/chip-nexus/dataspec.yml --premade=bpnet9 --config=/home/ec2-user/bpnet/examples/chip-nexus/config.gin . --override='train.epochs=10' --in-memory

This worked fine so I attempted to calculate contributions scores with the following command:

bpnet contrib /home/ec2-user/bpnet_output_dir/ --method=deeplift --contrib-wildcard='*/profile/wn' /home/ec2-user/bpnet_output_dir/contrib.deeplift.h5 --overwrite

However, I got the following error:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/bpnet/bin/bpnet", line 8, in <module>
    sys.exit(main())
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/bpnet/__main__.py", line 38, in main
    argh.dispatch(parser)
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/argh/dispatching.py", line 174, in dispatch
    for line in lines:
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/argh/dispatching.py", line 277, in _execute_command
    for line in result:
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/argh/dispatching.py", line 260, in _call
    result = function(*positional, **keywords)
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/bpnet/cli/contrib.py", line 241, in bpnet_contrib
    batch_size=None)  # don't second-batch
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/bpnet/seqmodel.py", line 225, in contrib_score
    fn = self._contrib_deeplift_fn(x, name, preact_only=preact_only)
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/bpnet/seqmodel.py", line 145, in _contrib_deeplift_fn
    self.model = load_model(temp.name)
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/keras/engine/saving.py", line 419, in load_model
    model = _deserialize_model(f, custom_objects, compile)
  File "/home/ec2-user/miniconda3/envs/bpnet/lib/python3.6/site-packages/keras/engine/saving.py", line 224, in _deserialize_model
    model_config = json.loads(model_config.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'

To troubleshoot, I tried adding print statements in a few of the traceback files but I wasn't able to figure out what the initial cause of the error was.

Any help or guidance would be greatly appreciated!!

Thank you!

[Avsecz]

There seems to be an issue with model serialization. Can you check which TF and Keras version were installed?

[np519]

Thank you for getting back to me!

Tensorflow: 1.14.0 and Keras: 2.2.4.

[Avsecz]

Hm. Is does the EC2 instance have a GPU attached? Can you try running it with tensorflow-gpu==1.7?

[Avsecz]

I also recommend trying to initialize the keras model in python and try to save (model.save('path')) and re-load it. (using keras.models.load_model())

[np519]

Thank you for the additional tips!! An update:

I was using an m5.xlarge EC2 instance which does not have a GPU attached. I restarted the colab tutorial on an g4dn.xlarge instance which does contain a GPU. This did not fix the error and I had the same output as my first comment.

I tried switching the tensorflow version to tensorflow-gpu==1.7 and this also did not fix the issue.

When I tried reloading the keras model using ipython with the command keras.models.load_model() I again received the same error output as my original comment.

The solution: pip install 'h5py==2.10.0' --force-reinstall it seems there was a dependency error for h5py files and changing the version fixed the problem. Woohoo!!

[Avsecz]

That's great to hear! Thanks for posting the solution.