keras?

[srobb1]

Hello,

Any idea what might be going on here?

I installed keras as documented in your install, but when I try to run, I get an error that it is not installed

(py36) smr@hickory $ conda install keras=2.2.4
Collecting package metadata (repodata.json): done
Solving environment: done

# All requested packages already installed.

(py36) smr@hickory $ python3 ../deepte/DeepTE/DeepTE.py
Traceback (most recent call last):
  File "../deepte/DeepTE/DeepTE.py", line 16, in <module>
    from scripts import DeepTE_pipeline_no_modification as pipeline_no_m
  File "/n/projects/smr/organisms/nematostella/Nvec/genomes/Nvec200/analysis/mite-hunter/deepte/DeepTE/scripts/DeepTE_pipeline_no_modification.py", line 12, in <module>
    from keras.models import load_model
ImportError: No module named 'keras'

Thanks for any insight, Sofia

[yanhaidong1]

Hi Sofia,

Thanks for using the DeepTE. The message shows the 'keras' has not been installed successfully in your python3. you can type 'which python3' in the linux to check what the path of the executive python3. I assume the python3 in your case may not come from the conda. You should use the python from the conda to run. Another way is to go into the 'python3' and 'import keras'. If you can successfully load without any problem, it means you install in a right way.

Best, Haidong

[srobb1]

Thanks. When I deactivated and reactivated my environment, this issue was correct. Not sure why, but good. Now I have something else

(py36) mp181vbhv2j:DeepTE smr$ ./DeepTE.py -d . -o results -i nvec200.1.families_db.fasta -sp M -m_dir Metazoans_model -fam Domain
Using TensorFlow backend.
start time is 1601916567.827049
Step1: transfer fasta data to CNN input data
Step2: classify TEs
Step2: 2) domain information is not exist
Traceback (most recent call last):
  File "./DeepTE.py", line 263, in <module>
    main()
  File "./DeepTE.py", line 238, in main
    pipeline_no_m.classify_pipeline(model_dir, input_CNN_data_file, temp_store_opt_dir, sp_type,te_fam)
  File "/Users/smr/src/DeepTE/scripts/DeepTE_pipeline_no_modification.py", line 304, in classify_pipeline
    predict_te(model_file_dic[model_name], model_name, x_all_test_list, y_all_test_nm_list,input_spe_type,y_all_test_nm_list)
  File "/Users/smr/src/DeepTE/scripts/DeepTE_pipeline_no_modification.py", line 208, in predict_te
    model = load_model(model)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/saving.py", line 419, in load_model
    model = _deserialize_model(f, custom_objects, compile)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/saving.py", line 225, in _deserialize_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/saving.py", line 458, in model_from_config
    return deserialize(config, custom_objects=custom_objects)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/layers/__init__.py", line 55, in deserialize
    printable_module_name='layer')
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 145, in deserialize_keras_object
    list(custom_objects.items())))
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/sequential.py", line 301, in from_config
    model.add(layer)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/sequential.py", line 181, in add
    output_tensor = layer(self.outputs[0])
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/base_layer.py", line 457, in __call__
    output = self.call(inputs, **kwargs)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/layers/core.py", line 883, in call
    output = self.activation(output)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/activations.py", line 31, in softmax
    return K.softmax(x)
  File "/Users/smr/anaconda3/envs/py36/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3231, in softmax
    return tf.nn.softmax(x, axis=axis)
TypeError: softmax() got an unexpected keyword argument 'axis'

[yanhaidong1]

Hi Sofia, I am glad you solve the previous problem. Have you used the 'example_data/input_test.fasta' file to do a test? I just tried the test data and can obtain the results using your command line. Also, you can send your example data to me and I can have a try. My email is hy17471@uga.edu. Best wishes Haidong

[srobb1]

I get the same TypeError: softmax() got an unexpected keyword argument 'axis' when I use the example/input_Test.fasta

I can send my file, but I do not think it is my fasta file or fasta format. I would love to be able to run this on my system. I have other files I would like to run in addition to the one I am trying to run now.

I very carefully followed the install directions for conda. Looks like the issues are coming from Tensorflow. Should I try a different version of tensorflow than the one listed in the documentation?

Thank you very much for your time. Sofia

[srobb1]

there is a problem with my tensorflow. I think i missed an error on install. I just read that tensorflow-gpu does not work with mac? What are the system requiremtents? i can try another machine

[yanhaidong1]

Hi Sofia, Can you try to install tensorflow instead of tensorflow-gpu? This works on my mac using the tensorflow. Best wishes Haidong

[srobb1]

ok. i deleted my entire environment, started again with the instructions in the doc, but installed tensorflow instead of the gpu version.

i get this error now:

(py36) mp181vbhv2j:DeepTE smr$ ./DeepTE.py -d . -o . -i nvec200.1.families_db.fasta -sp M -m_dir Metazoans_model -fam Domain
Using TensorFlow backend.
start time is 1601932840.7463648
Step1: transfer fasta data to CNN input data
Step2: classify TEs
Step2: 2) domain information is not exist
2020-10-05 15:20:41.055969: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2020-10-05 15:20:41.055992: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2020-10-05 15:20:41.055998: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2020-10-05 15:20:41.056003: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Step3: generate final output
Traceback (most recent call last):
  File "./DeepTE.py", line 263, in <module>
    main()
  File "./DeepTE.py", line 245, in main
    combine_opt.generate_fasta(ipt_seq, output_dir + '/opt_DeepTE.txt', output_dir)
  File "/Users/smr/src/DeepTE/scripts/DeepTE_combine_opt.py", line 124, in generate_fasta
    with open (opt_DeepTE_file,'r') as ipt:
FileNotFoundError: [Errno 2] No such file or directory: './/opt_DeepTE.txt'

the opt_DeepTE.txt is not being generated. I tried again with the example fasta. I get the same message.

Sofia

[yanhaidong1]

Hi Sofia, It looks like you ran successfully. Can you download again the DeepTE and have a try since I modify the script to fix this problem. Also can you check your working directory, working_dir/store_temp_opt_dir/, the 'Domain_results.txt' is what you needed. After running again using the latest version, you can find the opt_DeepTE.txt in your results directory. Best wishes Haidong

[srobb1]

ok! that worked!! thank you for your help!!

Also, i forgot to include in my last message that i had to make a change in this file: py36/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py

in this function i had to change axis=axis to just axis

def softmax(x, axis=-1):
    """Softmax of a tensor.

    # Arguments
        x: A tensor or variable.
        axis: The dimension softmax would be performed on.
            The default is -1 which indicates the last dimension.

    # Returns
        A tensor.
    """
    return tf.nn.softmax(x, axis)

was: return tf.nn.softmax(x, axis=axis) now: return tf.nn.softmax(x, axis)

[yanhaidong1]

Hi Sofia, Sounds good! thanks for the efforts to figure it out. It's helpful for other users. Best wishes, Haidong