search

oushujun / EDTA

Extensive de-novo TE Annotator
https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1905-y
GNU General Public License v3.0
346 stars 73 forks source link

ValueError: File format not supported #484

Closed jwli-code closed 4 weeks ago

jwli-code commented 3 months ago

`Parameters: --genome /public/home/jwli/2024_disk/Brasscia_2024/00.data/BP11.chrplus.genome.fasta --species others --anno 1 --sensitive 1 --step all --t 20

Sat Jul 20 01:30:39 CST 2024 Dependency checking: All passed!

Sat Jul 20 01:31:05 CST 2024 Obtain raw TE libraries using various structure-based programs:

<< output from stderr >> Sat Jul 20 01:31:05 CST 2024 EDTA_raw: Check dependencies, prepare working directories.

Sat Jul 20 01:31:19 CST 2024 Start to find LTR candidates.

Sat Jul 20 01:31:19 CST 2024 Identify LTR retrotransposon candidates from scratch.

Sat Jul 20 04:37:12 CST 2024 Finish finding LTR candidates.

Sat Jul 20 04:37:12 CST 2024 Start to find SINE candidates.

Sat Jul 20 07:20:59 CST 2024 Finish finding SINE candidates.

Sat Jul 20 07:20:59 CST 2024 Start to find LINE candidates.

Sat Jul 20 07:20:59 CST 2024 Identify LINE retrotransposon candidates from scratch.

Sun Jul 21 22:55:24 CST 2024 Finish finding LINE candidates.

Sun Jul 21 22:55:24 CST 2024 Start to find TIR candidates.

Sun Jul 21 22:55:24 CST 2024 Identify TIR candidates from scratch.

Species: others Traceback (most recent call last): File "/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/TIR-Learner3.0.py", line 80, in TIRLearner_instance = TIRLearner(genome_file, genome_name, species, TIR_length, File "/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 72, in init self.execute() File "/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 110, in execute self.execute_M4() File "/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 634, in execute_M4 self["base"] = CNN_predict.execute(self) File "/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 108, in execute df = predict(df, TIRLearner_instance.genome_file_path, File "/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 59, in predict model = load_model(path_to_model) File "/public/home/jwli/micromamba/envs/EDTA2/lib/python3.10/site-packages/keras/src/saving/saving_api.py", line 193, in load_model raise ValueError( ValueError: File format not supported: filepath=/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/cnn0912_tf_savedmodel. Keras 3 only supports V3 .keras files and legacy H5 format files (.h5 extension). Note that the legacy SavedModel format is not supported by load_model() in Keras 3. In order to reload a TensorFlow SavedModel as an inference-only layer in Keras 3, use keras.layers.TFSMLayer(/public/home/jwli/micromamba/envs/EDTA2/share/EDTA/bin/TIR-Learner3.0/cnn0912_tf_savedmodel, call_endpoint='serving_default') (note that your call_endpoint might have a different name). Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.fa: No such file or directory at /public/home/jwli/micromamba/envs/EDTA2/share/EDTA/util/rename_tirlearner.pl line 19. Warning: LOC list BP11.chrplus.genome.fasta.mod.TIR.ext30.list is empty.

Error: Error while loading sequence Filter sequence based on TEsorter classifications. Unclassified sequences will also be output to the clean file. Usage: perl cleanup_misclas.pl sequence.fa.rexdb.cls.tsv Author: Shujun Ou (shujun.ou.1@gmail.com) 10/11/2019

mv: cannot stat 'BP11.chrplus.genome.fasta.mod.TIR.ext30.fa.pass.fa.dusted.cln.cln': No such file or directory cp: cannot stat 'BP11.chrplus.genome.fasta.mod.TIR.ext30.fa.pass.fa.dusted.cln.cln.list': No such file or directory cp: cannot stat 'BP11.chrplus.genome.fasta.mod.TIR.intact.raw.fa.anno.list': No such file or directory Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.gff3: No such file or directory. ERROR: No such file or directory at /public/home/jwli/micromamba/envs/EDTA2/share/EDTA/util/output_by_list.pl line 39. Warning: The TIR result file has 0 bp!

Mon Jul 22 04:16:04 CST 2024 Start to find Helitron candidates.

Mon Jul 22 04:16:04 CST 2024 Identify Helitron candidates from scratch. `

FayeFang17 commented 2 months ago

Hi Jiwi-code, sorry for the late reply, may I ask which version of EDTA did you install, and would it be possible for you to provide your file for me(fangn@umich.edu) to test out the issue? Thanks!

Faye

AsimKumarGhosh commented 2 months ago

Hi @jwli-code, is this problem solved? If yes, then how you did it? Thanks

yyliang12 commented 1 month ago

Hi, I also encountered the same problem. The EDTA version that I used was v2.2.0.

Sat Sep 21 13:41:56 CST 2024    Identify TIR candidates from scratch.

Species: others
Traceback (most recent call last):
  File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/TIR-Learner3.0.py", line 80, in <module>
    TIRLearner_instance = TIRLearner(genome_file, genome_name, species, TIR_length,
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 72, in __init__
    self.execute()
  File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 110, in execute
    self.execute_M4()
  File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 634, in execute_M4
    self["base"] = CNN_predict.execute(self)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 108, in execute
    df = predict(df, TIRLearner_instance.genome_file_path,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 59, in predict
    model = load_model(path_to_model)
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yanyang_liang/miniconda3/envs/EDTA/lib/python3.12/site-packages/keras/src/saving/saving_api.py", line 204, in load_model
    raise ValueError(
ValueError: File format not supported: filepath=/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/cnn0912_tf_savedmodel. Keras 3 only supports V3 `.keras` files and legacy H5 format files (`.h5` extension). Note that the legacy SavedModel format is not supported by `load_model()` in Keras 3. In order to reload a TensorFlow SavedModel as an inference-only layer in Keras 3, use `keras.layers.TFSMLayer(/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/cnn0912_tf_savedmodel, call_endpoint='serving_default')` (note that your `call_endpoint` might have a different name).
Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.fa: No such file or directory at /home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/util/rename_tirlearner.pl line 19.
Warning: LOC list P1_hap2_FINAL_All.fasta.mod.TIR.ext30.list is empty.

Error: Error while loading sequence

Any helps will be appreciated!

Yanyang.

oushujun commented 1 month ago

Hello,

Please install using the list of conda dependencies. In the Installation-Conda-others section of the readme, the last one specify the list. Please check out other recent issues for installation.

Shujun

On Sat, Sep 21, 2024 at 5:01 AM Ryann Leung @.***> wrote:

Hi, I also encountered the same problem. The EDTA version that I used was v2.2.0.

Sat Sep 21 13:41:56 CST 2024 Identify TIR candidates from scratch.

Species: others Traceback (most recent call last): File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/TIR-Learner3.0.py", line 80, in TIRLearner_instance = TIRLearner(genome_file, genome_name, species, TIR_length, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 72, in init self.execute() File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 110, in execute self.execute_M4() File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 634, in execute_M4 self["base"] = CNN_predict.execute(self) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 108, in execute df = predict(df, TIRLearner_instance.genome_file_path, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 59, in predict model = load_model(path_to_model) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yanyang_liang/miniconda3/envs/EDTA/lib/python3.12/site-packages/keras/src/saving/saving_api.py", line 204, in load_model raise ValueError( ValueError: File format not supported: filepath=/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/cnn0912_tf_savedmodel. Keras 3 only supports V3 .keras files and legacy H5 format files (.h5 extension). Note that the legacy SavedModel format is not supported by load_model() in Keras 3. In order to reload a TensorFlow SavedModel as an inference-only layer in Keras 3, use keras.layers.TFSMLayer(/home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/bin/TIR-Learner3.0/cnn0912_tf_savedmodel, call_endpoint='serving_default') (note that your call_endpoint might have a different name). Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.fa: No such file or directory at /home/yanyang_liang/miniconda3/envs/EDTA/share/EDTA/util/rename_tirlearner.pl line 19. Warning: LOC list P1_hap2_FINAL_All.fasta.mod.TIR.ext30.list is empty.

Error: Error while loading sequence

Any helps will be appreciated!

Yanyang.

— Reply to this email directly, view it on GitHub https://github.com/oushujun/EDTA/issues/484#issuecomment-2365068676, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNX4NH3H2TNYCYOQUH3NQLZXUYWLAVCNFSM6AAAAABLHMG2J6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRVGA3DQNRXGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

yyliang12 commented 1 month ago

Hi Prof. Ou @oushujun :

Thanks for talking care of this problem!

Through several tests, I think it might be some conflicts between TIR_Learner and some of the dependencies within the conda environment. I tried to use the EDTA v2.0.1, use the Singularity image, change another input genome and etc., but none of them worked. Finally I installed a new TIR_Learner into the EDTA conda environment with:

mamba install -n EDTA tir-learner

Then modified the code 

my $TIR_Learner = "$script_path/bin/TIR-Learner3.0";
python3 $TIR_Learner/TIR-Learner3.0.py

into

my $TIR_Learner = "$script_path/../TIR-Learner3";
python3 $TIR_Learner/TIR-Learner.py

of the script EDTA_raw.pl respectively. After that everything went well!

Maybe some of you guys @jwli-code @AsimKumarGhosh can try to do so or following to the instruction given by Prof. Ou above.

Thanks, Yanyang.

oushujun commented 4 weeks ago

@yyliang12 The problem is that you installed the EDTA2 conda env but used it to run on the master branch. Please read about this issue for more information: #497

Closing it for now.

Thanks! Shujun