Lisa model goes into D state indefinitely

[kaltinel]

Thank you for your respond before regarding the proper usage of prefix and configuration file update.

I am trying to run the lisa_modelagain, with AR.symbols, and the job is going into uninterruptible (D) stateafter a benign warning :

lisa_model --species hg38 --gene_set /data/lisa/demo/AR.symbols --threads 2 --foreground None --epigenome "DNase" --background None --prefix /data/lisa/demo/hg38 --new_h5=None --covariates=None --only_newhdf5=None --sample_number 1

/opt/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
(1175,)

If I choose the other option in epigenome, it sees the dataset as the tuple number changes, but it is still in the D state and nothing ever prints: lisa_model --species hg38 --gene_set /data/lisa/demo/AR.symbols --threads 2 --foreground None --epigenome "H3K27ac" --background None --prefix /data/lisa/demo/hg38 --new_h5=None --covariates=None --only_newhdf5=None --sample_number 1

/opt/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
(2173,)

I am not sure why the process goes into D state, and how can I provide I/O for it to start working. I would appreciate the help.

[qinqian]

What does D state mean? Could you please provide us a screenshot?

[kaltinel]

Sure! D state is in the state where the running job does not accept any signals from outside and cannot be stopped before it completes a process or gets out of its running loop:

Related discussions : https://unix.stackexchange.com/questions/16738/when-a-process-will-go-to-d-state

[qinqian]

It is an unexpected situation which we never saw before. Can you use lisa model instead of lisa_model to run again after cd into /data/lisa/demo directory? A demo command line is like following:

lisa model --method="all" --web=False --new_rp_h5=None --new_count_h5=None --species hg38 --epigenome "['DNase', 'H3K27ac']" --cluster=False --covariates=False --random=True --prefix AR --background=dynamic_auto_tad --stat_background_number=1000 --threads 4 AR.gene_symbol

[kaltinel]

Sure, now it does not even start as something was missing in the configuration ( I believe some of the parameters): lisa model --method="all" --web=False --new_rp_h5=None --new_count_h5=None --species hg38 --epigenome "['DNase', 'H3K27ac']" --cluster=False --covariates=False --random=True --prefix AR --background=dynamic_auto_tad --stat_background_number=1000 --threads 4 AR.symbol

/opt/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Error: profile given but no config.yaml found. Profile has to be given as either absolute path, relative path or name of a directory available in either /etc/xdg/snakemake or /home/kal/.config/snakemake.

Also, under demo, I have these files ( I am just adding them in here as a list to not to miss anything ):

AR.Snakefile.model         
AR.symbol.yml  
hg38.Snakefile.model  
lisa_results_meta_table_human_with_gene_sets.xls  
run.sh          
AR.symbol                  
AR.yml         
hg38.yml              
lisa_results_meta_table_mouse_with_gene_sets.xls  
test.Snakefile.model
AR.symbol.Snakefile.model  
hg38/           
lisa_meta.xls         
lisa_v1.0_hg38.tar.gz                             
test.yml

[qinqian]

This problem is caused by the snakemake version issue, could you check snakemake --version? For Lisa, the snakemake needs to be downgraded to version 4.0.0.

[kaltinel]

Thank you for the response. Indeed my snakemake is 5.4.1v. I tried to set the snakemake version to 4.0.0 in bioconda, however it does not see the renewal.

[qinqian]

Can you try this conda install -c bioconda snakemake==4.0.0? I need to update Lisa to be compatible with the latest snakemake version later.

[kaltinel]

I tried every possible solution to set my snakemakeenvironment to 4.0.0 however it is not working as conda always complains with : CondaValueError: Malformed version string '~': invalid character(s). which appears to be a bug related to conda. I would appreciate any insight to it as I would really like to use LISA locally.

[qinqian]

I found a solution to the problem here. First, upgrade the conda base; second, override all channels.

[kaltinel]

Thank you for this help, I tried all the possible solutions but unfortunately no luck. I also tried making a virtual environment, but each solution is coming up with its own specific error. Currently, I am not able to re-set my snakemake version to 4.0.0. I will hopefully use LISA when it accepts 5.4.1.

[qinqian]

I have updated LISA to accept the latest snakemake (5.7.1), you can upgrade LISA by using conda install -c qinqian lisa.