search

nservant / HiC-Pro

HiC-Pro: An optimized and flexible pipeline for Hi-C data processing
Other
383 stars 183 forks source link

/tmp/hicmk.3522:1: *** missing separator. Stop. #193

Closed RuiqinZheng closed 5 years ago

RuiqinZheng commented 6 years ago

Hi! I don't know why when I run the HiC-pro, it comes error $ ./HiC-Pro -i ~/analysis/trimmomatic_data/2X4X/2X2/ -o ./test -c ~/analysis/hicpro_data/config_2X2.txt /tmp/hicmk.3522:1: *** missing separator. Stop. and I can not find the /tmp/hicmk.3522:1

nservant commented 6 years ago

Hi, That's surprising. On which system are you ? Do you have a /tmp directory ? Best

RuiqinZheng commented 6 years ago

I hava a tmp directory , I try to find the file hicmk.3522.1 ,but there's no such a file under tmp directory. and the result also occured on other servers. I also download the latest version of HiC-Pro, and then install it. But the error message occured again.

nservant commented 6 years ago

Could you show me your ~/analysis/hicpro_data/config_2X2.txt file please ?

RuiqinZheng commented 6 years ago

Please change the variable settings below if necessary

#########################################################################

Paths and Settings - Do not edit !

#########################################################################

TMP_DIR = tmp LOGS_DIR = logs BOWTIE2_OUTPUT_DIR = bowtie_results MAPC_OUTPUT = hic_results RAW_DIR = rawdata

#######################################################################

SYSTEM AND SCHEDULER - Start Editing Here !!

####################################################################### N_CPU = 2 LOGFILE = hicpro.log

JOB_NAME = IMR90_split JOB_MEM = 10gb JOB_WALLTIME = 6:00:00 JOB_QUEUE = batch JOB_MAIL = nservant@curie.fr

#########################################################################

Data

#########################################################################

PAIR1_EXT = _R1 PAIR2_EXT = _R2

#######################################################################

Alignment options

#######################################################################

FORMAT = phred33 MIN_MAPQ = 20

BOWTIE2_IDX_PATH = /public/home/rqzheng/tools//HiC-Pro_2.8.1_devel/HiC-Pro_2.8.1_devel/annotation/Arabidopsis_thaliana/TAIR10 BOWTIE2_GLOBAL_OPTIONS = --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder BOWTIE2_LOCAL_OPTIONS = --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder

#######################################################################

Annotation files

#######################################################################

REFERENCE_GENOME = TAIR10 GENOME_SIZE = chrom_TAIR10.sizes

#######################################################################

Allele specific analysis

####################################################################### Please change the variable settings below if necessary

#########################################################################

Paths and Settings - Do not edit !

#########################################################################

TMP_DIR = tmp LOGS_DIR = logs BOWTIE2_OUTPUT_DIR = bowtie_results MAPC_OUTPUT = hic_results RAW_DIR = rawdata

#######################################################################

SYSTEM AND SCHEDULER - Start Editing Here !!

####################################################################### N_CPU = 2 LOGFILE = hicpro.log

JOB_NAME = IMR90_split JOB_MEM = 10gb JOB_WALLTIME = 6:00:00 JOB_QUEUE = batch JOB_MAIL = nservant@curie.fr

#########################################################################

Data

#########################################################################

PAIR1_EXT = _R1 PAIR2_EXT = _R2

#######################################################################

Alignment options

#######################################################################

FORMAT = phred33 MIN_MAPQ = 20

BOWTIE2_IDX_PATH = /public/home/rqzheng/tools//HiC-Pro_2.8.1_devel/HiC-Pro_2.8.1_devel/annotation/Arabidopsis_thaliana/TAIR10 BOWTIE2_GLOBAL_OPTIONS = --very-sensitive -L 30 --score-min L,-0.6,-0.2 --end-to-end --reorder BOWTIE2_LOCAL_OPTIONS = --very-sensitive -L 20 --score-min L,-0.6,-0.2 --end-to-end --reorder

#######################################################################

Annotation files

#######################################################################

REFERENCE_GENOME = TAIR10 GENOME_SIZE = chrom_TAIR10.sizes

#######################################################################

Allele specific analysis

#######################################################################

ALLELE_SPECIFIC_SNP =

#######################################################################

Digestion Hi-C

#######################################################################

GENOME_FRAGMENT = HindIII_resfrag_TAIR10.bed LIGATION_SITE = AAGCTAGCTT MIN_FRAG_SIZE = 100 MAX_FRAG_SIZE = 100000 MIN_INSERT_SIZE = 100 MAX_INSERT_SIZE = 600

#######################################################################

Hi-C processing

#######################################################################

MIN_CIS_DIST = GET_ALL_INTERACTION_CLASSES = 1 GET_PROCESS_SAM = 1 RM_SINGLETON = 1 RM_MULTI = 1 RM_DUP = 1

#######################################################################

Contact Maps

#######################################################################

BIN_SIZE = 30000 50000 75000 100000 150000 200000 250000 300000 MATRIX_FORMAT = upper

#######################################################################

Normalization

####################################################################### MAX_ITER = 100 FILTER_LOW_COUNT_PERC = 0.02 FILTER_HIGH_COUNT_PERC = 0 EPS = 0.1

nservant commented 6 years ago

hum ... yes, it's normal because the tmp files are removed as soon as an exit signal is trapped. See the hic.inc.sh script https://github.com/nservant/HiC-Pro/blob/master/scripts/hic.inc.sh Then, the tmp files are used to read the config ... but by eyes, it looks ok. Not sure how or when the "missing operator" error is generated I will try to find some explanation ...

RuiqinZheng commented 6 years ago

ok thanks! I'll waiting for your reply!

nservant commented 6 years ago

Could you try that : with $conf is your config file

cat $conf > /tmp/tmp000 echo "_dummytarget:" >> /tmp/tmp000 make -f /tmp/tmp0000 -p -n

RuiqinZheng commented 6 years ago

[20:55:59] rqzheng@admin:~/analysis/hicpro_data : $ cat config_2X2.txt >/tmp/tmp000 $ echo "dummy_target:" >> /tmp/tmp000 $ make -f /tmp/tmp0000 -p -n |less make: /tmp/tmp0000: No such file or directory

following is the result when I use ‘|less’

Copyright (C) 2006 Free Software Foundation, Inc.

This is free software; see the source for copying conditions.

There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A

PARTICULAR PURPOSE.

nservant commented 6 years ago

Sorry, there is an error ;

cat $conf > /tmp/tmp000 echo "dummy_target:" >> /tmp/tmp000 make -f /tmp/tmp000 -p -n > out

RuiqinZheng commented 6 years ago

this is my result: [21:02:18] rqzheng@admin:~/analysis/hicpro_data : $ make -f /tmp/tmp000 -p -n > out /tmp/tmp000:1: *** missing separator. Stop.

RuiqinZheng commented 6 years ago

[21:02:18] rqzheng@admin:~/analysis/hicpro_data : $ make -f /tmp/tmp0000 -p -n|less make: /tmp/tmp0000: No such file or directory make: *** No rule to make target `/tmp/tmp0000'. Stop.

nservant commented 6 years ago

Ok. So the 'make' command is generating the error. But I don't know how to fix it :) Any information about your make version for instance ?

RuiqinZheng commented 6 years ago

$ make configure make -f ./scripts/install/Makefile CONFIG_SYS=./config-install.txt make[1]: Entering directory `/public/home/rqzheng/tools/HiC-Pro' ./scripts/install/install_dependencies.sh -c ./config-install.txt -o /public/home/rqzheng/tools/HiC-Pro-new/HiC-Pro_2.11.0-beta -q Make sure internet connection works for your shell prompt under current user's privilege ... Starting HiC-Pro installation ... Checking dependencies ... Checking Python libraries ... The required Python libraries appear to be already installed. Checking R installation ... The required R packages appear to be already installed. Bowtie2 Aligner appears to be already installed. Samtools appears to be already installed. Dependencies checked ! Check HiC-Pro configuration ... Warning : Scheduler system not defined - Default is Torque/PBS Configuration for TORQUE/PBS system.

make[1]: Leaving directory `/public/home/rqzheng/tools/HiC-Pro'

RuiqinZheng commented 6 years ago

#########################################################################

Paths and Settings - Start editing here !

#########################################################################

PREFIX = /public/home/rqzheng/tools/HiC-Pro-new BOWTIE2_PATH = SAMTOOLS_PATH = R_PATH = PYTHON_PATH = CLUSTER_SYS =

RuiqinZheng commented 6 years ago

$ make install (g++ -Wall -O2 -std=c++0x -o build_matrix /public/home/rqzheng/tools/HiC-Pro/scripts/src/build_matrix.cpp; mv build_matrix /public/home/rqzheng/tools/HiC-Pro/scripts) (g++ -Wall -O2 -std=c++0x -o cutsite_trimming /public/home/rqzheng/tools/HiC-Pro/scripts/src/cutsite_trimming.cpp; mv cutsite_trimming /public/home/rqzheng/tools/HiC-Pro/scripts) runner=rqzheng

Installing the iced package in --user repository (cp /public/home/rqzheng/tools/HiC-Pro/scripts/src/ice_mod/iced/scripts/ice /public/home/rqzheng/tools/HiC-Pro/scripts; cd /public/home/rqzheng/tools/HiC-Pro/scripts/src/ice_mod/; /public/home/rqzheng/tools/python27/bin/python setup.py install --user;) /public/home/rqzheng/tools/python27/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'include_package_data' warnings.warn(msg) /public/home/rqzheng/tools/python27/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe' warnings.warn(msg) Warning: Assuming default configuration (iced/utils/{setup_utils,setup}.py was not found)Appending iced.utils configuration to iced Ignoring attempt to set 'name' (from 'iced' to 'iced.utils') Appending iced.datasets configuration to iced Ignoring attempt to set 'name' (from 'iced' to 'iced.datasets') non-existing path in 'iced/io': '../src/cblas' non-existing path in 'iced/io': '../src/cblas' Appending iced.io configuration to iced Ignoring attempt to set 'name' (from 'iced' to 'iced.io') non-existing path in 'iced': '../src/cblas' non-existing path in 'iced': '../src/cblas' Appending iced configuration to Ignoring attempt to set 'name' (from '' to 'iced') running install running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler options running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options running build_src buildsrc building extension "iced.io.fastio" sources building extension "iced.io.read" sources building extension "iced.normalization" sources building extension "iced.filter" sources building data_files sources build_src: building npy-pkg config files running build_py running build_ext customize UnixCCompiler customize UnixCCompiler using build_ext running build_scripts running install_lib running install_scripts changing mode of /public/home/rqzheng/.local/bin/ice to 755 running install_data running install_egg_info Removing /public/home/rqzheng/.local/lib/python2.7/site-packages/iced-0.4.2-py2.7.egg-info Writing /public/home/rqzheng/.local/lib/python2.7/site-packages/iced-0.4.2-py2.7.egg-info running install_clib customize UnixCCompiler make: realpath: Command not found cp -Ri /public/home/rqzheng/tools/HiC-Pro /public/home/rqzheng/tools/HiC-Pro-new/HiC-Pro_2.11.0-beta HiC-Pro installed in !

nservant commented 6 years ago

i see that there is other unix error like ; realpath: Command not found I'm afraid you will have some unix compatibilty issue ? Are you on a Mac ? The easiest would be to use the singularity image of HiC-Pro which therefore do not require any system environment (you just need singularity installed)

RuiqinZheng commented 6 years ago

I'm on the centos system. I'll try to use the singularity . Thanks !

RuiqinZheng commented 6 years ago

but ..can you give me a html or some other else about the singularity?

nservant commented 6 years ago

I'm also on Centos ... that's weird than a command as 'realpath' is not defined. The same for your make ... I do not really know what's going on. For Singularity, look at the main README file. https://github.com/nservant/HiC-Pro

You can either build your own Singularity image from the 'Singularity file'. Or just download it (there is a link on the page), and run

singularity exec hicpro_latest_ubuntu.img HiC-Pro -h

RuiqinZheng commented 6 years ago

but when I not run it with data, it seems no problem [21:32:44] rqzheng@admin:~/tools/HiC-Pro-new/HiC-Pro_2.11.0-beta/bin : $ HiC-Pro usage : HiC-Pro -i INPUT -o OUTPUT -c CONFIG [-s ANALYSIS_STEP] [-p] [-h] [-v] Use option -h|--help for more information

nservant commented 6 years ago

Yes, because the error occurs when it reads the config file

RuiqinZheng commented 6 years ago

I know the reason! because the config file I use is from the older version of HiC-Pro , when I try to use the new config file under the new version of the HiC-Pro ,it run! Thanks for your reply

L1angyan commented 2 years ago

I know the reason! because the config file I use is from the older version of HiC-Pro , when I try to use the new config file under the new version of the HiC-Pro ,it run! Thanks for your reply

Hello Ruiqin, I met the same error. However, I used the config-hicpro.txt download from the HiC-Pro installing directory not the previous version. It also did not work. Can you give me some advice to solve the problems? I am looking forward your reply.