[scAPP] test 10x cooladmin jobsubmit after cellranger.

biomystery commented 4 years ago

JYH_1046_1_2

biomystery commented 4 years ago

A runTimeWarning

password promoted


submitting cool admin job
/home/zhc268/anaconda2/envs/django/lib/python3.6/site-packages/django/db/models/fields/__init__.py:1423: RuntimeWarning: DateTimeField CoolAdminSubmission.date_submitted received a naive datetime (2019-12-13 13:11:47.883213) while time zone support is active.
RuntimeWarning)
paramString:  VERSION="V4",USEHARMONY="False",SNAPUSEPEAK="False",SNAPSUBSET="0",DOCHROMVAR="False",READINPEAK="0.0",TSSPERCELL="7.0",MINREADPERCELL="500",SNAPBINSIZE="5000 100000",SNAPNDIMS="25",GENOMETYPE="mm10",
[13/Dec/2019 13:11:47] "POST /singlecell/Ajax/SubmitCA/ HTTP/1.1" 200 22
Password: [13/Dec/2019 13:13:20] "GET /metadata/mymeta/ HTTP/1.1" 200 9906

biomystery commented 4 years ago

./utility/coolAdmin.sh zhc268@ucsd.edu "JYH_1047_1_2" VERSION="V4",USEHARMONY="False",SNAPUSEPEAK="False",SNAPSUBSET="0",DOCHROMVAR="False",READINPEAK="0",TSSPERCELL="7",MINREADPERCELL="500",SNAPBINSIZE="5000 100000",SNAPNDIMS="25",GENOMETYPE="mm10",

biomystery commented 4 years ago

error 1: qsub: option requires an argument -- 'M' error 2: SNAPBINSIZE="5000 100000", --need quoted.

biomystery commented 4 years ago

(bds_atac) [zhc268@tscc-login2 ~]$ cat LIMS_10x_cooladmin.e17712547

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

python2: can't open file '/home/zhc268/code/snATAC/snATAC_pipeline/clustering_pipeline.py': [Errno 2] No such file or directory

(bds_atac) [zhc268@tscc-login2 ~]$ cat LIMS_10x_cooladmin.o17712547
Additional arguments to be used: -perform_chromVAR_analysis True
version: v4
genome type: mm10
output name: JYH_1047_1_2
dataset name: JYH_1047_1_2
promoter file: /home/opoirion/data/ref_genomes/mouse/mm10/mm10_all_genes_refseq_TSS_promoter_2000.bed
ref barcode list: /projects/ps-epigen/outputs/10xATAC/JYH_1047_1_2/outs/singlecell.csv
bed file: /projects/ps-epigen/outputs/10xATAC/JYH_1047_1_2/outs/fragments.tsv.gz
reads in peaks: 0
tss per cell: 7
min number of reads per cell: 0.0
Additional arguments to be used:  -snap_bin_size 5000 100000  -snap_ndims 25  -perform_chromVAR_analysis True  -perform_cicero_analysis True
Nodes:        tscc-3-6

biomystery commented 4 years ago

4c93352 solve double quote but have bug:

clustering_pipeline.py: error: argument -min_number_of_reads_per_cell: invalid int value: '0.0'

biomystery commented 4 years ago

./utility/coolAdmin.sh zhc268@ucsd.edu "JYH_1046_1_2" VERSION=\"V4\",USEHARMONY=\"False\",SNAPUSEPEAK=\"False\",SNAPSUBSET=\"0\",DOCHROMVAR=\"False\",READINPEAK=\"0.0\",TSSPERCELL=\"7.0\",MINREADPERCELL=\"500\",SNAPBINSIZE=\"5000\ 100000\",SNAPNDIMS=\"25\",GENOMETYPE=\"mm10\",

--->

qsub -k oe -M zhc268@ucsd.edu -v OUTPUTNAME=JYH_1046_1_2,INPUTNAME=JYH_1046_1_2,DATASETNAME=JYH_1046_1_2,VERSION="V4",USEHARMONY="False",SNAPUS\
EPEAK="False",SNAPSUBSET="0",DOCHROMVAR="False",READINPEAK="0.0",TSSPERCELL="7.0",MINREADPERCELL="500",SNAPBINSIZE="5000 100000",SNAPNDIMS="25"\
,GENOMETYPE="mm10" /projects/ps-epigen/software/snATACCoolAdmin_LIMS/10x_model.bash

(bds_atac) [zhc268@tscc-login2 ~]$ cat LIMS_10x_cooladmin.o17712588
Additional arguments to be used: -perform_chromVAR_analysis True
version: v4
genome type: mm10
output name: JYH_1046_1_2
dataset name: JYH_1046_1_2
promoter file: /home/opoirion/data/ref_genomes/mouse/mm10/mm10_all_genes_refseq_TSS_promoter_2000.bed
ref barcode list: /projects/ps-epigen/outputs/10xATAC/JYH_1046_1_2/outs/singlecell.csv
bed file: /projects/ps-epigen/outputs/10xATAC/JYH_1046_1_2/outs/fragments.tsv.gz
reads in peaks: 0.0
tss per cell: 7.0
min number of reads per cell: 0.0
Additional arguments to be used:  -snap_bin_size 5000 100000  -snap_ndims 25  -perform_chromVAR_analysis True  -perform_cicero_analysis True

biomystery commented 4 years ago

https://github.com/epigen-UCSD/snATACCoolAdmin_LIMS/commit/f2a62db27c3aec038fa7ba07ab23f076f11b7bce

biomystery commented 4 years ago

Configuration of the .bashrc / .zshrc

The following path: /home/opoirion/go/local/bin should be included in the global path. Otherwise, our GO package to process snATAC-Seq file should be installed.

# Edit ~/.bashrc or ~/.zshrc
vim ~/.bashrc
#Then add the go exectuable path to the global path
PATH=$PATH:/home/opoirion/go/local/bin
# Then close the file
# then source the file
source ~/.bashrc

Add follow: https://gitlab.com/Grouumf/snATAC

Test using v2 as default

biomystery commented 4 years ago

sh: BAMutils: command not found
Warning: Permanently added 'ns104190.ip-147-135-44.us,147.135.44.237' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
Traceback (most recent call last):
  File "/home/opoirion/code/snATAC/snATAC_pipeline/clustering_pipeline.py", line 110, in <module>
    main()
  File "/home/opoirion/code/snATAC/snATAC_pipeline/clustering_pipeline.py", line 99, in main
    raise Exception('Exception: {0}'.format(trace))
Exception: Exception: Traceback (most recent call last):
  File "/home/opoirion/code/snATAC/snATAC_pipeline/clustering_pipeline.py", line 73, in main
    ref_barcode_list=ARGS.ref_barcode_list)
  File "/home/opoirion/code/snATAC/snATAC_pipeline/bed_and_bigwig_class.py", line 289, in create_subset_of_bed_file
    self._exec_cmd_no_check(cmd, outfile=output_file_name)
  File "/home/opoirion/code/snATAC/snATAC_pipeline/base_pipeline.py", line 494, in _exec_cmd_no_check
    raise(e)
AssertionError

Error

biomystery commented 4 years ago

https://github.com/epigen-UCSD/epigen_ucsd_django/commit/35624c2712de32b0f141a8748d4e0be4dad85413

http://ns104190.ip-147-135-44.us:8088/dataset_report?dataset_name=MM_166&output_folder_name=output_LIMS&token=40859255049624087c2d666c8821cbc7

biomystery commented 4 years ago

http://ns104190.ip-147-135-44.us:8088/dataset_report?dataset_name=MM_181&output_folder_name=output_LIMS&token=cffd6871e8c72bd4985f16f083822f0a

epigen-UCSD / epigen_ucsd_django