Config.yaml error DropSeqPipe v0.4

[Hofphi]

Hey @Hoohm,

I was excited to see that you released a new version of DropSeqPipe featuring the new version of dropseq-tools (v2.0). I tried running the v0.4 release locally to check if everything is working as expected. Unfortunately I have a really "basic" problem right at the beginning of running the pipeline.

I couldn't even start the pipeline and got this error:

WorkflowError in line 14 of /home/kirstin/Tools/dropSeqPipe2/Snakefile:
Config file is not valid JSON or YAML. In case of YAML, make sure to not mix whitespace and tab indentation.
  File "/home/kirstin/Tools/dropSeqPipe2/Snakefile", line 14, in <module>

I revised the config.yaml various times and also replaced it with a fresh git clone of DropSeqPipe but I am still getting the same error. I even tried to implement the command configfile: "config.yaml" to load the config.yaml in the snakefile of the previous release, but without success (same error message as above). I think that something is wrong with the Snakefile. Do you have any idea of what is going on?

I ran snakemake --use-conda --directory /home/kirstin/Tools/dropSeqPipe2/ in a single species scenario!

Here is my config.yaml output:

CONTACT:
  email: ''
  person: ''
LOCAL:
    temp-directory: /mnt/DATA/DropSeq_Philipp/tmpdir
    memory: 4g
    raw_data: /home/kirstin/Tools/dropSeqPipe2/RAW_DATA
    results: /home/kirstin/Tools/dropSeqPipe2/RESULT_DIR
META:
    species:
        SPECIES_ONE: homo_sapiens
            build: grch37
            release: 94
    ratio: 0.2
    reference-directory: /mnt/DATA/DropSeq_Philipp/References
    gtf_biotypes: /home/kirstin/Tools/dropSeqPipe2/gtf_biotypes.yaml

FILTER:
    barcode-whitelist: ''
    5-prime-smart-adapter: AAGCAGTGGTATCAACGCAGAGT
    cell-barcode:
        start: 1
        end: 12
    UMI-barcode:
        start: 13
        end: 20
    cutadapt:
        adapters-file: /home/kirstin/Tools/dropSeqPipe2/templates/NexteraPE-PE.fa
        R1:
            quality-filter: 20
            maximum-Ns: 1
            extra-params: ''
        R2:
            quality-filter: 20
            minimum-adapters-overlap: 6
            minimum-length: 15
            extra-params: ''
MAPPING:
    STAR:
        genomeChrBinNbits: 18
        outFilterMismatchNmax: 10
        outFilterMismatchNoverLmax: 0.3
        outFilterMismatchNoverReadLmax: 1
        outFilterMatchNmin: 0
        outFilterMatchNminOverLread: 0.66
        outFilterScoreMinOverLread: 0.66
EXTRACTION:
    LOCUS:
        - CODING
        - UTR
    strand-strategy: SENSE
    UMI-edit-distance: 1
    minimum-counts-per-UMI: 0

[Hoohm]

That's my bad. Documentation is not clear. SPECIES_ONE should be homo_sapiens.

On Thu, Dec 20, 2018, 16:33 Philipp notifications@github.com wrote:

Hey @Hoohm https://github.com/Hoohm,

I was excited to see that you released a new version of DropSeqPipe featuring the new version of dropseq-tools (v2.0). I tried running the v0.4 release locally to check if everything is working as expected. Unfortunately I have a really "basic" problem right at the beginning of running the pipeline.

I couldn't even start the pipeline and got this error:

WorkflowError in line 14 of /home/kirstin/Tools/dropSeqPipe2/Snakefile: Config file is not valid JSON or YAML. In case of YAML, make sure to not mix whitespace and tab indentation. File "/home/kirstin/Tools/dropSeqPipe2/Snakefile", line 14, in

I revised the config.yaml various times and also replaced it with a fresh git clone of DropSeqPipe but I am still getting the same error. I even tried to implement the command configfile: "config.yaml" to load the config.yaml in the snakefile of the previous release, but without success (same error message as above). I think that something is wrong with the Snakefile. Do you have any idea of what is going on?

I ran snakemake --use-conda --directory /home/kirstin/Tools/dropSeqPipe2/ in a single species scenario!

Here is my config.yaml output:

CONTACT: email: '' person: '' LOCAL: temp-directory: /mnt/DATA/DropSeq_Philipp/tmpdir memory: 4g raw_data: /home/kirstin/Tools/dropSeqPipe2/RAW_DATA results: /home/kirstin/Tools/dropSeqPipe2/RESULT_DIR META: species: SPECIES_ONE: homo_sapiens build: grch37 release: 94 ratio: 0.2 reference-directory: /mnt/DATA/DropSeq_Philipp/References gtf_biotypes: /home/kirstin/Tools/dropSeqPipe2/gtf_biotypes.yaml

FILTER: barcode-whitelist: '' 5-prime-smart-adapter: AAGCAGTGGTATCAACGCAGAGT cell-barcode: start: 1 end: 12 UMI-barcode: start: 13 end: 20 cutadapt: adapters-file: /home/kirstin/Tools/dropSeqPipe2/templates/NexteraPE-PE.fa R1: quality-filter: 20 maximum-Ns: 1 extra-params: '' R2: quality-filter: 20 minimum-adapters-overlap: 6 minimum-length: 15 extra-params: '' MAPPING: STAR: genomeChrBinNbits: 18 outFilterMismatchNmax: 10 outFilterMismatchNoverLmax: 0.3 outFilterMismatchNoverReadLmax: 1 outFilterMatchNmin: 0 outFilterMatchNminOverLread: 0.66 outFilterScoreMinOverLread: 0.66 EXTRACTION: LOCUS:

• CODING
• UTR strand-strategy: SENSE UMI-edit-distance: 1 minimum-counts-per-UMI: 0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hoohm/dropSeqPipe/issues/67, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNXaKqhHMSCLPq8TB6UiT8up2LpQk-iks5u663dgaJpZM4ZcaW6 .

[Hoohm]

And build should be 37

On Thu, Dec 20, 2018, 17:50 Patrick Roelli patrick.roelli@gmail.com wrote:

That's my bad. Documentation is not clear. SPECIES_ONE should be homo_sapiens.

On Thu, Dec 20, 2018, 16:33 Philipp notifications@github.com wrote:

Hey @Hoohm https://github.com/Hoohm,

I was excited to see that you released a new version of DropSeqPipe featuring the new version of dropseq-tools (v2.0). I tried running the v0.4 release locally to check if everything is working as expected. Unfortunately I have a really "basic" problem right at the beginning of running the pipeline.

I couldn't even start the pipeline and got this error:

WorkflowError in line 14 of /home/kirstin/Tools/dropSeqPipe2/Snakefile: Config file is not valid JSON or YAML. In case of YAML, make sure to not mix whitespace and tab indentation. File "/home/kirstin/Tools/dropSeqPipe2/Snakefile", line 14, in

I revised the config.yaml various times and also replaced it with a fresh git clone of DropSeqPipe but I am still getting the same error. I even tried to implement the command configfile: "config.yaml" to load the config.yaml in the snakefile of the previous release, but without success (same error message as above). I think that something is wrong with the Snakefile. Do you have any idea of what is going on?

I ran snakemake --use-conda --directory /home/kirstin/Tools/dropSeqPipe2/ in a single species scenario!

Here is my config.yaml output:

CONTACT: email: '' person: '' LOCAL: temp-directory: /mnt/DATA/DropSeq_Philipp/tmpdir memory: 4g raw_data: /home/kirstin/Tools/dropSeqPipe2/RAW_DATA results: /home/kirstin/Tools/dropSeqPipe2/RESULT_DIR META: species: SPECIES_ONE: homo_sapiens build: grch37 release: 94 ratio: 0.2 reference-directory: /mnt/DATA/DropSeq_Philipp/References gtf_biotypes: /home/kirstin/Tools/dropSeqPipe2/gtf_biotypes.yaml

FILTER: barcode-whitelist: '' 5-prime-smart-adapter: AAGCAGTGGTATCAACGCAGAGT cell-barcode: start: 1 end: 12 UMI-barcode: start: 13 end: 20 cutadapt: adapters-file: /home/kirstin/Tools/dropSeqPipe2/templates/NexteraPE-PE.fa R1: quality-filter: 20 maximum-Ns: 1 extra-params: '' R2: quality-filter: 20 minimum-adapters-overlap: 6 minimum-length: 15 extra-params: '' MAPPING: STAR: genomeChrBinNbits: 18 outFilterMismatchNmax: 10 outFilterMismatchNoverLmax: 0.3 outFilterMismatchNoverReadLmax: 1 outFilterMatchNmin: 0 outFilterMatchNminOverLread: 0.66 outFilterScoreMinOverLread: 0.66 EXTRACTION: LOCUS:

• CODING
• UTR strand-strategy: SENSE UMI-edit-distance: 1 minimum-counts-per-UMI: 0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hoohm/dropSeqPipe/issues/67, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNXaKqhHMSCLPq8TB6UiT8up2LpQk-iks5u663dgaJpZM4ZcaW6 .

[Hoohm]

I'll make a schema validation soon in order to feel with those small configuration issues

On Thu, Dec 20, 2018, 17:51 Patrick Roelli patrick.roelli@gmail.com wrote:

And build should be 37

On Thu, Dec 20, 2018, 17:50 Patrick Roelli patrick.roelli@gmail.com wrote:

That's my bad. Documentation is not clear. SPECIES_ONE should be homo_sapiens.

On Thu, Dec 20, 2018, 16:33 Philipp notifications@github.com wrote:

Hey @Hoohm https://github.com/Hoohm,

I was excited to see that you released a new version of DropSeqPipe featuring the new version of dropseq-tools (v2.0). I tried running the v0.4 release locally to check if everything is working as expected. Unfortunately I have a really "basic" problem right at the beginning of running the pipeline.

I couldn't even start the pipeline and got this error:

WorkflowError in line 14 of /home/kirstin/Tools/dropSeqPipe2/Snakefile: Config file is not valid JSON or YAML. In case of YAML, make sure to not mix whitespace and tab indentation. File "/home/kirstin/Tools/dropSeqPipe2/Snakefile", line 14, in

I revised the config.yaml various times and also replaced it with a fresh git clone of DropSeqPipe but I am still getting the same error. I even tried to implement the command configfile: "config.yaml" to load the config.yaml in the snakefile of the previous release, but without success (same error message as above). I think that something is wrong with the Snakefile. Do you have any idea of what is going on?

I ran snakemake --use-conda --directory /home/kirstin/Tools/dropSeqPipe2/ in a single species scenario!

Here is my config.yaml output:

CONTACT: email: '' person: '' LOCAL: temp-directory: /mnt/DATA/DropSeq_Philipp/tmpdir memory: 4g raw_data: /home/kirstin/Tools/dropSeqPipe2/RAW_DATA results: /home/kirstin/Tools/dropSeqPipe2/RESULT_DIR META: species: SPECIES_ONE: homo_sapiens build: grch37 release: 94 ratio: 0.2 reference-directory: /mnt/DATA/DropSeq_Philipp/References gtf_biotypes: /home/kirstin/Tools/dropSeqPipe2/gtf_biotypes.yaml

FILTER: barcode-whitelist: '' 5-prime-smart-adapter: AAGCAGTGGTATCAACGCAGAGT cell-barcode: start: 1 end: 12 UMI-barcode: start: 13 end: 20 cutadapt: adapters-file: /home/kirstin/Tools/dropSeqPipe2/templates/NexteraPE-PE.fa R1: quality-filter: 20 maximum-Ns: 1 extra-params: '' R2: quality-filter: 20 minimum-adapters-overlap: 6 minimum-length: 15 extra-params: '' MAPPING: STAR: genomeChrBinNbits: 18 outFilterMismatchNmax: 10 outFilterMismatchNoverLmax: 0.3 outFilterMismatchNoverReadLmax: 1 outFilterMatchNmin: 0 outFilterMatchNminOverLread: 0.66 outFilterScoreMinOverLread: 0.66 EXTRACTION: LOCUS:

• CODING
• UTR strand-strategy: SENSE UMI-edit-distance: 1 minimum-counts-per-UMI: 0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hoohm/dropSeqPipe/issues/67, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNXaKqhHMSCLPq8TB6UiT8up2LpQk-iks5u663dgaJpZM4ZcaW6 .

[Hofphi]

Hey Patrick,

thanks for your answer. I already checked if the species name or the build was the issue but I still get the same error message pointing to line 14 in the snakefile. Seems like some file is corrupt but I could not figure out the problem yet

[Hoohm]

Here is what you should have:

META:
    species:
        homo_sapiens:
            build: 37
            release: 94

On Thu, Dec 20, 2018, 18:08 Philipp notifications@github.com wrote:

Hey Patrick,

thanks for your answer. I already checked if the species name or the build was the issue but I still get the same error message pointing to line 14 in the snakefile. Seems like some file is corrupt but I could not figure out the problem yet

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hoohm/dropSeqPipe/issues/67#issuecomment-449068720, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNXaKdjimklO-2y18poQL_GXoOMDTBEks5u68QBgaJpZM4ZcaW6 .

[Hoohm]

Don't copy paste it directly. I'm doing this on my phone. I'll add some example on the documentation tomorrow

[Hoohm]

I changed the example on the documentation website. I hope this clears things up. Let me know how it works for you

[Hofphi]

Thanks for the nice documentation Patrick!

I found my error! I had two config files one was in the working directory and another copy in the templates folder. On the previous release I did the same but without any problems. I guess it has something to do with the changes in the snakefile of dropSeqPipe release v0.4.

# Load configuration file
configfile: "config.yaml"

changed to

# Load configuration files

try:
    configfile_path = config['configfile_path']
except:
    configfile_path = "config.yaml"    
configfile: configfile_path

Anyway, everything is running fine now. Sorry for the trouble, you can close the issue!

Have a nice Christmas!

[Hoohm]

Merry christmas!