Issue When Employing `RegionsOfInterest`.

[g-pacheco]

Dear @MikkelSchubert,

I am trying to run PaleoMix v2.0.0a0 but I am getting an error when trying to enter a BED file:

paleomix bam dryrun --max-threads 8 --adapterremoval-max-threads 8 --bwa-max-threads 8 --log-level debug --log-file UkData_PaleoMix.log UKData_PaleoMix.yaml
INFO Writing debug log to 'UkData_PaleoMix.log'
INFO Reading makefile 'UKData_PaleoMix.yaml'
WARNING option has been deprecated and will be removed in the future: Options :: Features :: Coverage
WARNING option has been deprecated and will be removed in the future: Options :: Features :: Depths
WARNING option has been deprecated and will be removed in the future: Options :: Features :: Summary
ERROR Error reading makefiles: Makefile requirement not met at 'Genomes :: RockDove_DoveTail_ReRun :: RegionsOfInterest':
ERROR   Expected value: key in 'Path'
ERROR   Observed value: 'RegionsOfInterest'
ERROR   Observed type:  str

The dryrun runs normally when I comment the two RegionsOfInterest lines:

#    RegionsOfInterest:
#      Loci: /projects/mjolnir1/people/dqm353/Pigeons/Reference/FPGP_FinalRun.EcoT22I_Extended.bed

paleomix bam dryrun --max-threads 8 --adapterremoval-max-threads 8 --bwa-max-threads 8 --log-level debug --log-file UkData_PaleoMix.log UKData_PaleoMix.yaml
INFO Writing debug log to 'UkData_PaleoMix.log'
INFO Reading makefile 'UKData_PaleoMix.yaml'
WARNING option has been deprecated and will be removed in the future: Options :: Features :: Coverage
WARNING option has been deprecated and will be removed in the future: Options :: Features :: Depths
WARNING option has been deprecated and will be removed in the future: Options :: Features :: Summary
INFO Validating FASTA files
INFO Building BAM pipeline for 'UKData_PaleoMix.yaml'
INFO Checking file dependencies
INFO Checking for auxiliary files
INFO Determining state of pipeline tasks
INFO Checking required software on localhost
INFO   [✓] AdapterRemoval v2.3.3
INFO   [✓] BWA v0.7.17
INFO   [✓] Python v3.9.12
INFO   [✓] ln
INFO   [✓] samtools v1.15.1
INFO Number of tasks:             30
INFO Number of done tasks:        3
INFO Number of runable tasks:     3
INFO Number of queued tasks:      24
INFO Number of outdated tasks:    0
INFO Number of failed tasks:      0
INFO Dry run completed successfully
DEBUG Shutting down local worker
INFO Log-file written to '/maps/projects/mjolnir1/people/dqm353/Pigeons/FPG/UKData/UkData_PaleoMix.log'

I am attaching here my .YAML and my .BED files.

PaleoMixFiles.zip

Any ideas of what could be happening? I reckon I am overlooking something but I cannot really tell what that would be.

Many thanks in advance, George.

[MikkelSchubert]

Hi George,

Paleomix v2.0.0a0 is currently not in a state where I'd recommend it for general use (the a is for alpha). More specifically, support for regions of interest have been removed temporarily (along with coverage, depth, and summary statistics), since I intend to rework those features when I have time.

So unless you need specific functionality that is only in the development branch, then I'd recommend that you use the latest release, namely v1.3.6.

Best, Mikkel

[g-pacheco]

Hello @MikkelSchubert,

Thanks very much for your swift reply.

Oh, sorry. I gave you the error from v2.0.0a0 because I thought that would be preferable -- my bad. I also get an error when I try to run v1.3.6 with a .BED file -- here it is:

paleomix bam dryrun --max-threads 2 --adapterremoval-max-threads 2 --bwa-max-threads 2 --log-level info --log-file UkData_PaleoMix.log UKData_PaleoMix.yaml
16:49:55 INFO Reading makefile 'UKData_PaleoMix.yaml'
16:49:55 INFO Validating FASTA files
16:49:55 INFO Building BAM pipeline for 'UKData_PaleoMix.yaml'
Traceback (most recent call last):
  File "/projects/mjolnir1/apps/conda/paleomix/bin/paleomix", line 8, in <module>
    sys.exit(entry_point())
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/main.py", line 118, in entry_point
    return main(sys.argv[1:])
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/main.py", line 114, in main
    return module.main(argv[1:])
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/pipelines/bam/pipeline.py", line 255, in main
    return run(args, pipeline_variant=pipeline)
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/pipelines/bam/pipeline.py", line 208, in run
    nodes = pipeline_func(config, makefile)
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/pipelines/bam/pipeline.py", line 120, in build_pipeline_full
    parts.add_statistics_nodes(config, makefile, target)
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/pipelines/bam/parts/statistics.py", line 37, in add_statistics_nodes
    nodes.extend(_build_depth(config, target, makefile["Prefixes"]))
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/pipelines/bam/parts/statistics.py", line 74, in _build_depth
    DepthHistogramNode(
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/nodes/commands.py", line 112, in __init__
    command=builder.finalize(),
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/atomiccmd/builder.py", line 227, in finalize
    self._object = AtomicCmd(self.call, **self.kwargs)
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/atomiccmd/command.py", line 143, in __init__
    arguments = self._process_arguments(id(self), self._command, kwargs)
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/atomiccmd/command.py", line 329, in _process_arguments
    cls._validate_arguments(arguments)
  File "/projects/mjolnir1/apps/conda/paleomix/lib/python3.9/site-packages/paleomix/atomiccmd/command.py", line 380, in _validate_arguments
    raise ValueError(
ValueError: TEMP_IN_INDEX cannot contain dir component: './Manchester_01.RockDove_DoveTail_ReRun.bam.bai'

Please let me know should you need any other information.

Kind regards, George.

[MikkelSchubert]

Well, that's not supposed to happen. Thanks for letting me know! I'll try to publish a fix tomorrow.

In the mean time, you should still be able to get coverage/summary statistics if you temporarily disable the Depths feature.

Best, Mikkel

[g-pacheco]

Thanks a lot, Mikkel. Have a great evening!

[MikkelSchubert]

I've now released a new version of PALEOMIX that should hopefully fix the problem (v1.3.7). Let me know if you run into any other issues.

Best, Mikkel

[g-pacheco]

Hello Mikkel,

Thanks a lot for acting on this so quickly. I am glad to confirm that this new version did fix the problem.

Greetings from Silkeborg, George.