deconvolution error

[stevenedwards]

I can run the workflow on the exampleData to completion using "fusion" and "_transform" switch but when using "deconvolution" and "_transform" I get the following error message as soon as the job starts:

MissingInputException in line 377 of /cfs/klemming/nobackup/s/sedw/snakemake-workflows-master/spim_registration/timelapse/Snakefile: Missing input files for rule deconvolution: example_Data_merge.xml_timelapse_transfor

Any idea what's wrong?

edit: when using "deconvolution" with "_external_trafo" the deconvolution works. Is it not possible to use "deconvolution with "_transform"? if so, will: matrix_transform: '"0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0"', provide no downsampling?

[schmiedc]

Could be a possible bug. As it is written both ways should be allow. I will have to test this.

As you correctly pointed out a different transformation matrix will have the same effect. Without downsampling it would be: '"1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0"' As in the scaling is not changed.

Hope that provides you a quick fix.

Cheers, Christopher

[stevenedwards]

Thanks for the workflow and quick response! If the quick fix holds up (I will test tomorrow) you may close the issue.

[Xqua]

Hi I've been running into the same issue.

The SnakeFile script is looking as input for the following files:

for Fusion

input: [ str("{xml_base}_merge.xml_" + config["common"]["transformation_switch"] ) ]

For Deconvolution

input: [ str("{xml_base}_merge.xml_" + config["common"]["transformation_switch"] + config["common"]["external_trafo_switch"] ) ],

But ! The problem arise that the external_Trafo_Switch is never called, Fresh start example :

Job counts:
    count   jobs
    2   deconvolution
    1   define_output
    1   define_xml_czi
    1   done
    1   hdf5_xml
    1   hdf5_xml_output
    2   registration
    2   resave_hdf5
    2   resave_hdf5_output
    1   xml_merge
    14

It never calls on the rule: rule external_transform:

And thus the SnakeFile fails because that file is never "created"

MissingInputException in line 377 of /net/rcnfs10/srv/export/extavour_lab/share_root/Lab/SPIM/snakemake-workflows/spim_registration/timelapse/Snakefile:
Missing input files for rule deconvolution:
spaguetisquash_merge.xml_timelapse_duplicate_external_trafo

One quick Fix is to just copy the merged XML file ...

[sommerc]

Hi all, any news for this? I am having the exact same problem. Cheers, Chris

[MM603]

Hello, Same problem... Did you solve it somehow? Cheers,

[schmiedc]

Hi, can you provide the .config.yaml you both are using? I will have a look at it.

Cheers, Christopher

[MM603]

Hi, Thanks for your quick response! Here is my config file

config.zip

Cheers, Mj

[schmiedc]

@MM603 @sommerc @Xqua

you want to run the deconvolution without downsampling?

It seems that this option is broken. A fix is to still run the deconvolution with the external transform by setting the external_trafo_switch: "_external_trafo"

But using an identity transform: matrix_transform: '"1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0"'

This means that there is no downsampling. If you want to run the deconvolution with downsampling then set the external_trafo_switch: "_external_trafo" Then use a matrix_transform for the downsampling you want to select: 2x downsampling is: matrix_transform: '"0.5, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0"'

I don't understand the matrix_transform you specified in your config.yaml. It seems you want to achieve an affine transformation, however this is not the idea behind the external transformation in the workflow.

Could you please try that and report back if any of these options work for you?

Thank you!

[Xqua]

Sorry yes that's basically what I ended up doing (transform 1.0 0.0 etc) this worked fine for me !

Another trick was to touch spaguetisquash_merge.xml_timelapse_duplicate_external_trafo

which creates an empty file, bypassing the step.

[schmiedc]

@Xqua

Thanks a lot! I will just change the comments in the config.yaml. So this problem is solved then.

Cheers, Christopher

[schmiedc]

@MM603 @sommerc @stevenedwards @Xqua

Updated the config.yaml accordingly: https://github.com/mpicbg-scicomp/snakemake-workflows/pull/32

Issue can be closed

[MM603]

Thanks!

[sommerc]

Great thanks!