'utf-8' codec can't encode characters in position 16154-16159: surrogates not allowed

Donbbit commented 2 months ago

I run saw realign after I did register on StereoMap: saw realign \ --id=Adjuated \ --sn=test \ --count-data=./result/ \ --realigned-image-tar=./test_4.0.1.tar.gz

and I got the error:

Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... [Mon Sep 23 23:58:30 2024] Error in rule report: jobid: 0 input: .............. output: .............. log: .............. RuleException: UnicodeEncodeError in file saw-8.0.2/config/realign_base.smk, line 598: 'utf-8' codec can't encode characters in position 16154-16159: surrogates not allowed File "saw-8.0.2/config/realign_base.smk", line 598, in __rule_report File "saw-8.0.2/config/util.py", line 19, in run_command File "saw-8.0.2/anaconda/lib/python3.8/concurrent/futures/thread.py", line 57, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2024-09-23T213243.410662.snakemake.log

How can I solve this problem？

Clouate commented 2 months ago

Hi, It would be helpful if you could provide a complete log, such as 'STEREO_A NALYSIS_WORKFLOW_PROCESSING/REPORT/log_report'. This error was usually caused by the StereoMap/SAW running directory or data path containing some abnormal symbols (non-English), such as some linux systems do not support Chinese symbol encoding.

Donbbit commented 2 months ago

Hi, It would be helpful if you could provide a complete log, such as 'STEREO_A NALYSIS_WORKFLOW_PROCESSING/REPORT/log_report'. This error was usually caused by the StereoMap/SAW running directory or data path containing some abnormal symbols (non-English), such as some linux systems do not support Chinese symbol encoding.

Thanks for your reply, but the STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/log_report is empty ![Uploading QQ截图20240924144708.png…]()

Clouate commented 2 months ago

Or could you provide us the 'pipeline-logs/log'?

Donbbit commented 2 months ago

Or could you provide us the 'pipeline-logs/log'?

The log is: Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Job stats: job count

all 1 cellcut 1 register 1 report 1 saturation 1 spatialcluster 1 tissuecut 1 tissuecut_micro 1 visualization 1 total 9

Select jobs to execute...

[Mon Sep 23 21:32:44 2024] rule register: input: ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANNOTATION/TEST.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam.summary.stat output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/log_register jobid: 3 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/performance_register reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag threads: 8 resources: tmpdir=/tmp/237730.1.st.q

Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... python saw-8.0.2/lib/register/register/main.pyc -i TEST_SC_20240923_170227_4.0.1.tar.gz -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE --sn TEST -v ORIGIN/outs/feature_expression/TEST.raw.gef --gpu -1 --core 8 -w True >Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/log_register 2>&1 python saw-8.0.2/lib/imagetools/main.pyc ipr2img -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST_SC_20240923_170227_4.0.1.tar.gz -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE -d tissue cell >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/log_register 2>&1 [Mon Sep 23 22:27:56 2024] Finished job 3. 1 of 9 steps (11%) done Select jobs to execute...

[Mon Sep 23 22:27:56 2024] rule tissuecut: input: ORIGIN/outs/feature_expression/TEST.merge.barcodeReadsCount.txt, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissue_fig, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/lasso_mask.flag log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_tissuecut jobid: 2 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/performance_tissuecut reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/lasso_mask.flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef; Input files updated by another job: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag threads: 8 resources: tmpdir=/tmp/237730.1.st.q

Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... python saw-8.0.2/lib/saw_tissuecut/Tissuecut.pyc -i ORIGIN/outs/feature_expression/TEST.raw.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX --dnbfile ORIGIN/outs/feature_expression/TEST.merge.barcodeReadsCount.txt -s Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST_ssDNA_tissue_cut.tif --sn TEST -O Transcriptomics -d -t 8 >Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_tissuecut 2>&1 saw-8.0.2/lib/geftools/bin/geftools bgef -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.ssDNA.gef -O Transcriptomics -b 1,5,10,20,50,100,150,200 -t 1 >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_tissuecut 2>&1 [Mon Sep 23 22:35:44 2024] Finished job 2. 2 of 9 steps (22%) done Select jobs to execute...

[Mon Sep 23 22:35:45 2024] rule tissuecut_micro: input: ORIGIN/outs/feature_expression/TEST.merge.barcodeReadsCount.txt, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/micro_flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/lasso_mask.flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut_micro_flag log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_tissuecut_micro jobid: 8 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/performance_tissuecut_micro reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut_micro_flag; Input files updated by another job: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/lasso_mask.flag threads: 8 resources: tmpdir=/tmp/237730.1.st.q

Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... [Mon Sep 23 22:35:47 2024] Finished job 8. 3 of 9 steps (33%) done Select jobs to execute...

[Mon Sep 23 22:35:47 2024] rule cellcut: input: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/flag log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut jobid: 4 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/performance_cellcut reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/flag; Input files updated by another job: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag resources: tmpdir=/tmp/237730.1.st.q

[Mon Sep 23 22:35:47 2024] rule spatialcluster: input: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.bin200_1.0.h5ad log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/log_spatialcluster jobid: 6 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/performance_spatialcluster reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.bin200_1.0.h5ad; Input files updated by another job: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef resources: tmpdir=/tmp/237730.1.st.q

[Mon Sep 23 22:35:47 2024] rule saturation: input: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_57_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_58_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_59_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_60_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_61_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_62_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_63_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_64_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_81_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_82_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_83_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_84_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_85_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_86_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_87_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_88_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_89_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_90_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_91_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_92_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_93_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_94_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_95_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_96_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANNOTATION/TEST.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam.summary.stat output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/saturation.json log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/log_saturation jobid: 7 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/performance_saturation reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/saturation.json; Input files updated by another job: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef resources: tmpdir=/tmp/237730.1.st.q

Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... saw-8.0.2/lib/saw_saturation/saturation -i ORIGIN/outs/feature_expression/TEST_raw_barcode_gene_exp.txt --tissue Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT --bcstat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_57_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_58_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_59_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_60_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_61_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_62_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_63_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_64_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_81_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_82_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_83_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_84_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_85_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_86_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_87_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_88_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_89_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_90_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_91_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_92_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_93_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_94_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_95_1.CIDMap.stat ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_96_1.CIDMap.stat --summary ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANNOTATION/TEST.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam.summary.stat >Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/log_saturation 2>&1 python saw-8.0.2/lib/saw_spatialcluster/spatial_cluster.pyc -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef -s 200 -r 1.0 -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.bin200_1.0.h5ad >Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/log_spatialcluster 2>&1 saw-8.0.2/lib/geftools/bin/geftools cgef -i ORIGIN/outs/feature_expression/TEST.raw.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.cellbin.gef -m Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST_ssDNA_mask.tif >Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 python saw-8.0.2/lib/saw_cellcorrect/cell_correct.pyc -i ORIGIN/outs/feature_expression/TEST.raw.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE -m Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST_ssDNA_mask.tif -d 10 >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 [Mon Sep 23 22:37:56 2024] Finished job 6. 4 of 9 steps (44%) done [Mon Sep 23 22:40:47 2024] Finished job 7. 5 of 9 steps (56%) done python saw-8.0.2/lib/imagetools/main.pyc img2rpi -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST_ssDNA_mask_edm_dis_10.tif -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST.rpi -g ssDNA/CellMask_adjusted -b 2 10 50 100 150 >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 python saw-8.0.2/lib/saw_cellcluster/cell_cluster.pyc -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST.adjusted.cellbin.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.cellbin_1.0.adjusted.h5ad >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 python saw-8.0.2/lib/cellshape/cellShape.pyc -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/TEST.adjusted.cellbin.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 python saw-8.0.2/lib/saw_cellcluster/cell_cluster.pyc -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.cellbin.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.cellbin_1.0.h5ad >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 python saw-8.0.2/lib/cellshape/cellShape.pyc -i Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.cellbin.gef -o Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE >>Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/log_cellcut 2>&1 [Mon Sep 23 23:58:28 2024] Finished job 4. 6 of 9 steps (67%) done Select jobs to execute...

[Mon Sep 23 23:58:28 2024] rule report: input: ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_57_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_58_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_59_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_60_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_61_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_62_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_63_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_64_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_81_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_82_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_83_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_84_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_85_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_86_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_87_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_88_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_89_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_90_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_91_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_92_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_93_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_94_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_95_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_96_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_57_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_58_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_59_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_60_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_61_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_62_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_63_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_64_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_81_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_82_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_83_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_84_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_85_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_86_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_87_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_88_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_89_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_90_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_91_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_92_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_93_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_94_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_95_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_96_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANNOTATION/TEST.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam.summary.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.bin200_1.0.h5ad, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/saturation.json, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut_micro_flag output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/AnalysisReport/report.html log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/log_report jobid: 5 benchmark: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/performance_report reason: Missing output files: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/AnalysisReport/report.html; Input files updated by another job: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/saturation.json, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut_micro_flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.bin200_1.0.h5ad resources: tmpdir=/tmp/237730.1.st.q

Building DAG of jobs... Using shell: /bin/bash Provided cores: 8 Rules claiming more threads will be scaled down. Select jobs to execute... [Mon Sep 23 23:58:30 2024] Error in rule report: jobid: 0 input: ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_57_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_58_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_59_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_60_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_61_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_62_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_63_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_64_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_81_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_82_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_83_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_84_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_85_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_86_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_87_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_88_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_89_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_90_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_91_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_92_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_93_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_94_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_95_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_96_1.CIDMap.stat, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_57_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_58_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_59_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_60_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_61_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_62_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_63_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_64_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_81_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_82_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_83_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_84_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_85_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_86_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_87_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_88_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_89_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_90_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_91_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_92_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_93_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_94_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_95_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ALIGNMENT/E200022415_L01_96_1.Log.final.out, ORIGIN/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANNOTATION/TEST.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam.summary.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut.stat, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/TEST.tissue.gef, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/IMAGE/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/flag, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/ANALYSIS/TEST.bin200_1.0.h5ad, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/saturation.json, Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/EXPRESSION_MATRIX/tissuecut_micro_flag output: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/AnalysisReport/report.html log: Adjuated_TEST/STEREO_ANALYSIS_WORKFLOW_PROCESSING/REPORT/log_report (check log file(s) for error details)

RuleException: UnicodeEncodeError in file saw-8.0.2/config/realign_base.smk, line 598: 'utf-8' codec can't encode characters in position 16154-16159: surrogates not allowed File "saw-8.0.2/config/realign_base.smk", line 598, in __rule_report File "saw-8.0.2/config/util.py", line 19, in run_command File "saw-8.0.2/anaconda/lib/python3.8/concurrent/futures/thread.py", line 57, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2024-09-23T213243.410662.snakemake.log

Clouate commented 2 months ago

It seems that your chip SN number and running path are a bit strange. The relevant path in the log should be an absolute path rather than a relative path. Could you change the path and rerun it? Make sure there are no Chinese or special characters in the path. Please also avoid path with Chinese and special characters when using StereoMap on Windows. Or run find STEREO_ANALYSIS_WORKFLOW_PROCESSING -name "log_*" to provide all our log files for further troubleshooting.

Donbbit commented 2 months ago

It seems that your chip SN number and running path are a bit strange. The relevant path in the log should be an absolute path rather than a relative path. Could you change the path and rerun it? Make sure there are no Chinese or special characters in the path. Please also avoid path with Chinese and special characters when using StereoMap on Windows. Or run find STEREO_ANALYSIS_WORKFLOW_PROCESSING -name "log_*" to provide all our log files for further troubleshooting.

Actually the path is an absolute path but I remove the front path when I paste it, and when I used StereoMap on Windows I have noticed to avoid path with Chinese and special characters. The 'utf-8' error might be relative to some settings with my python enviroment? Could u be kind to give me your e-mail so I can send u the logs? Or EXPRESSION_MATRIX folder contains cellbin.gef and tissue.gef , so maybe I can just use this result without REPORT output ?

Thank you ever so much~

Clouate commented 1 month ago

@Donbbit Hi, I have received your log file and found that the result you used for realign is from saw v8.0.2 count pipeline on the STOmics Cloud. I noticed that when you were running the saw count, you filled in the --tissue parameter with '皮肤'. However, it seems that STOmics cloud uses UTF-8 encoding instead of Unicode encoding when outputting results. Normally, the Chinese word '皮肤' would be encoded by Unicode to "\u76AE\u80A4", but the results on you HPC show that it was encoded by UTF-8 to "\uDCE7\uDC9A\uDCAE\uDCE8\uDC82\uDCA4", as shown in the following figure, which leads to the encoding error in this issue. 1727720886358

Therefore, I suggest that you could modify the 'tissue' information in the 'pipeline-logs/config. yaml' of the saw count pipeline to English before running saw realign, or avoid using Chinese word when submitting the saw count pipeline.

STOmics / SAW

'utf-8' codec can't encode characters in position 16154-16159: surrogates not allowed #149