Closed jairaj-mathur closed 1 month ago
Yichen-Si
commented
1 month ago The source of the issue seems to be that the command line tool "bc" is missing from your system so the postprocessing step of sorting the pixel level output was not done correctly. Could you try getting bc?
jairaj-mathur
commented
1 month ago Once I install it, how do I tell ficture where is bc installed? I am still getting the same error after installation. @Yichen-Si
hyunminkang
commented
1 month ago I believe that bc needs to be available in your ${PATH}.
jairaj-mathur
commented
1 month ago This solved my issue. Thank you!
Hi all, I am running the run together demo, and running into this issue: command executed:
ficture run_together --in-tsv ficture/examples/data/transcripts.tsv.gz \ --in-minmax ficture/examples/data/coordinate_minmax.tsv \ --in-feature ficture/examples/data/feature.clean.tsv.gz \ --bgzip /templates/FICTURE/bin/bgzip \ --tabix /templates/FICTURE/bin/tabix \ --out-dir /templates/FICTURE/output1 --all
error:
Creating minibatch from ficture/examples/data/transcripts.tsv.gz...
ficture make_spatial_minibatch --input ficture/examples/data/transcripts.tsv.gz --output /templates/FICTURE/output1/batched.matrix.tsv --mu_scale 1.0 --batch_size 500 --batch_buff 30 --major_axis Y INFO:root:Random seed 1727478548.7393394 ['X', 'random_index', 'Y', 'gene', 'MoleculeID', 'Count'] INFO:root:Read blocks of pixels: 674.99 x 267.61 INFO:root:Read blocks of pixels: 674.99 x 539.15 INFO:root:Output region (675.99, 540.15) (6689.0, 7365.0) x (6771.0, 7311.1) INFO:root:Left over size 74443 (74443, 29.99) INFO:root:Read blocks of pixels: 674.99 x 165.84 INFO:root:Read blocks of pixels: 674.99 x 165.84 INFO:root:Output region (675.99, 166.84) (6689.0, 7365.0) x (7280.2, 7447.0) sort -k 2,2n -k 1,1g /templates/FICTURE/output1/batched.matrix.tsv | gzip -c > /templates/FICTURE/output1/batched.matrix.tsv.gz rm /templates/FICTURE/output1/batched.matrix.tsv
Creating DGE for 12um...
ficture make_dge --key Count --input ficture/examples/data/transcripts.tsv.gz --output /templates/FICTURE/output1/hexagon.d_12.tsv --hex_width 12 --n_move 2 --min_ct_per_unit 50 --mu_scale 1.0 --precision 2 --major_axis Y INFO:root:Random seed 1727478574.1533086 WARNING:root:The designated major key is not one of the specified count columns, --count_header is ignored ['X', 'Y', 'gene', 'MoleculeID', 'Count'] INFO:root:Processing 999568 pixels (1000000 6772.0, 7039.61). INFO:root:Sliding offset 0, 0. Add 1310 units, median count 739.0, 1310 units so far. INFO:root:Sliding offset 0, 1. Add 1265 units, median count 714.0, 2575 units so far. INFO:root:Sliding offset 1, 0. Add 1315 units, median count 754.0, 3890 units so far. INFO:root:Sliding offset 1, 1. Add 1264 units, median count 711.0, 5154 units so far. INFO:root:Left over size 98694 (7016, 7040). INFO:root:Processing 1098287 pixels (1098694 7015.62, 7311.15). INFO:root:Sliding offset 0, 0. Add 1388 units, median count 715.0, 6542 units so far. INFO:root:Sliding offset 0, 1. Add 1420 units, median count 724.0, 7962 units so far. INFO:root:Sliding offset 1, 0. Add 1388 units, median count 718.0, 9350 units so far. INFO:root:Sliding offset 1, 1. Add 1430 units, median count 713.0, 10780 units so far. INFO:root:Left over size 60216 (7287, 7311). INFO:root:Processing 522391 pixels (522587 7287.16, 7447.0). INFO:root:Sliding offset 0, 0. Add 728 units, median count 593.0, 11508 units so far. INFO:root:Sliding offset 0, 1. Add 734 units, median count 607.0, 12242 units so far. INFO:root:Sliding offset 1, 0. Add 719 units, median count 590.0, 12961 units so far. INFO:root:Sliding offset 1, 1. Add 727 units, median count 609.5, 13688 units so far. INFO:root:Left over size 85622 (7423, 7447). sort -k 1,1n /templates/FICTURE/output1/hexagon.d_12.tsv | gzip -c > /templates/FICTURE/output1/hexagon.d_12.tsv.gz rm /templates/FICTURE/output1/hexagon.d_12.tsv
Creating LDA for 12um and 12 factors...
mkdir -p /templates/FICTURE/output1/analysis/nF12.d_12/figure ficture fit_model --input /templates/FICTURE/output1/hexagon.d_12.tsv.gz --output /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12 --feature ficture/examples/data/feature.clean.tsv.gz --nFactor 12 --epoch 3 --epoch_id_length 2 --unit_attr X Y --key Count --min_ct_per_feature 20 --test_split 0.5 --R 10 --thread 1 INFO:root:Read data with 3422 units, 347 features INFO:root:0: -3169.06, -3181.73 INFO:root:Counter({6: 489, 5: 319, 8: 182, 3: 162, 0: 104, 7: 102, 11: 101, 4: 85, 2: 65, 10: 64, 9: 27, 1: 11}) INFO:root:R=0, 29.89, 31.88, 7.21s INFO:root:1: -3167.50, -3180.95 INFO:root:Counter({6: 489, 5: 374, 8: 210, 11: 149, 7: 118, 3: 99, 0: 92, 2: 55, 10: 47, 4: 40, 9: 26, 1: 12}) INFO:root:R=1, 29.67, 30.88, 7.22s INFO:root:2: -3168.94, -3182.15 INFO:root:Counter({6: 483, 5: 388, 11: 185, 7: 148, 8: 111, 3: 104, 0: 99, 4: 84, 2: 43, 9: 32, 10: 20, 1: 14}) INFO:root:R=2, 28.99, 31.22, 7.17s INFO:root:3: -3171.91, -3184.96 INFO:root:Counter({6: 481, 5: 412, 11: 172, 7: 160, 8: 93, 3: 91, 0: 82, 2: 60, 4: 51, 9: 51, 10: 41, 1: 17}) INFO:root:R=3, 28.80, 30.53, 7.21s INFO:root:4: -3168.53, -3181.67 INFO:root:Counter({6: 452, 5: 395, 11: 162, 8: 138, 7: 138, 0: 111, 3: 99, 2: 61, 10: 56, 4: 47, 9: 42, 1: 10}) INFO:root:R=4, 29.49, 30.46, 7.17s INFO:root:5: -3169.06, -3181.90 INFO:root:Counter({6: 533, 5: 420, 8: 133, 3: 126, 7: 121, 11: 111, 2: 71, 0: 61, 10: 60, 9: 36, 4: 25, 1: 14}) INFO:root:R=5, 29.29, 30.91, 7.21s INFO:root:6: -3168.92, -3182.02 INFO:root:Counter({6: 466, 5: 391, 11: 172, 8: 150, 7: 126, 3: 102, 0: 101, 10: 52, 4: 47, 9: 47, 2: 46, 1: 11}) INFO:root:R=6, 29.53, 30.72, 7.16s INFO:root:7: -3167.90, -3181.40 INFO:root:Counter({6: 486, 5: 407, 8: 154, 11: 146, 7: 132, 3: 102, 0: 83, 2: 60, 10: 47, 9: 44, 4: 40, 1: 10}) INFO:root:R=7, 29.52, 30.96, 7.26s INFO:root:8: -3166.54, -3179.80 INFO:root:Counter({6: 440, 5: 380, 8: 174, 11: 146, 7: 134, 0: 110, 3: 106, 10: 79, 2: 57, 9: 48, 4: 27, 1: 10}) INFO:root:R=8, 30.34, 31.59, 7.21s INFO:root:9: -3166.46, -3179.64 INFO:root:Counter({6: 488, 5: 370, 11: 159, 8: 145, 7: 124, 3: 101, 0: 93, 10: 65, 2: 63, 4: 53, 9: 38, 1: 12}) INFO:root:R=9, 29.67, 30.25, 7.12s R 8 30.342741 0 29.886129 9 29.671403 1 29.668263 6 29.532584 7 29.518949 4 29.488823 5 29.293693 2 28.993412 3 28.803011 Name: Score, dtype: float64 INFO:root:Result file /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.fit_result.tsv.gz ficture choose_color --input /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.fit_result.tsv.gz --output /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12 --cmap_name turbo /databricks/python3/lib/python3.11/site-packages/sklearn/manifold/_mds.py:298: FutureWarning: The default value of
normalized_stresswill change to'auto'in version 1.4. To suppress this warning, manually set the value ofnormalized_stress. warnings.warn( ficture plot_base --input /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.fit_result.tsv.gz --output /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.coarse --fill_range 7.0 --color_table /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.rgb.tsv --plot_um_per_pixel 1 --plot_discretized INFO:root:Read region 3422 pixels in region 679.0 x 656.0 656 500 INFO:root:Filling pixels 0 - 500 / 655 INFO:root:Filling pixels 500 - 655 / 655 INFO:root:Start constructing RGB image INFO:root:Made fractional image /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.coarse.png INFO:root:Made hard threshold image /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.coarse.top.png touch /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.doneCreating projection for 12um and 12 factors, at 12um
Performing pixel-level decoding..
Sorting and reformatting the pixel-level output..
Performing pseudo-bulk differential expression analysis..
Drawing pixel-level output image...
ficture transform --input ficture/examples/data/transcripts.tsv.gz --output_pref /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.prj_12.r_4 --model /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.model.p --key Count --major_axis Y --hex_width 12 --n_move 3 --min_ct_per_unit 20 --mu_scale 1.0 --thread 1 --precision 2 INFO:root:Model loaded with 347 features and 12 factors INFO:root:Transformed 1 batches with total 11874 units, 0.364859min INFO:root:Transformed 2 batches with total 24776 units, 0.773665min INFO:root:Transformed 3 batches with total 31488 units, 0.991802min INFO:root:Transformed 4 batches with total 31658 units, 1.015078min ficture slda_decode --input /templates/FICTURE/output1/batched.matrix.tsv.gz --output /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5 --model /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.model.p --anchor /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.prj_12.r_4.fit_result.tsv.gz --anchor_in_um --neighbor_radius 5 --mu_scale 1.0 --key Count --precision 0.1 --lite_topk_output_pixel 3 --lite_topk_output_anchor 3 --thread 1 INFO:root:347 genes and 12 factors are read from input model INFO:root:Read 31322 grid points INFO:root:Read 1917666 pixels, forming 1 batches. INFO:root:Read 1 batches ((1917666, 347)) INFO:root:Output 1878492 pixels and 24157 anchors INFO:root:Read 515602 pixels, forming 1 batches. INFO:root:Read 1 batches ((515602, 347)) INFO:root:Output 515602 pixels and 7775 anchors bash /templates/FICTURE/output1/sort_decode.sh /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.pixel.tsv.gz /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.pixel.sorted.tsv.gz ficture/examples/data/coordinate_minmax.tsv nF12.d_12 100 100 3 /templates/FICTURE/bin/bgzip /templates/FICTURE/bin/tabix 6690, 7365; 6772, 7447 /templates/FICTURE/output1/sort_decode.sh: line 20: bc: command not found /templates/FICTURE/output1/sort_decode.sh: line 21: bc: command not found ficture de_bulk --input /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.posterior.count.tsv.gz --output /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.bulk_chisq.tsv --min_ct_per_feature 20 --max_pval_output 0.001 --min_fold_output 1.5 --thread 1 Read posterior count over 347 genes and 12 factors Testing 347 genes over 12 factors ficture factor_report --path /templates/FICTURE/output1/analysis/nF12.d_12 --pref nF12.d_12.decode.prj_12.r_4_5 --color_table /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.rgb.tsv ['0' '1' '2' '3' '4' '5' '6' '7' '8' '9' '10' '11'] /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.factor.info.html ficture plot_pixel_full --input /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.pixel.sorted.tsv.gz --color_table /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.rgb.tsv --output /templates/FICTURE/output1/analysis/nF12.d_12/figure/nF12.d_12.decode.prj_12.r_4_5.pixel.png --plot_um_per_pixel 0.5 --full 11:17:08 PM Background color 000000 11:17:08 PM Read color table (12) Index(['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11'], dtype='object', name='Name') 11:17:08 PM Read header {'K': 12, 'TOPK': 3, 'BLOCK_SIZE': 2000, 'BLOCK_AXIS': 'X', 'INDEX_AXIS': 'Y', 'OFFSET_X': 6690, 'OFFSET_Y': 6772, 'SIZE_X': '', 'SIZE_Y': '', 'SCALE': 100} Traceback (most recent call last): File "/databricks/python3/bin/ficture", line 8, in
sys.exit(main())
^^^^^^
File "/databricks/python3/lib/python3.11/site-packages/ficture/cli.py", line 43, in main
function(sys.argv[2:])
File "/databricks/python3/lib/python3.11/site-packages/ficture/scripts/plot_pixel_full.py", line 65, in plot_pixel_full
loader = BlockIndexedLoader(args.input, args.xmin, args.xmax, args.ymin, args.ymax, args.full, not args.org_coord, idtype=dty)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/databricks/python3/lib/python3.11/site-packages/ficture/loaders/pixel_factor_loader.py", line 47, in init
self.xmax = min(xmax, self.meta["SIZE_X"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'str' and 'float'
make: *** [/templates/FICTURE/output1/Makefile:52: /templates/FICTURE/output1/analysis/nF12.d_12/nF12.d_12.decode.prj_12.r_4_5.done] Error 1