search

QI2lab / merfish3d-analysis

3D MERFISH data processing
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

ValueError: XA and XB must have the same number of columns (i.e. feature dimension.) #9

Closed julianantolin closed 3 months ago

julianantolin commented 3 months ago

Description

I am running test_pixeldecode.py. The expected result is for pixel decoding to be performed on preprocessed data. I get the error "ValueError: XA and XB must have the same number of columns (i.e. feature dimension.)"

Is this a problem with the data preprocessing or something else?

What I Did

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File /home/julian/wf-merfish/test_pixeldecode.py:[1](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/test_pixeldecode.py:1)
----> 1 decode_factory.run_decoding(lowpass_sigma=(3,1,1),
      [2](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/test_pixeldecode.py:2)                             distance_threshold=0.8, # ~0.547
      [3](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/test_pixeldecode.py:3)                             magnitude_threshold=.3, # 
      [4](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/test_pixeldecode.py:4)                             minimum_pixels=27, 
      [5](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/test_pixeldecode.py:5)                             skip_extraction=True)

File ~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:980, in PixelDecoder.run_decoding(self, lowpass_sigma, distance_threshold, magnitude_threshold, minimum_pixels, skip_extraction)
    [978](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:978) if not(np.any(lowpass_sigma==0)):
    [979](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:979)     self._lp_filter(sigma=lowpass_sigma)
--> [980](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:980) self._decode_pixels(distance_threshold = distance_threshold,
    [981](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:981)                     magnitude_threshold = magnitude_threshold)
    [982](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:982) if not(skip_extraction):
    [983](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:983)     self._extract_barcodes(minimum_pixels)

File ~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:470, in PixelDecoder._decode_pixels(self, distance_threshold, magnitude_threshold)
    [468](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:468) scaled_pixel_traces = self._clip_pixel_traces(scaled_pixel_traces)
    [469](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:469) normalized_pixel_traces, pixel_magnitude_trace = self._normalize_pixel_traces(scaled_pixel_traces)
--> [470](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:470) distance_trace, codebook_index_trace = self._calculate_distances(normalized_pixel_traces,self._decoding_matrix)
    [472](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:472) del normalized_pixel_traces
    [473](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:473) cp.get_default_memory_pool().free_all_blocks()

File ~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:421, in PixelDecoder._calculate_distances(pixel_traces, codebook_matrix)
    [418](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:418)     codebook_matrix = np.hstack([np.zeros((len(codebook_matrix),1)), codebook_matrix])
    [419](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:419)     codebook_matrix = cp.asarray(codebook_matrix,dtype=cp.float32)[1:]
--> [421](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:421) distances = cdist(cp.ascontiguousarray(pixel_traces.T),cp.ascontiguousarray(codebook_matrix),metric='euclidean')
    [422](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:422) min_indices = cp.argmin(distances, axis=1)
    [423](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/wf_merfish/postprocess/PixelDecoder.py:423) min_distances = cp.min(distances, axis=1)

File ~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:495, in cdist(XA, XB, metric, out, **kwargs)
    [493](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:493)     raise ValueError('XB must be a 2-dimensional array.')
    [494](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:494) if s[1] != sB[1]:
--> [495](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:495)     raise ValueError('XA and XB must have the same number of columns '
    [496](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:496)                      '(i.e. feature dimension.)')
    [498](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:498) mA = s[0]
    [499](https://file+.vscode-resource.vscode-cdn.net/home/julian/wf-merfish/~/miniconda3/envs/wf-merfish/lib/python3.10/site-packages/cupyx/scipy/spatial/distance.py:499) mB = sB[0]

ValueError: XA and XB must have the same number of columns (i.e. feature dimension.)
dpshepherd commented 3 months ago

Can you please print the shape of pixel_traces and codebook_matrix just before the error?

This error suggests your data dimensionality does not match the codebook dimensionality.

julianantolin commented 3 months ago

The shapes are: (16, 4194304) (251, 17) Indeed mismatched

On Wed, Jun 12, 2024 at 11:18 AM Doug Shepherd @.***> wrote:

Can you please print the shape of pixel_traces and codebook_matrix just before the error?

This error suggests your data dimensionality does not match the codebook dimensionality.

— Reply to this email directly, view it on GitHub https://github.com/QI2lab/wf-merfish/issues/9#issuecomment-2163647294, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHWX2LJ4PDPHAYQHNB4AHTZHCGILAVCNFSM6AAAAABJG3V7WCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRTGY2DOMRZGQ . You are receiving this because you authored the thread.Message ID: @.***>

dpshepherd commented 3 months ago

Issue with Zhuang codebook. Need to write some extra checks, but will close for now as it's working locally.