search

thenineteen / Semiology-Visualisation-Tool

Data driven 3D brain visualisation of semiology. Semiology to anatomy translator based on over 4600 patients from 309 peer-reviewed articles.
MIT License
9 stars 6 forks source link

Combining two semiologies when either returns a value of 1 for the GIF parcellations: doesn't work #137

Closed thenineteen closed 4 years ago

thenineteen commented 4 years ago

I'm using the following custom semiologies e.g. mappings TLe and mappings TLf. Both work individually as expected, TLe showing the transverse temporal and TLf showing the planum temporale.

However, when combined, it doesn't work. As you can see, it is missing planum temporale:

image

thenineteen commented 4 years ago

probably related in someway to the fix for #135

thenineteen commented 4 years ago

also when combining the TLf with e.g. Visual, the combination doesn't seem to bump up the score of planum temporale. The STG had 16 from Visual and 0 from TLf, whereas planum temporale had 16 from Visual and 1 from TLf but both get 64:

image

fepegar commented 4 years ago

It's hard to debug this because all the lines printed by mega_analysis:

$ python -c "import mega_analysis; np = mega_analysis.Semiology('mappings TLf', symptoms_side=mega_analysis.semiology.Laterality.LEFT, dominant_hemisphere=mega_analysis.semiology.Laterality.LEFT).get_num_datapoints_dict(); print(np)"
Excluded post-ictal semiology in specific query
WARNING:root:The inspect_result lat col has NaNs/zero where it should not: autofilled
/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/site-packages/pandas/core/indexing.py:966: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 994, in emit
    msg = self.format(record)
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 840, in format
    return fmt.format(record)
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 577, in format
    record.message = record.getMessage()
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "<string>", line 1, in <module>
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/semiology.py", line 202, in get_num_datapoints_dict
    query_lateralisation_result = self.query_lateralisation()
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/semiology.py", line 185, in query_lateralisation
    pts_dominant_hemisphere_R_or_L=self.dominant_hemisphere.value,
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/crosstab/mega_analysis/QUERY_LATERALISATION.py", line 130, in QUERY_LATERALISATION
    row_to_one_map = pivot_result_to_one_map(row, one_map, raw_pt_numbers_string='pt #s',
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/crosstab/mega_analysis/mapping.py", line 77, in pivot_result_to_one_map
    str([col for col in pivot_result if col not in one_map])
Message: 7
Arguments: ('localisation column(s) in the pivot_result which cannot be found in one_map', 'These columns are: ', "['Localising', 'Lateralising', 'CL', 'IL', 'BL (Non-lateralising)', 'DomH', 'NonDomH']")
{185: 1.0, 186: 1.0}
fepegar commented 4 years ago

Please report the contents of the Slicer Python console for bugs related to Slicer, i.e.

/Users/fernando/git/Semiology-Visualisation-Tool/slicer/SemiologyVisualisation.py:885: RuntimeWarning: invalid value encountered in greater
  windowMin = scoresArray[scoresArray > 0].min() if scoresArray.any() else 0
Traceback (most recent call last):
  File "/Users/fernando/git/Semiology-Visualisation-Tool/slicer/SemiologyVisualisation.py", line 655, in updateColors
    self.scoresVolumeNode,
  File "/Users/fernando/git/Semiology-Visualisation-Tool/slicer/SemiologyVisualisation.py", line 885, in getScoresVolumeNode
    windowMin = scoresArray[scoresArray > 0].min() if scoresArray.any() else 0
  File "/Applications/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/numpy/core/_methods.py", line 34, in _amin
    return umr_minimum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation minimum which has no identity
fepegar commented 4 years ago

This happens because there are NaNs in the result from Semiology.query_lateralisation.

thenineteen commented 4 years ago

would converting the NaNs to zeros solve the problem then? df.fillna(0)

thenineteen commented 4 years ago

It's hard to debug this because all the lines printed by mega_analysis:

$ python -c "import mega_analysis; np = mega_analysis.Semiology('mappings TLf', symptoms_side=mega_analysis.semiology.Laterality.LEFT, dominant_hemisphere=mega_analysis.semiology.Laterality.LEFT).get_num_datapoints_dict(); print(np)"
Excluded post-ictal semiology in specific query
WARNING:root:The inspect_result lat col has NaNs/zero where it should not: autofilled
/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/site-packages/pandas/core/indexing.py:966: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 994, in emit
    msg = self.format(record)
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 840, in format
    return fmt.format(record)
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 577, in format
    record.message = record.getMessage()
  File "/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "<string>", line 1, in <module>
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/semiology.py", line 202, in get_num_datapoints_dict
    query_lateralisation_result = self.query_lateralisation()
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/semiology.py", line 185, in query_lateralisation
    pts_dominant_hemisphere_R_or_L=self.dominant_hemisphere.value,
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/crosstab/mega_analysis/QUERY_LATERALISATION.py", line 130, in QUERY_LATERALISATION
    row_to_one_map = pivot_result_to_one_map(row, one_map, raw_pt_numbers_string='pt #s',
  File "/Users/fernando/git/Semiology-Visualisation-Tool/mega_analysis/crosstab/mega_analysis/mapping.py", line 77, in pivot_result_to_one_map
    str([col for col in pivot_result if col not in one_map])
Message: 7
Arguments: ('localisation column(s) in the pivot_result which cannot be found in one_map', 'These columns are: ', "['Localising', 'Lateralising', 'CL', 'IL', 'BL (Non-lateralising)', 'DomH', 'NonDomH']")
{185: 1.0, 186: 1.0}

would logging.debug help? 0f2eaafd5ae579f775955b9404e32749ee04182d

thenineteen commented 4 years ago

when this issue is closed, will continue with #75

fepegar commented 4 years ago

would logging.debug help? 0f2eaaf

Yes but the main problem comes from this code in mapping.py:

if (len([col for col in pivot_result if col not in one_map]) > 0):
    logging.error(len([col for col in pivot_result if col not in one_map]),
                    'localisation column(s) in the pivot_result which cannot be found in one_map',
                    'These columns are: ',
                    str([col for col in pivot_result if col not in one_map])
                    )

You're passing multiple arguments to logging.error as it was print, but that doesn't work. You need to pass a single string to it.

P.S.: You can use ```python at the beginning of the code block to specify the syntax. That and using a nice indentation will make the code more readable and it will be easier to solve the issues.

fepegar commented 4 years ago

Also, logging.error doesn't stop the program. Is that what you want? Why not raising an Exception?

fepegar commented 4 years ago

The logging.warning in QUERY_LATERALISATION.py also prints stuff all the time. Can you use logging.debug instead?

fepegar commented 4 years ago

And I'm getting this message again:

/usr/local/Caskroom/miniconda/base/envs/ali/lib/python3.6/site-packages/pandas/core/indexing.py:966: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s
thenineteen commented 4 years ago

The logging.warning in QUERY_LATERALISATION.py also prints stuff all the time. Can you use logging.debug instead?

I see. I thought the debug console of slicer had 3 levels including warnings and errors (the tabs we can view)? Otherwise will do

fepegar commented 4 years ago

The debug console is set to capture levels >= DEBUG, that's why you see everything. In Python, the level is WARNING by default:

In [1]: import logging

In [2]: logging.info('sdf')

In [3]: logging.debug('sdf')

In [4]: logging.warning('sfd')
WARNING:root:sfd

In [5]: logging.error('sadf')
ERROR:root:sadf
fepegar commented 4 years ago

Fixed in #144 and #145.