Filter roi count zero

[will-moore]

In preparing metadata workshop, wanting to know which images have 0 ROIs, I found that filtering for ROI_count < 1 didn't show any results because no data exist if no ROIs are found.

Also, loading table data ROI_count for a Dataset where NO images have any ROIs causes an error on numpy.amin(values) in views.py because values is an empty list.

Both these can be fixed by adding data[image_id] = 0 for images with no ROIs. This also improves the appearance of the ROI_count column in the table since we don't have empty table cells.

To test:

• Filter a Dataset of Images for ROI_Count < 1.
• Add ROI_Count column to the table layout and see that 0 appears for images with no ROIs.

[pwalczysko]

On web-dev-merge, user:outreach, Dataset http://web-dev-merge.openmicroscopy.org/webclient/?show=dataset-23611. There are 2 images in this Dataset, none of them has any ROIs.

• select the Dataset
• go to the right of the central pane and select Parade from the dropdown
• select filtering and table display as indicated on the screenshot
• observe error

Traceback (most recent call last):

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)

  File "/opt/hudson/workspace/WEB-DEV-merge-deploy/OMERO.py/lib/python/omeroweb/decorators.py", line 483, in wrapped
    retval = f(request, *args, **kwargs)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/omero_parade/views.py", line 181, in get_data
    'histogram': list(histogram)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/http/response.py", line 535, in __init__
    data = json.dumps(data, cls=encoder)

  File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)

  File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)

  File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)

  File "/home/hudson/virtualenv/lib/python2.7/site-packages/django/core/serializers/json.py", line 115, in default
    return super(DjangoJSONEncoder, self).default(o)

  File "/usr/lib64/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")

TypeError: 0 is not JSON serializable

<WSGIRequest
path:/omero_parade/data/Uk9JX2NvdW50/,
GET:<QueryDict: {u'_': [u'1525368653608'], u'dataset': [u'23611']}>,
POST:<QueryDict: {}>,
COOKIES:{'__utma': '266053329.2142887685.1495792616.1500022766.1500022766.1',
 '_ga': 'GA1.2.2142887685.1495792616',
 '_gid': 'GA1.2.1628398333.1525088725',
 'csrftoken': '4YSve37bIcV2uyRnV5WM7qYZoYelyNG7',
 'phpbb3_a9mfv_k': '071de47f5f9c91fc',
 'phpbb3_a9mfv_sid': '5e6b2251fd77efdc8f958adc8cbd484f',
 'phpbb3_a9mfv_u': '2759',
 'style_cookie': 'null'},
META:{u'CSRF_COOKIE': u'4YSve37bIcV2uyRnV5WM7qYZoYelyNG7',
 'HTTP_ACCEPT': 'application/json, text/javascript, */*; q=0.01',
 'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
 'HTTP_ACCEPT_LANGUAGE': 'en-GB,en;q=0.5',
 'HTTP_CONNECTION': 'close',
...
 'HTTP_REFERER': 'http://web-dev-merge.openmicroscopy.org/webclient/',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:59.0) Gecko/20100101 Firefox/59.0',
 'HTTP_X_FORWARDED_FOR': '134.36.162.68',
 'HTTP_X_FORWARDED_PROTO': 'http',
 'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest',
 'PATH_INFO': u'/omero_parade/data/Uk9JX2NvdW50/',
 'QUERY_STRING': 'dataset=23611&_=1525368653608',
...

[chris-allan]

That error is similar to what @jburel saw on #21. I have not been able to reproduce this locally. Perhaps @will-moore could had an early JSON conversion to the data view with a try block which catches the TypeError and logs all the input and re-raises the exception. We can then use that information to find out what's wrong with the data being passed along.

Could be that the problem is numpy version compatibility. Can we check out which version of numpy web-dev-merge is running?

[kennethgillen]

Looks to me like numpy==1.7.1

[root@spider-2 ~]# ps auxf | grep -Po "\b.*merge.*?\b"
hudson   23406  0.0  0.1 213428 11632 ?        S    00:29   0:05 /home/hudson/virtualenv/bin/python
 /home/hudson/virtualenv/bin/gunicorn -D -p /opt/hudson/workspace/WEB-DEV-merge
[root@spider-2 ~]# su hudson
[hudson@spider-2 root]$ source /home/hudson/virtualenv/bin/activate
(virtualenv)[hudson@spider-2 root]$ /home/hudson/virtualenv/bin/pip freeze | grep num
numexpr==2.5.2
numpy==1.7.1

[chris-allan]

The minimum numpy version is set at 1.9 for OMERO.web in all the requirements files to my knowledge. Not that I would expect 1.7.1 to not work. Probably worth looking at @kennethgillen.

/cc @jburel, @joshmoore

[will-moore]

Hopefully should be fixed by #33