script py3 fixes - Githubissues

will-moore commented 4 years ago

Fixes adding Shapes to panels when exporting as TIFFs. To test:

Add a bunch of ROIs to some image panels
Export as TIFF

will-moore commented 4 years ago

Travis tests are failing with ModuleNotFoundError: No module named 'numpy' so we need numpy for tests. I was looking to see where this is handled in omero-mapr, which has the same requirement, but I don't see it?

joshmoore commented 4 years ago

Add it to setup.py? Pretty sure I rolled back my commit since you said it wasn't necessary. (https://github.com/ome/omero-mapr/commit/328c51ef73918ae0b200f300bf4afa086e1a45eb)

will-moore commented 4 years ago

Re: https://github.com/ome/omero-mapr/commit/328c51ef73918ae0b200f300bf4afa086e1a45eb That was due to a 'warning' message from webgateway since it uses numpy for some things, but they weren't being used for mapr. However, as discussed - since BlitzGateway uses numpy for some things, we should add it as a requirement of omero-py. See https://github.com/ome/omero-py/pull/139

snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#108. See the console output for more details. Possible conflicts:

PR #351 will-moore 'Handle unicode figure names in export'
- setup.py

--conflicts

will-moore commented 4 years ago

Tried exploring omero-test-infra...

$ NOCLEAN=true .omero/docker app
...
running build_ext
Traceback (most recent call last):
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1028, in process_custom_settings
    setattr(module, global_name, mapping(global_value))
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

$ .omero/compose exec web bash
bash-4.2$ whoami
omero-web

./omero config get
cd ../lib
./python3
>>> import os
>>> os.sys.path
['', '/opt/omero/web/OMERO.web/lib/python', '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/opt/omero/web/venv3/lib64/python3.6/site-packages', '/opt/omero/web/venv3/lib/python3.6/site-packages']

pwd
/opt/omero/web/venv3/lib64
bash-4.2$ ../bin/python3
>>> import omero
>>> from omeroweb import settings 
Traceback (most recent call last):
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1028, in process_custom_settings
    setattr(module, global_name, mapping(global_value))
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1043, in <module>
    'DEPRECATED_SETTINGS_MAPPINGS')
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1032, in process_custom_settings
    (global_name, key, global_value, e.message, description))
AttributeError: 'JSONDecodeError' object has no attribute 'message'

BUT, can't edit python code (not root)....

$ .omero/compose exec -u root web bash
[root@22fb5e33fb9d setup]# whoami
root
# ps -aux | grep django
..../opt/omero/web/venv3/bin/python3  .... /opt/omero/web/OMERO.web/
# cd /opt/omero/web/venv3/bin
# python3
>>> import omero .  # ModuleNotFound

# cd /opt/omero/web/venv3/
# bin/python3

vi /opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py

Found with print(global_value) in omeroweb/settings.py process_custom_settings() that omero.web.apps value [omero_figure] was not quoted correctly (should be ["omero_figure"]

snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#109. See the console output for more details. Possible conflicts:

PR #351 will-moore 'Handle unicode figure names in export'
- setup.py

--conflicts

snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#110. See the console output for more details. Possible conflicts:

PR #351 will-moore 'Handle unicode figure names in export'
- setup.py

--conflicts

will-moore commented 4 years ago

Figure export tests are failing on travis due to missing reportlab. Locally I find it is available on the server container in py2 but not Python-3:

# remove any old docker containers
$ docker rm -f $(docker ps -aq)
$ NOCLEAN=true .omero/docker app --verbose
# shell into server container...
$ .omero/compose exec omero bash

bash-4.2$ cd /opt/omero/server/OMERO.server/
bash-4.2$ ls
LICENSE.txt  bin  etc  history.rst  lib  omero.envvars  share  sql  var
bash-4.2$ python
Python 2.7.5 (default, Aug  4 2017, 00:39:18) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import reportlab

# find where omero is running - python3
$ ps -aux | grep omero
omero-s+   141  0.1  0.0 231300    72 ?        S    14:17   0:01 /opt/omero/server/venv3/bin/python /opt/omero/server/OMERO.server/bin/omero admin start --foreground

bash-4.2$ /opt/omero/server/venv3/bin/python
Python 3.6.8 (default, Aug  7 2019, 17:28:10) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import reportlab 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'reportlab'

Reportlab is installed in ./.omeroci/app-srv with

yum -y install python-reportlab python-markdown

Need to ensure it is installed for python3...

will-moore commented 4 years ago

Looks like /opt/omero/server/venv3/bin/pip install reportlab should work...

jburel commented 4 years ago

it is probably python3-reportlab

jburel commented 4 years ago

now that everything is installed in virtualenv the steps for figure should be split into 2:

install figure in web venv
install script in server venv with its dependencies

will-moore commented 4 years ago

Locally I'm seeing an issue with install of scripts:

>       assert script_service.getParams(id) is not None

test/integration/test_figure_scripts.py:51: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = 13cc1d5d-9208-47ea-ba37-e1acfc79c78c/e59c8e23-35ec-4d74-8d42-d8d629844fd7omero.api.IScript -t -e 1.1:tcp -h 172.22.0.4 -p 36937 -t 60000
scriptID = 63, _ctx = None

    def getParams(self, scriptID, _ctx=None):
>       return _M_omero.api.IScript._op_getParams.invoke(self, ((scriptID, ), _ctx))
E       omero.InternalException: exception ::omero::InternalException
E       {
E           serverStackTrace = 
E           serverExceptionClass = 
E           message = Sha1s don't match! expected 4cfaf0e9cbf68fb5dc25bc2816f8c3b83a7248e7, found cd16eafce5be6f5f47bf66ce78d6ac730115ce88
E       }

/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_api_IScript_ice.py:842: InternalException
====================== 5 failed, 1 passed in 6.85 seconds ======================

snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#142. See the console output for more details. Possible conflicts:

PR #351 will-moore 'Handle unicode figure names in export'
- .omeroci/app-srv
- setup.py

~~--conflicts~~ Conflict resolved in build OMERO-plugins-push#113. See the console output for more details.

snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#127. See the console output for more details. Possible conflicts:

PR #351 will-moore 'Handle unicode figure names in export'
- setup.py
- .omeroci/app-srv

~~--conflicts~~ Conflict resolved in build OMERO-plugins-push#114. See the console output for more details.

snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#128. See the console output for more details. Possible conflicts:

PR #351 will-moore 'Handle unicode figure names in export'
- .omeroci/app-srv
- setup.py

~~--conflicts~~ Conflict resolved in build OMERO-plugins-push#144. See the console output for more details.

jburel commented 4 years ago

Tested the functionality of this PR Figure exported as TIFF contains the various shapes and the settings are preserved. @will-moore will first need to sort out this PR before we can merge

will-moore commented 4 years ago

@jburel Rebased to have only my 1-character fix!

jburel commented 4 years ago

Retested today. It works as expected Merging

ome / omero-figure

script py3 fixes #354