search

ome / omero-mapr

An OMERO.web app allowing to browse the data through attributes linked to the image
https://pypi.org/project/omero-mapr/
GNU Affero General Public License v3.0
5 stars 12 forks source link

Reactivate infra #57

Closed will-moore closed 4 years ago

will-moore commented 4 years ago

This tests omero-test-infra when the omero_mapr/__init__.py imports the urls.py. I think we often don't see the failure of urls.py to import because it doesn't happen when the omero_mapr module itself is imported. This is on top of #56 (alternative sibling PR since we are both working on this)

will-moore commented 4 years ago

After that last commit I see:

+ python .//setup.py test
testimport urls.py...
ImportError
running pytest

but no stacktrace to show how the import failed.

However, I see this later:

Running PyYAML-5.2b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-8zzdcwu0/PyYAML-5.2b1/egg-dist-tmp-rvh9xhou
Traceback (most recent call last):
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 33, in <module>
    from omero import ApiUsageException, ServerError
ImportError: cannot import name 'ApiUsageException'

However, running .omero/docker app locally I instead see the same import error at a different point

+ python setup.py clean
testimport urls.py...
ImportError
running clean
Traceback (most recent call last):
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 35, in <module>
    from omero import ApiUsageException, ServerError
ImportError: cannot import name 'ApiUsageException'
will-moore commented 4 years ago

Locally, this fails:

>>> from omero import ApiUsageException
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name ApiUsageException

but works if you first do import omero.clients (which fails flake8)!

After adding import omero.clients in the last commit above, we now fail travis with

+ python setup.py clean
testimport urls.py...
Traceback (most recent call last):
  File "setup.py", line 38, in <module>
    version = __import__('omero_mapr').get_version()
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 38, in <module>
    from .mapr_settings import mapr_settings
  File "/omero-mapr/omero_mapr/mapr_settings.py", line 73, in <module>
    TEMPLATES = getattr(settings, 'TEMPLATES', [])
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting TEMPLATES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
will-moore commented 4 years ago

After removing the usage of ApiException and ServerError, we are still seeing 

+ python setup.py clean
testimport urls.py...
Traceback (most recent call last):
  File "setup.py", line 38, in <module>
    version = __import__('omero_mapr').get_version()
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 35, in <module>
    from .mapr_settings import mapr_settings
  File "/omero-mapr/omero_mapr/mapr_settings.py", line 73, in <module>
    TEMPLATES = getattr(settings, 'TEMPLATES', [])
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting TEMPLATES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
will-moore commented 4 years ago

Failing with:

+ python setup.py clean
ERROR:omeroweb.webgateway.views:No numpy installed
testimport urls.py...
Traceback (most recent call last):
  File "setup.py", line 41, in <module>
    version = __import__('omero_mapr').get_version()
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 36, in <module>
    DEFAULT_CONFIG = list(mapr_settings.CONFIG.keys())[0]
IndexError: list index out of range
snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#101. See the console output for more details. Possible conflicts:

--conflicts

will-moore commented 4 years ago

To test locally need to

docker pull openmicroscopy/omero-web:5.6
will-moore commented 4 years ago 
NOCLEAN=true .omero/docker app
.omero/compose exec web bash
/opt/omero/web/OMERO.web/bin/omero config get | grep mapr

shows nothing, so it looks like the mapr config didn't get picked up, which could explain why 

DEFAULT_CONFIG = list(mapr_settings.CONFIG.keys())[0]
IndexError: list index out of range
snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#102. See the console output for more details. Possible conflicts:

--conflicts

will-moore commented 4 years ago

@joshmoore @manics Any idea how to debug the reading of the config file in https://github.com/ome/omero-mapr/blob/master/.omeroci/app-config to see why this might not be read by omero-test-infra?

joshmoore commented 4 years ago

I updated .omeroci/app-config to print "XXX". I found a "config.xml not found issue". (This would have turned into a hard-failure with your recently merged PR). https://github.com/ome/omero-test-infra/pull/52 fixes the issue. With that PR, however, I'm seeing:

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1050, in <module>
    'DEPRECATED_SETTINGS_MAPPINGS')
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1039, in process_custom_settings
    (global_name, key, global_value, e.message, description))
AttributeError: 'JSONDecodeError' object has no attribute 'message'
+ clean_up
snoopycrimecop commented 4 years ago

Conflicting PR. Removed from build OMERO-plugins-push#105. See the console output for more details. Possible conflicts:

--conflicts

will-moore commented 4 years ago

With the latest fix to omero-test-infra: https://github.com/ome/omero-test-infra/pull/53 and last commits above, I now see tests running locally but Error on import. Tried importing to another server (don't know login for docker-compose omero):

$ NOCLEAN=true .omero/docker app
...
=============== 12 passed, 1 xpassed, 28 error in 11.74 seconds ================

$ .omero/compose exec web bash
bash-4.2$ cd /opt/omero/web/OMERO.web/
2$ bin/omero import /opt/omero/web/venv3/lib/python3.6/site-packages/omeroweb/webclient/static/webclient/image/table_layout.png
Server: [localhost:4064]idr1-slot2.openmicroscopy.org
Username: [omero-web]user-3
Password:
Created session for user-3@idr1-slot2.openmicroscopy.org:4064. Idle timeout: 10 min. Current group: read-only-1
Traceback (most recent call last):
  File "/opt/omero/web/venv3/bin/omero", line 131, in <module>
    rv = omero.cli.argv()
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1756, in argv
    cli.invoke(args[1:])
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1187, in invoke
    stop = self.onecmd(line, previous_args)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1264, in onecmd
    self.execute(line, previous_args)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1346, in execute
    args.func(args)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/plugins/import.py", line 518, in importer
    self.do_import(command_args, xargs)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/plugins/import.py", line 529, in do_import
    stdout=out, stderr=err)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/java.py", line 140, in popen
    check_java(command)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/java.py", line 31, in check_java
    raise Exception("Java could not be found. (Executable=%s)" % command[0])
Exception: Java could not be found. (Executable=java)

Don't know if this is the same as errors in tests:

        out, err = popen.communicate()
        rc = popen.wait()
        if rc != 0:
            raise Exception("import failed: [%r] %s\n%s\n%s" % (
>               args, rc, out, err))
E           Exception: import failed: [['/opt/omero/web/venv3/bin/python', './bin/omero', '-s', 'omero', '-k', '9084df87-51f5-4922-8cc2-b707882f92d9', '-p', '4064', 'import', '--skip', 'all', '--output', 'legacy', '--', path('/opt/omero/web/OMERO.web/var/omero/tmp/omero_omero-web/336/import_fake_file_01zyg1mji&&plates=1&plateAcqs=1&plateCols=6&plateRows=1&fields=1.fake')]] 1
E           b''
E           b'  File "./bin/omero", line 2\n    . /opt/omero/web/venv3/bin/activate\n    ^\nSyntaxError: invalid syntax\n'
manics commented 4 years ago

Which version of the omero-web docker image are you running?

will-moore commented 4 years ago

@manics Don't know. Whatever is specified by omero-test-infra.

will-moore commented 4 years ago

Travis is green. 😄 I assume the tests are running and passing (but don't see them mentioned in the console).

will-moore commented 4 years ago

Going to tidy up and open another clean PR