Add client support for an OMERO auto-connection file on public websites

manics commented 4 years ago

Background

For a long time the IDR has recommended the use of a custom connection method instead of the standard OMERO.py client or gateway methods. This was originally done to allow load-balancing since the OMERO binary protocol does not allow session pinning on a front end proxy:

https://github.com/IDR/deployment/blob/IDR-0.8.2/ansible/idr-haproxy.yml#L3-L10

We can't do proper session pinning with the OMERO binary protocol because there's no way to insert or extract an identifier, so instead we assign separate front-end ports for each back-end server and rely on the client to choose a random port for each connection.

To support autoconfiguration of clients we also create a json config file with a list of connections parameters that clients should randomly choose from.

This custom connection method fetches a json configuration file from the IDR website if a base hostname is provided. For example, conn = connection('idr.openmicroscopy.org', ...) will attempt to download a file connection/omero-client.json from the host using https, in this case https://idr.openmicroscopy.org/connection/omero-client.json

This connection file effectively allows custom configuration to be pushed to the clients- normally this would require code changes in the client to override the defaults, or a knowledgable user who knows and remembers to pass additional configuration to their client.

Most recently this is being used to provide a default username and password to connect to the IDR:

https://github.com/IDR/deployment/pull/246

Future benefits to us or others include:

Provide an easy to remember hostname e.g. example.org for users to configure their clients, whilst deploying OMERO.server on a different address- there is no requirement for connection/omero-client.json to reside on the same host or domain as OMERO.server
Easily switch the public credentials if necessary for any reason without actively notifying clients
Provide a standard way for other public resources to provide public OMERO API access
Ability to force all clients to connect using an alternative transport such as wss (websockets)

Proposal

Add support for https://HOSTNAME/connection/omero-client.json to OMERO.py clients and BlitzGateway.
Optionally add support to clients in other languages.

Alternative: `.well-known`

.well-known is standard for defining well-known locations on a web server providing metadata:

https://en.wikipedia.org/wiki/List_of_/.well-known/_services_offered_by_webservers

We could consider using something like https://HOSTNAME/.well-known/omero/client.json instead.

joshmoore commented 4 years ago

I didn't know of .well-known but following some form of convention makes sense.
I'm certainly for also adding to the Java Gateway. Unsure if it's worth adding to C++ at the moment. cc: @dominikl
One thing it's perhaps pre-designing for is: what happens when/if login moves to being HTTP based rather than Ice based?

manics commented 4 years ago

If we move to http login I think this would still work, since client.json is a set of overrides, so the following are all possible

omero-client.json refers to the same host and port
omero-client.json is empty [] so nothing is overridden
client gets a 404 in response to the omero-client.json request and proceeds with a normal login

joshmoore commented 4 years ago

Hmmm...but "port" probably doesn't make any sense via HTTP. I wonder then if it shouldn't be a different endpoint:

/$prefix/ice-client.json
/$prefix/http-client.json

though maybe in a future HTTP world this all just goes away.

manics commented 4 years ago

If BlitzGateway is updated to match idr.connection it means handling a one-argument form differently from the multi-argument form:

Currently BlitzGateway('idr.openmicroscopy.org', <other parameters ...>) connects to idr.openmicroscopy.org:4064
The single argument form BlitzGateway('idr.openmicroscopy.org') would fetch omero-client.json. Is this OK?

jburel commented 4 years ago

what about the websocket case which is now critical for many notebooks/data access? everything included in omero-client.json? i.e.

[
"--Ice.Default.Router=OMERO.Glacier2/router:wss -p 443 -h idr.openmicroscopy.org -r /omero-ws",
"--omero.user=xxx",
"--omero.pass=xxx"
]

manics commented 4 years ago

Yes, that should also work if we want to switch everyone to using websockets. IIRC we've tested that before.

joshmoore commented 4 years ago

https://github.com/ome/design/issues/106#issuecomment-651832358 Is this OK?

I guess since this optionally gathers the json login info and then falls back to previous behavior (?) it would be ok. Not sure if it wouldn't be safer to have a separate class to do the next-gen login. (Another option might be something like BlitzGateway(omero.client(host))...)

manics commented 4 years ago

Existing behaviour 😀

>>> from omero.gateway import BlitzGateway
>>> c = BlitzGateway('idr.openmicroscopy.org')
Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/omero/clients.py", line 297, in _initData
    self.__ic = Ice.initialize(id)
  File "/venv/lib/python3.8/site-packages/Ice.py", line 705, in initialize
    communicator = IcePy.Communicator(args, data)
Ice.EndpointParseException: exception ::Ice::EndpointParseException
{
    str = unrecognized argument `not' in endpoint `ssl -p 4064 -h <"omero.host" not set>'
}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/lib/python3.8/site-packages/omero/gateway/__init__.py", line 1594, in __init__
    self._resetOmeroClient()
  File "/venv/lib/python3.8/site-packages/omero/gateway/__init__.py", line 2144, in _resetOmeroClient
    self.c = omero.client(
  File "/venv/lib/python3.8/site-packages/omero/__init__.py", line 67, in client
    return omero.clients.BaseClient(*args, **kwargs)
  File "/venv/lib/python3.8/site-packages/omero/clients.py", line 170, in __init__
    self._initData(id)
  File "/venv/lib/python3.8/site-packages/omero/clients.py", line 301, in _initData
    raise omero.ClientError(msg)
omero.ClientError: No host specified. Use omero.client(HOSTNAME), ICE_CONFIG, or similar.

jburel commented 4 years ago

That's the problem I mentioned on Monday and we adjust the notebook accordingly before switching to websockets

manics commented 4 years ago

@jburel Could you explain a bit more? I don't see how it's related to websockets.

jburel commented 4 years ago

it is not. I first had to adjust the way to connect then we started to use binder for most notebooks so I had to adjust anyway.

ome / design