Import from blissdata instead of bliss

[woutdenolf]

The blissdata package is a much lighter dependency than bliss. It is available on pypi and anaconda.

# PyMca5/PyMcaCore/RedisTools.py

from bliss.config import get_sessions_list
from bliss.config.settings import scan as rdsscan
from bliss.data.node import get_node, get_nodes

# PyMca5/PyMcaCore/RedisTools.py

from bliss.config import get_sessions_list  # TODO
from blissdata.settings import scan as rdsscan
from blissdata.data.node import get_node, get_nodes

get_sessions_list is unfortunately not in blissdata. I'll try to make that happen.

FYI: soon blissdata will also work without gevent.

[vasole]

Did you get a get_sessions_list replacement? It is to merge #1039 and get rid of the current deprecation warning.

[mguijarr]

Hi there 😄

I would like to revive this issue, I would be interested to have PyMca working with the new blissdata (blissdata > 1.0.0) here at PSI.

How can I help with this ?

About the sessions list: personally I would make it as a command line option, to specify the list of sessions we are interested in. No discovery.

[woutdenolf]

personally I would make it as a command line option, to specify the list of sessions we are interested in. No discovery.

Discovery is mandatory imo.

[mguijarr]

personally I would make it as a command line option, to specify the list of sessions we are interested in. No discovery.

Discovery is mandatory imo.

I see it as a problem, for people who do not use Beacon or BLISS yaml configuration files. How to identify a session?

It could also be, that the "Data source" button prompts user for the session name. Users know the name of the BLISS session they use. The successful attempts to connect to a session could then be stored in PyMCA local preferences configuration (if it exists ?), to provide selection via a combo box to speed up next times...

It could also be a command line argument like --bliss-sessions sixc,optics for example.

It is important to take into account, that if BEACON_HOST is undefined it should need to look at REDIS_DATA_URL to know how to reach Redis (or a command line argument --redis).

[woutdenolf]

Sorry, I don't agree with pushing complexity onto third-party applications. Blissdata could easily provide a function that returns a list of currently available BLISS sessions.

[mguijarr]

Ok.

What do you propose in the case when there is no Beacon ? No YAML files, no way to know sessions. Maybe it is possible to discover sessions from recorded data in Redis ?

[vasole]

I agree with @woutdenolf that the available sessions should be discoverable.

BTW, just by chance (untested) does pymca -f --spec=NAME_OF_BLISS_SESSION --shm=None help?

[mguijarr]

Well, at the moment newest blissdata >= 1.0.0 does not work at all because those imports fail:

from blissdata.settings import scan as rdsscan
from blissdata.data.node import get_node, get_nodes

(in src/PyMca5/PyMcaCore/RedisTools.py )

So, I suppose it does not work at ESRF neither - people use files apparently.

[mguijarr]

available sessions should be discoverable.

I had a deeper look at this - blissdata can already communicate with Beacon (the beamline configuration server), so far so good, so we make queries. As of today, though, there is no knowledge of BLISS objects (Session is one) on the server side. Everything is interpreted in the client (which is good, and bad, depending what has to be done !).

I think blissdata should not import bliss itself. So, the client logic has to be reimplemented in blissdata for finding sessions from yaml files contents.

But it is not trivial. get_sessions_list code does:

    for name in config.names_list:
        c = config.get_config(name)
        if c.get("class") != "Session":
            continue
        if c.get_inherited("plugin") != "session":
            continue
        all_sessions.append(name)

It is going through indexed elements (pieces of yaml with name: something), checking if a class member with Session is present ; it also checks if it is the good configuration loader (aka "plugin"), by looking in the inherited fields of the indexed element... Maybe this last check could be skipped to simplify.

A better bet is maybe to rely on the Configuration web application. It is always present at ESRF beamlines. In this case getting sessions list is as easy as a GET request: http://localhost:9030/config/tree/sessions

Personally I would go for this for having sessions discoverable from blissdata.

Still what to do if there is no Beacon ? I see PyMca already has the third-party application logic pushed to it with --spec and --shm ; we could do as suggested by @vasole and hijack --spec to explicitly specify BLISS sessions on the command line.

[vasole]

Well, ESRF people were never encouraged to use PyMca for online data despite offering features not offered by flint. So I do not find surprising that something got broken (and unnoticed) with recent bliss developments.

The funny thing is that today PyMca can access bluesky data but not bliss.

[mguijarr]

The funny thing is that today PyMca can access bluesky data but not bliss.

Are you referring to https://github.com/vasole/pymca/pull/1064 ?

It is not merged, and only a draft.

I would like to fix the direct BLISS support, the code in RedisTools.py is less than 400 lines... I am not very familiar with this, though.

[vasole]

Are you referring to #1064 ?

It is not merged, and only a draft.

No, I am talking about https://github.com/vasole/pymca/pull/1074 and https://github.com/vasole/pymca/pull/1075 that are merged and that support bluesky tiled via silx.

[mguijarr]

What is the link between PyMca and silx ? I mean, if it brings more benefits (like, having it compatible with silx view for example) instead of having BLISS support in PyMca like it was done, it would maybe be useful to go through silx like it is done for Tiled ?

I am a bit lost.

[vasole]

Hi,

The direct reading via blissdata should be operational.

I guess it got broken when trying to use blissdata:

https://github.com/vasole/pymca/pull/1039

Perhaps using the direct access to Redis still works. I had foreseen an ImportError but if you are getting a different exception the fallback is not used.

[mguijarr]

Perhaps using the direct access to Redis still works.

No, it's not possible it has completely changed - different way to organize data, different structures, different keys... Everything

It got broken when passing from "old" blissdata to version 1.0

[vasole]

OK. Then the access via blissdata has to work,.

[vasole]

The implemented support was very basic. It was dealing with the equivalent of the SPEC file part of the data (motors, counters and MCAs) because I was wrapping the information as a SPEC file. If you know how to obtain that information (scans, motors, counters and MCAs) via blissdata it should be easy to adapt RedisTools.py

A complete support would imply accessing blissdata via an h5py API, but I think/hope that it is foreseen as a blissdata feature at some point.

[woutdenolf]

What do you propose in the case when there is no Beacon ?

Why Beacon? Everything is in Redis.

[mguijarr]

What do you propose in the case when there is no Beacon ?

Why Beacon? Everything is in Redis.

Sessions list is not in Redis ? Or maybe I miss something.

But, if there a way to introspect Redis to know live sessions : so far so good. I am all for it.

[vasole]

@mguijarr

The branch redis https://github.com/vasole/pymca/tree/redis contains some preliminary work with session discovery.

The button to access the session should work and a preliminary list of scans should work. If that is not the case it means some files are still to be copied from my side.

I was using scan.info but it is not documented enough.

This is the feedback I got from @woutdenolf and a pointer about where to look at:

scan.info is basically a big dictionary without any formal rules about its structure.

This class is a start, perhaps you find it useful:

from blissdata.schemas.scan_info import ScanInfoDict