Display builder OPI

[hinxx]

Is there a way to convert, in bulk, existing AD OPI, from any known format (.edl, .opi, ?) to Display Builder (.bob)? Are there any plans on adding .bob files to ADCore/ADApp/op/.. folder?

[MarkRivers]

Ideally someone would write an medm to .bob converter, similar to the existing medm to .opi converter. medm is the only OPI with converters to everything else. It should not be too hard, since the opi converter can be used as a model.

Once that is done then typing “make” at the top level of areaDetector would create all .bob files.

[hinxx]

Answer from Kay Kasemir, pointed to by Oksana Ivashkevych:

This is about two different steps in the EDM -> OPI -> BOB pipeline.

The Eclipse-based CS-Studio has a command line option to convert EDM files to *.opi, see https://github.com/ControlSystemStudio/cs-studio/issues/2385:

css -nosplash
   -application org.csstudio.opibuilder.converter.edl
    /path/to/file1.edl
    /path/to/file2.edl 

Once you have a .opi file, ideally you don't need to do anything to open it in the Display Builder. But if you really prefer to have a .bob file, phoebus can do that:

phoebus -main org.csstudio.display.builder.model.Converter -output /tmp /path/to/example/tank.opi
Converting: /path/to/example/tank.opi => /tmp/tank.bob

I'll try to do this in my free time just to confirm that ADCore + 1 or 2 detectors can be successfully converted.

[MarkRivers]

Is this 2-step conversion likely to result in quality loss compared to a 1-step direct conversion to .bob?

[hinxx]

I have no idea. I need to give it a try, as I just learned the steps yesterday.

[hinxx]

Finally managed to find some free time to perform a batch conversion of BOY opi to BOB.

Here are commands, executed on Linux Mint using OpenJDK 12.0.1, with today's nightly build of Phoebus from https://controlssoftware.sns.ornl.gov/css_phoebus/.

Batch conversion command from already autoconverted BOY panels found in ADCore to BOB; NOTE: I had to use absolute paths to BOY files:

hinxx@obzen ~/ess/phoebus $ bash start_phoebus.sh -main org.csstudio.display.builder.model.Converter -output $(pwd)/opi/bob $(pwd)/opi/boy/*.opi
hinxx@obzen ~/ess/phoebus $ NOTE: Picked up JDK_JAVA_OPTIONS:  -DCA_DISABLE_REPEATER=true -Dnashorn.args=--no-deprecation-warning -Djdk.gtk.verbose=false -Djdk.gtk.version=2  -Dprism.forceGPU=true -Dlogback.configurationFile=/home/hinxx/ess/phoebus/logback.xml -Dorg.csstudio.javafx.rtplot.update_counter=false
2019-06-28 19:23:07 CONFIG [org.phoebus.product.Launcher] Loading settings from /home/hinxx/ess/phoebus/phoebus-0.0.1/settings.ini
Converting: /home/hinxx/ess/phoebus/opi/boy/ADAttrFile.opi => /home/hinxx/ess/phoebus/opi/bob/ADAttrFile.bob
Converting: /home/hinxx/ess/phoebus/opi/boy/ADBase.opi => /home/hinxx/ess/phoebus/opi/bob/ADBase.bob
Converting: /home/hinxx/ess/phoebus/opi/boy/ADBuffers.opi => /home/hinxx/ess/phoebus/opi/bob/ADBuffers.bob
[...]
Converting: /home/hinxx/ess/phoebus/opi/boy/NDStdArrays.opi => /home/hinxx/ess/phoebus/opi/bob/NDStdArrays.bob
Converting: /home/hinxx/ess/phoebus/opi/boy/NDTimeSeries.opi => /home/hinxx/ess/phoebus/opi/bob/NDTimeSeries.bob
Converting: /home/hinxx/ess/phoebus/opi/boy/NDTimeSeriesPlot.opi => /home/hinxx/ess/phoebus/opi/bob/NDTimeSeriesPlot.bob
Converting: /home/hinxx/ess/phoebus/opi/boy/NDTransform.opi => /home/hinxx/ess/phoebus/opi/bob/NDTransform.bob

Make sure that .bob file is loaded/used in actions/embedded displays instead of .opi:

hinxx@obzen ~/ess/phoebus $ sed -i -e 's/\.opi/.bob/' opi/bob/*.bob

Comparing file BOY vs. BOB disk usage:

hinxx@obzen ~/ess/phoebus $ du -hs opi/boy
5.9M    opi/boy
hinxx@obzen ~/ess/phoebus $ du -hs opi/bob
2.1M    opi/bob

Started adsimdetector with stock plugins loaded.

Starting phoebus while none of the UI options were changed, i.e. colors and fonts were left at defaults:

hinxx@obzen ~/ess/phoebus $ bash start_phoebus.sh -resource $(pwd)/opi/bob/ADTop.bob
hinxx@obzen ~/ess/phoebus $ NOTE: Picked up JDK_JAVA_OPTIONS:  -DCA_DISABLE_REPEATER=true -Dnashorn.args=--no-deprecation-warning -Djdk.gtk.verbose=false -Djdk.gtk.version=2  -Dprism.forceGPU=true -Dlogback.configurationFile=/home/hinxx/ess/phoebus/logback.xml -Dorg.csstudio.javafx.rtplot.update_counter=false
2019-06-28 19:23:57 CONFIG [org.phoebus.product.Launcher] Loading settings from /home/hinxx/ess/phoebus/phoebus-0.0.1/settings.ini
2019-06-28 19:23:57 INFO [org.phoebus.product.Launcher] Phoebus (PID 15404)
2019-06-28 19:23:57 WARNING [org.phoebus.ui.application.PhoebusApplication] Cannot locate logbook factory 'inmemory'
2019-06-28 19:24:07 INFO [org.phoebus.applications.update] Checking https://controlssoftware.sns.ornl.gov/css_phoebus/nightly/phoebus-linux.zip
2019-06-28 19:24:07 INFO [org.phoebus.applications.update] Current version  : 2019-06-28 23:59
[...]

For reference sake, contents of the start_phoebus.sh, copied and slightly edited from USPAS 2019 VM found at https://controlssoftware.sns.ornl.gov/training/2019_USPAS/:

hinxx@obzen ~/ess/phoebus $ cat start_phoebus.sh 
#!/bin/sh

export JAVA_HOME=$HOME/ess/phoebus/jdk-12.0.1
export PATH=$JAVA_HOME/bin:$PATH
cd $HOME/ess/phoebus/phoebus-0.0.1

JAR=product-0.0.1.jar

##HK OPT="-settings /home/training/epics-train/tools/phoebus.ini"
# To get one instance, use server mode
#OPT="$OPT -server 4918"

JDK_JAVA_OPTIONS=" -DCA_DISABLE_REPEATER=true"
JDK_JAVA_OPTIONS+=" -Dnashorn.args=--no-deprecation-warning"
JDK_JAVA_OPTIONS+=" -Djdk.gtk.verbose=false -Djdk.gtk.version=2  -Dprism.forceGPU=true"
JDK_JAVA_OPTIONS+=" -Dlogback.configurationFile=/home/hinxx/ess/phoebus/logback.xml"
JDK_JAVA_OPTIONS+=" -Dorg.csstudio.javafx.rtplot.update_counter=false"
export JDK_JAVA_OPTIONS

java -jar $JAR $OPT $SETTINGS "$@" &

[hinxx]

First impressions are very good, with more tests to follow.

[kasemir]

Make sure that .bob file is loaded/used in actions/embedded displays instead of .opi: hinxx@obzen ~/ess/phoebus $ sed -i -e 's/.opi/.bob/' opi/bob/*.bob

That should not be necessary. Assume you have a.bob with buttons or embedded displays that load b.opi. If there is a b.bob, it will be used instead of b.opi.

The idea is that the .opi files for the most part load OK, no conversion is necessary. In those cases where you do need to tweak them, you load the .opi in the editor, tweak, save (which saves it as a .bob file). You can then keep both b.opi and b.bob. The old BOY code will load the .opi. The display builder will give preference to the *.bob file.

[hinxx]

@kasemir, thanks for explaining! I was not aware that it would pick up (existing) BOB screen even if the embedded screen would point to BOY screen. Then again, I was trying to make a complete move from the BOY towards BOB file format as an exercise. Plus, our controls group at ESS is not keen on having BOY screens sitting around, for various reasons..

[MarkRivers]

OK to close this?

[hinxx]

I would like to add more information because it might make a difference.

Looking at the phoebus 4.6.0 doc (phoebus-4.6.0/doc/phoebus/app/display/convert-medm/doc/index.html) I noticed that EDM/MEDM screens can now be converted to BOB, without doing intermediate step of converting to OPI (something that might have been the case when this issue was opened). Phoebus was downloaded from https://controlssoftware.sns.ornl.gov/css_phoebus/.

@MarkRivers would you be open to adding autoconverted BOB files to areaDetector distro now that phoebus can do it directly from EDM/MEDM?

[MarkRivers]

@MarkRivers would you be open to adding autoconverted BOB files to areaDetector distro now that phoebus can do it directly from EDM/MEDM?

Sure. This assumes I can easily get a pre-built version of phoebus and that the conversion can be done from the command line.

For css I was getting frequent updates via the nightly builds as Kay fixed problems with the converter. I seem to recall reading that the css nightly builds were no longer working?

Are there nightly builds for phoebus in case we need to improve the converter?

[kasemir]

For what it's worth, you don't actually need to convert them beforehand. Sites that have EDM screens and start to use phoebus can configure the auto-conversion, so you simply open an EDM screen and it's converted. All you need to do is configure

org.csstudio.display.converter.edm/auto_converter_dir=/some/path

see https://github.com/shroffk/phoebus/blob/master/app/display/convert-edm/src/main/resources/edm_converter_preferences.properties

[kasemir]

This page which is also in the online help explains how to call the converters from the command line, and a little more on the EDM auto-converter: https://github.com/shroffk/phoebus/blob/master/app/display/convert-medm/doc/index.rst

You can get nightly binaries from https://controlssoftware.sns.ornl.gov/css_phoebus/

[kasemir]

I've given up on building the Eclipse-based CS-Studio. Those nightly builds on the SNS site end in May 2019, and I've been unable to build a functional binary since.

Kunal @shroffk , do you have nightly builds for the Eclipse-based CSS?

[shroffk]

Yes, I have one which builds the 4.6.4 with the phoebus integration. My build from last night was successful.

I use the following pipelines (without docker) https://github.com/cs-studio-nsls2/build

[MarkRivers]

Thanks. I don't think I'll need another nightly build of CSS since we don't be working on that converter anymore.

I will try to get the Phoebus conversions implemented. Although Kay points out that it can be done locally and transparently, it may be useful to have the converted files in a central location.

[shroffk]

Thanks. I don't think I'll need another nightly build of CSS since we don't be working on that converter anymore.

While there are very few changes being made to BOY, I hope that there will still be support for AD opi screens for the time being. NSLS2, FRIB, ITER, and many other sites are still in the process of migrating to BOB and will need the opi screens during the migration process.

I can work on making the nsls2 nightly build publicly available if necessary.

[prjemian]

As long as the conversion tool can be called from a linux command line, I'd be happy to also contribute them into the various synApps modules. I suggest putting them within a folder named bob. sibling to the adl, opi, and other folders with screens. The usual practice of having an autoconvert subdirectory will separate these autoconverted screens from any new screens designed within phoebus.

[kasemir]

You can call it from the command line, somewhat like this:

export JAVA_HOME=/path/to/openjdk11-or-later
export PATH=$JAVA_HOME/bin:$PATH

wget https://controlssoftware.sns.ornl.gov/css_phoebus/nightly/phoebus-linux.zip
unzip phoebus-linux.zip
rm phoebus-linux.zip

cd phoebus-4.6.0/
./phoebus.sh -main org.csstudio.display.converter.edm.Converter -help
./phoebus.sh -main org.csstudio.display.converter.medm.Converter -help

[MarkRivers]

While there are very few changes being made to BOY, I hope that there will still be support for AD opi screens for the time being.

Yes, to be clear I have no plans to stop supporting opi screens as well. As @prjemian suggested I will make new op/bob/autoconvert directories.

And if the css converter ever needs some fixes I'll be happy to update to the new version as long as there is a nightly build available.

[MarkRivers]

I have the autoconversion to Phoebus .bob files working in ADCore and ADSimDetector.

I have pushed the autoconverted files for these 2 modules to the master branch on Github.

This is a screen shot of simDetector.bob

[kasemir]

Good! I haven't head about MEDM conversion issues for a while, so I assume it's good enough for now. If not, let me know.

[MarkRivers]

I have now created .bob files for all repositories in areaDetector that contain .adl files. I also did ADPcoWin which only contains .edl files, so I converted from .edl.

Everything is pushed to Github.

[MarkRivers]

I am now adding bob files to all of the synApps modules. I am seeing one message that makes me think that the bob files will lose functionality compared to medm files.

2020-01-26 11:16:45 WARNING [org.csstudio.display.converter.medm] Message Button '>' release_msg='0' is ignored. Only writing press_msg='1' to $(P)pzt2_twf.VAL
2020-01-26 11:16:45 WARNING [org.csstudio.display.converter.medm] Message Button '>' release_msg='0' is ignored. Only writing press_msg='1' to $(P)pzt1_twf.VAL
2020-01-26 11:16:45 WARNING [org.csstudio.display.converter.medm] Message Button '<' release_msg='0' is ignored. Only writing press_msg='1' to $(P)pzt1_twr.VAL
2

Can Phoebus not handle mouse release events, or can the converter be improved?

[kasemir]

Our 'action button' does something when pressed: Open another file, write a PV, ... It doesn't do anything when released.

The 'bool button' writes 1/0 when pressed/released.

Can you attach one of the MEDM files that gets the message so I can check what MEDM widget gets mapped to a 'action button' and would better be mapped to a 'bool button'?

[MarkRivers]

medm has a "message button" widget that can send one string when pressed and another string when released. Many times it would map to a Phoebus "bool" button because the press message is 1 and the release message is 0. However, here is an example from one of the synApps medm files where the logic is opposite.

"message button" {
        object {
                x=142
                y=464
                width=211
                height=48
        }
        control {
                chan="ID11:bo3.VAL"
                clr=49
                bclr=30
        }
        label="Current OFF"
        press_msg="0"
        release_msg="1"
}

I have just done 2 searches through all of the synApps medm files (include my beamline-specific files). This file contains all occurrences of "press_msg" where the message is not 1. press.txt

This file contains all occurrences of "release_msg" where the message is not 0. release.txt

I think there are 4 cases to consider for the medm "message button": 1) Only a "press_msg": map to phoebus "action button". 2) Only a "release_msg": map to phoebus "action button". I don't think it will matter that the action is taken on press rather than release. 3) press_msg=1 and release_msg=0: map to phoebus "bool button" 4) All other combinations: this cannot be mapped to either the phoebus action or bool buttons. This is unfortunate. Could a new widget be added?

Here is another example of a medm "message button" that cannot be mapped to either "action button" or "bool button".

"message button" {
        object {
                x=915
                y=168
                width=45
                height=20
        }
        control {
                chan="13IDA:V2_close_request"
                clr=14
                bclr=0
        }
        label="Close"
        press_msg="Close_on"
        release_msg="Close_off"
        clrmod="alarm"
}

[kasemir]

Actually, the inverted case press_msg=0 and release_msg=1 can also map to the bool button because it has an 'inverted' mode. OK on also handling the case with only a 'release' message.

So the sole case that won't map is arbitrary press and release messages, like press_msg=42, release_msg=-6.28: In that case, we only perform the press_msg.

[MarkRivers]

I have completed the addition of .bob files to all areaDetector and synApps modules.

https://epics.anl.gov/tech-talk/2020/msg00299.php

[hinxx]

Thanks Mark!!

[kasemir]

Does anybody have MEDM binaries for RedHat 7(.7)? I have no way to experience the original *.adl screens, so hard to tell what's missing in the conversion. One thing that's certainly missing: It's converting "composite" widgets, but some files have top-level "children" elements that are not inside a composite. One example is synAps softMotorHelp.adl

[MarkRivers]

Does anybody have MEDM binaries for RedHat 7(.7)? I have no way to experience the original *.adl screens, so hard to tell what's missing in the conversion.

I could send you an medm binary for Centos 7, which would run on Redhat 7. However, it has all of these .so dependencies:

corvette:areaDetector/ADCore/iocBoot>ldd /usr/local/epics/extensions/bin/linux-x86_64/medm
        linux-vdso.so.1 =>  (0x00007fff6933c000)
        libca.so.3.14 => /usr/local/epics/base-3.14.12.6/lib/linux-x86_64/libca.so.3.14 (0x00007f3c84b7e000)
        libCom.so.3.14 => /usr/local/epics/base-3.14.12.6/lib/linux-x86_64/libCom.so.3.14 (0x00007f3c84923000)
        libXm.so.4 => /usr/lib64/libXm.so.4 (0x00007f3c84450000)
        libXt.so.6 => /usr/lib64/libXt.so.6 (0x00007f3c841e9000)
        libXp.so.6 => /usr/lib64/libXp.so.6 (0x00007f3c83fe0000)
        libXmu.so.6 => /usr/lib64/libXmu.so.6 (0x00007f3c83dc5000)
        libX11.so.6 => /usr/lib64/libX11.so.6 (0x00007f3c83a87000)
        libXext.so.6 => /usr/lib64/libXext.so.6 (0x00007f3c83875000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f3c8356e000)
        libm.so.6 => /usr/lib64/libm.so.6 (0x00007f3c8326c000)
        libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x00007f3c83056000)
        libc.so.6 => /usr/lib64/libc.so.6 (0x00007f3c82c89000)
        libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007f3c82a6d000)
        libreadline.so.6 => /usr/lib64/libreadline.so.6 (0x00007f3c82827000)
        librt.so.1 => /usr/lib64/librt.so.1 (0x00007f3c8261f000)
        libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007f3c8241b000)
        libXft.so.2 => /usr/lib64/libXft.so.2 (0x00007f3c82205000)
        libjpeg.so.62 => /usr/lib64/libjpeg.so.62 (0x00007f3c81fb0000)
        libpng15.so.15 => /usr/lib64/libpng15.so.15 (0x00007f3c81d85000)
        libSM.so.6 => /usr/lib64/libSM.so.6 (0x00007f3c81b7d000)
        libICE.so.6 => /usr/lib64/libICE.so.6 (0x00007f3c81961000)
        libXau.so.6 => /usr/lib64/libXau.so.6 (0x00007f3c8175d000)
        libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x00007f3c81535000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3c84de8000)
        libtinfo.so.5 => /usr/lib64/libtinfo.so.5 (0x00007f3c8130b000)
        libfontconfig.so.1 => /usr/lib64/libfontconfig.so.1 (0x00007f3c810ce000)
        libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x00007f3c80e0f000)
        libXrender.so.1 => /usr/lib64/libXrender.so.1 (0x00007f3c80c04000)
        libz.so.1 => /usr/lib64/libz.so.1 (0x00007f3c809ee000)
        libuuid.so.1 => /usr/lib64/libuuid.so.1 (0x00007f3c807e9000)
        libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x00007f3c805bf000)
        libbz2.so.1 => /usr/lib64/libbz2.so.1 (0x00007f3c803af000)

That means you need to install the packages that provide libXm.so, libXp.so, etc. Once you do that you might as well build medm from source. That is easy once you have the required packages installed.