Load xhptdc8_manager_configuration from YAML

sulimma commented 3 years ago

The purpose of this repository is to make using the xHPTDC8 time-to-digital converter simpler to use for end users.

Setting up the xhptdc8_manager_configuration structure is rather complicated. This shall be simplified by allowing to apply YAML files to the structure. So the workflow for the user would be:

xhptdc8_manager_configuration *cfg = new xhptdc8_manager_configuration;
xhptdc8_get_default_configuration(xhptdc8_mgr, cfg);
xhptdc8_apply_yaml(cfg, yaml_string);
xhptdc8_configure(hptdc8_mgr, cfg);

The YAML can describe an incomplete structure. When a structure member is present it is overwritten with the value from the YAML. Members that are missing in the YAML are left unchanged.

YAML

Structures shall be represented by maps, with the map key beeing the name of the structure member. Arrays shall also be represented by maps with the key beeing an integer. If the key -1 is specified the value shall be applied to all array elements. Bound checks shall be applied when parsing arrays.

The following YAML string will configure a simple grouping setup.

# this is a comment
test: "Hello"   # everything outside the manager is ignored 

manager:
  grouping:
    trigger_channel: 5
    range_start: 1000
    range_start: 1000000  
    enable: true  #enable is missing in header and documentation but will follow

Anchors and aliases shall be supported, which at least some YAML libraries do automatically:

# define 9 channels of CMOS offsets
dc_offsets: &CMOS
  -1: 1.18
# apply CMOS input settings to all 8 TDCs 
manager:
  device_configs: 
    -1: 
      dc_offset: *CMOS

Here you can experiment with YAML anchors and other features: http://yaml-online-parser.appspot.com/

Library Structure

Please help me decide which of the following options are best: a) Output a single .dll that contains both the original driver and the utility functions in a single dll b) Create a separate dll, users link with both dlls.

YAML parser

The easist way to parse YAML in C++ seems to be this library: https://github.com/jbeder/yaml-cpp/wiki/Tutorial Speed is not an issue as the parsed files are small. Setup the compile scripts to obtain the library from vcpkg or another package source before compilation.

Bassem-Ramzy commented 3 years ago

A wiki page is created to discuss the Library Structure

Bassem-Ramzy commented 3 years ago

@sulimma: _dcoffset in the example should be replaced with threshold as per the latest update

Bassem-Ramzy commented 3 years ago

RapidYAML: is more recent, and (claimed as) "faster" YAML parser; is found as well

sulimma commented 3 years ago

Speed is of no importance. I would say number one priority is ease of use. You can choose any library that is compatible to the Mozilla License.

sulimma commented 3 years ago

RapidYAML also seems reasonably simple to use, so please choose whatever you like best as a programmer.

sulimma commented 3 years ago

Please note that the arrays shall be implemented as maps not as sequences. The reason is that it shall be possible to assign only part of the array, which is not possible for sequences.

Bassem-Ramzy commented 3 years ago

Manager Configuration YAML Draft Documentation:

manager_config:
  device_configs:       # xhptdc8_manager_configuration sequence of 8 (configurations) elements  
  -                                     # xhptdc8_manager_configuration first element
   auto_trigger_period: 20      # integer, Create a trigger either periodically or randomly
   auto_trigger_random_exponent: 20 # integer, Create a trigger either periodically or randomly
   trigger_threshold:       # double[8], Set the threshold voltage for the input channels A . . .H and TRG, (-1.32V to +1.18V).
    - 0.3499                # decimal (double), Threshold 0 for Channel A
    # Next element
    # And, so on to threshold 7
    - 0.3499                # decimal (double), Threshold 7 for Channel H
    - 0.3499                # decimal (double), Threshold 8 for TRG
   trigger:         # Configures the polarity of the external trigger sources. Sequence size is 16
    - 
     falling: false         # (true, false), determines whether falling edges on the inputs create trigger  
     rising: true           # (true, false), determines whether rising edges on the inputs create trigger 
    # Next element
    # And, so on
    # Up to 16 elements in the sequence
   gating_block:        # xhptdc8_tiger_block sequence of 8 (gates) elements 
    -                               # xhptdc8_tiger_block
     mode:  0           # int, Enables the desired mode of operation for the tiger
     negate:    false           # (true, false), Inverts output polarity. Default=false.
     retrigger: false           # (true, false), Enables retrigger setting
     extend:    false           # (true, false), Not implemented.
     start: 0           # integer, The time during which the TiGer output is set
     stop:  0           # integer, The time during which the TiGer output is set
     sources:   1           # integer, A bit mask with a bit set for all trigger sources that can trigger this TiGer block. Default=1
    # Next element
    # And, so on
    # Up to 8 elements in the sequence
   tiger_block:         # xhptdc8_tiger_block sequence of 8 (TiGers) elements
    -                               # xhptdc8_tiger_block
     mode:  0           # int, Enables the desired mode of operation for the tiger
     negate:    0           # (true, false), Inverts output polarity. Default=false.
     retrigger: 0           # (true, false), Enables retrigger setting
     extend:    0           # (true, false), Not implemented.
     start: 0           # integer, The time during which the TiGer output is set
     stop:  0           # integer, The time during which the TiGer output is set
     sources:   1           # integer, A bit mask with a bit set for all trigger sources that can trigger this TiGer block. Default=1
    # Next element
    # And, so on
    # Up to 8 elements in the sequence
   channel :            # xhptdc8_channel sequence of 8 (channels) elements
    -                                   # xhptdc8_channel
     enable: false          # (true, false), Enable TDC channel  
     rising: true           # (true, false), Select which edge of the signal is measured by the TDC
    # Next element
    # And, so on
    # Up to 8 elements in the sequence
   adc_channel:         # xhptdc8_adc_channel
    enable: true            # (true, false)
    watchdog_readout: true      # (true, false)
    watchdog_interval: 0        # integer
    trigger_threshold: 0        # decimal (double)
   skip_alignment: 0        # (true, false)
   alignment_source: 0      # integer
   grouping :           # xhptdc8_grouping_configuration
    enabled:    true            # (true, false) Enable grouping. Default=false
    trigger_channel: 0          # integer, Channel number that is used to trigger the creation of a group.
    zero_channel:   2       # integer, Optionally a different channel can be used to calculate the relative timestamps in a group. Default=-1 (disables)
    zero_channel_offset: 0      # integer, in picoseconds is added to relative timestamps within a group.
    range_start:    0       # integer, Values in the interval from range_start to range_stop are included in the group
    range_stop: 0           # integer, Values in the interval from range_start to range_stop are included in the group
    trigger_deadtime: 0         # integer, After a trigger was detected additional triggers will be suppressed for this interval. Must not be negative.
    require_window_hit: 0       # (true, false), If set a group is only created if there is at least one hit in the window defined by window_start and window_stop
    window_start: 0         # integer
    window_stop:    0       # integer
    veto_mode:  0           # integer, A window defined by veto_start and veto_stop can be used to suppress hits
    veto_start: 0           # integer
    veto_stop:  0           # integer
    veto_relative_to_zero: 0        # (true, false), If set, the veto window is defined relative to the zero channel
    overlap:    false           # (true, false), Unsupported, must remain false.
  # xhptdc8_manager_configuration second element
  # Next element
  # And, so on
  # Up to 8 elements in the sequence

Bassem-Ramzy commented 3 years ago

@sulimma, what is -1 used for in the example?

# define 9 channels of CMOS offsets
dc_offsets: &CMOS
  -1: 1.18

sulimma commented 3 years ago

This means that the value shall be applied to the whole array.

Structures shall be represented by maps, with the map key beeing the name of the structure member.
Arrays shall also be represented by sequences with the key beeing an integer.
If the key -1 is specified the value shall be set to all array elements.
Bound checks shall be applied when parsing arrays.

I must apologize. It was my mistake that I specify that arrays shall be represented by sequences. As you can see be the word "also" I intended to write "maps", just as in the first line, but apparently I did not have much focus when writing this. Sorry.

But I did specify what shall happen for -1. :-)

Bassem-Ramzy commented 3 years ago

Me too, I missed it

Bassem-Ramzy commented 3 years ago

@sulimma When building the util library, shall the DLL & Header be saved to /lib (in both Driver & Dummy Library folders) or util/include/ & util/lib/

Bassem-Ramzy commented 3 years ago

We can also think about using github test automation (e.g. https://github.com/marketplace/actions/run-c-c-test, just found it with a fast search), to provide test scenarios for xhptdc8_apply_yaml

sulimma commented 3 years ago

Regarding the directory layout for the util compile results: I have very little experience with library deployment. I don't really now. Just do it one way or the other and then we will learn and change it. It does not feel right to duplicate it twice for the dummy and the real library. I can image either of the following three variants:

a) /lib/include /lib/x86util /lib/x64util

b) /lib/include /lib /lib #64 bit version distinguished by file name

c) /util/lib/include /util/lib/x86 /util/lib/x64

I suggest that you take a few minutes to think about how you would setup the makefiles for future issues like the readout tool and the info tool and then decide what works best for those.

Bassem-Ramzy commented 3 years ago

@sulimma, please review xhptdc8_apply_yaml YAML Specifications and let me know if any thing needs to be updated/added

Bassem-Ramzy commented 3 years ago

Regarding the directory layout for the util compile results: I have very little experience with library deployment. I don't really now. Just do it one way or the other and then we will learn and change it. It does not feel right to duplicate it twice for the dummy and the real library. I can image either of the following three variants:

a) /lib/include /lib/x86util /lib/x64util

b) /lib/include /lib /lib #64 bit version distinguished by file name

c) /util/lib/include /util/lib/x86 /util/lib/x64

I suggest that you take a few minutes to think about how you would setup the makefiles for future issues like the readout tool and the info tool and then decide what works best for those.

Very valid point. I'm in favor of the last scenario (c):

Similar to what Windows & Visual Studio do. e.g. C:\Program Files\Common Files\microsoft shared\VC\msdia90.dll & x86: C:\Program Files (x86)\Common Files\Microsoft Shared\VC\msdia90.dll. So it will be "standard" and "systematic".
DLL to have the same name for both platforms (x86 & x64).
Since they use the driver DLL APIs, I suggest we put them in the corresponding folder `./lib/' (copy of each in a corresponding folder Driver/Dummy x86/x64 .Lib .DLL as well as .h in include)
All libraries to be gathered in one folder, it will be simple when running applications (using them) regarding to the %PATH% settings, and this will be straightforward when building an installation for the driver. So, we have:
1. x86 Folder of Driver & Utils
2. x86 Folder of Dummy Driver & Utils
3. x64 Folder of Driver & Utils
4. x64 Folder of Dummy Driver & Utils Expectedly, 4 corresponding options in the installation (e.g. .msi file)
We just keep it that simple, and when files and libraries get messy, we can then think about a suitable different structure. OK to proceed?

sulimma commented 3 years ago

The spec is looking good. I would only change one thing: Duplicate mapping keys should result in an error:

     rising : true 
     rising : false

However, that is not crucial.

If what you are decribing is the default behaviour of the YAML library and it required extra effort to check for the error than we can prceed as you suggested.

Bassem-Ramzy commented 3 years ago

The spec is looking good. I would only change one thing: Duplicate mapping keys should result in an error:
     rising : true 
     rising : false 
However, that is not crucial.

If what you are decribing is the default behaviour of the YAML library and it required extra effort to check for the error than we can prceed as you suggested.

Yes, YAML library doesn't refuse that, so we can leave it as is.

Bassem-Ramzy commented 3 years ago

Library is developed as per the specifications in Readme Waiting for test result if any issue is generated

cronologic-de / xhptdc8_babel