[Feature request] Port the PROMless test script to an RPC method

lpetre-ulb commented 5 years ago

Brief summary of issue

@evka85 developed a very useful Python script to test the PROMless/slow control/trigger link issue. This script is available on some CTP7. For example on eagle23 at /mnt/persistent/gemdaq/python/reg_interface/ge11_promless_test.py.

Types of issue

[ ] Bug report (report an issue with the code)
[x] Feature request (request for change which adds functionality)

Expected Behavior

Since the Python tools have limitations with the complete address table, I suggest to implement this RPC method as a close port of Evaldas' script :

void programAllOptoHybridsLocal(uint32_t ohMask, uint32_t nOfIterations, uint8_t mode);

Where mode is one of the followings :

dontStop : always perform the given number of iterations. This gives us statistics.
stopOnSuccess : stops once all OHs in ohask are properly programmed. The mode could advantageously replace the programAllOptoHybrids function of cmsgemos.
stopOnFailure : stops as soon as a failure is detected, so one can investigate.

Obviously the remotely callable method should be implemeted as well.

Context (for feature requests)

We need a tool to test new OH/CTP7 firmware and to precisely diagnose any PROMless/slow control/trigger link failure.

lpetre-ulb commented 5 years ago

I cannot self-asign, but I already started working on the issue. It works for 1 OH at the moment, but is the "multi-link" extension is trivial.

bdorney commented 5 years ago

Since the Python tools have limitations with the complete address table:

Which python tools are you referring too? There are no limitations with the python API in the central SW repo's.

It's not clear from this issue what you are proposing or how it is different from HwAMC::programAllOptohybridFPGAs. Rather than make a new debugging tool in ctp7_modules it seems like it would be more appropriate to make a script under cmsgemos/gempython/tests to execute what you are proposing.

Moreover it's not clear why the current tool testConnectivity.py is unable to meet this need. Rather than re-inventing the wheel it would be better to update the current tools to incorporate any missing functionality.

bdorney commented 5 years ago

I've marked this as question since more info is needed; again based on the current issue description the appropriate place for this does not appear to be here.

lpetre-ulb commented 5 years ago

Since the Python tools have limitations with the complete address table:

Which python tools are you referring too? There are no limitations with the python API in the central SW repo's.

I'm referring to the CTP7 Python tools. As far as I know, the full address table .pickle file cannot be loaded in the CTP7 memory.

It's not clear from this issue what you are proposing or how it is different from HwAMC::programAllOptohybridFPGAs. Rather than make a new debugging tool in ctp7_modules it seems like it would be more appropriate to make a script under cmsgemos/gempython/tests to execute what you are proposing.

Moreover it's not clear why the current tool testConnectivity.py is unable to meet this need. Rather than re-inventing the wheel it would be better to update the current tools to incorporate any missing functionality.

Indeed it includes the functionality provided by HwAMC::programAllOptohybridFPGAs, but it is more versatile. I see good reasons move the functionality to the ctp7_modules

It can leverage the SCA API implemented by @jsturdy. Not using it would be re-inventing the wheel as well.
It can abstract differences between GE1/1 and GE2/1 (e.g. for the fiber mapping).
The DONE status must be read though the SCA just after the TTC hard reset, when it goes low.

A script in a higher level repo (cmsgemos/gempython/tests) is certainly the expected outcome of this RPC method.

mexanick commented 5 years ago

I'm referring to the CTP7 Python tools. As far as I know, the full address table .pickle file cannot be loaded in the CTP7 memory.

This is not correct. One can prepare a .pickle file on the host PC, slightly modify it and use on the CTP7. The only limitation is the size (with the current FW/address table size we can support up to about 4 optical links)

Considering the repository: this is not correct place to put the python code. GEM-specific expert-level python tools should go under xhal repository, while generic (not GEM-specific) scripts should go to reg_utils repo.

lpetre-ulb commented 5 years ago

I'm referring to the CTP7 Python tools. As far as I know, the full address table .pickle file cannot be loaded in the CTP7 memory.

This is not correct. One can prepare a .pickle file on the host PC, slightly modify it and use on the CTP7. The only limitation is the size (with the current FW/address table size we can support up to about 4 optical links)

So this is not the full address table. ;-) And we need more than 4 optical links; even at ULB we use OH0 and OH4 due to fibers configuration. Sure one could create a .pickle file with non-contiguous OHs, but that would be even more hackish.

Considering the repository: this is not correct place to put the python code. GEM-specific expert-level python tools should go under xhal repository, while generic (not GEM-specific) scripts should go to reg_utils repo.

My proposal is not a Python tool, its a real C++ RPC method. Sure there will be an expert-level Python interface in xHAL but the proposed RPC method can be used for simple OptoHybrid configuration as well.

mexanick commented 5 years ago

Considering the address table:

we should continue using the LMDB for ctp7 modules
there're other ways to strip the pickle address table:
- remove the meta-info (reg descriptions)
- switch to on-the-go address generation (this may impact the speed, but pickle-based utils aren't anyways for fast execution)

So far we are in general fine with the pickle files as is, we can introduce the above changes when we have a real use case. If you have one, please provide it.

lpetre-ulb commented 5 years ago

No, I don't have a real use case. Sure we could improve the current Python script but I my opinion it's worth moving a superset of it to the RPC modules. In the same time the RPC method version would allow an easy GE2/1 compatibility since it can abstract all the small differences from DAQ machine software, e.g. the fibers mapping and SCA GPIO mappings.

One more idea for the pickle-based tools. It should be easy to provide a library which creates "Nodes" from the LMDB database which is already there. Of course, we would loose the descriptions as well as the parent-children relationships on the CTP7 but from what I see there are not used in these tools.

jsturdy commented 5 years ago

Indeed it includes the functionality provided by HwAMC::programAllOptohybridFPGAs, but it is more versatile. I see good reasons move the functionality to the ctp7_modules

It can leverage the SCA API implemented by @jsturdy. Not using it would be re-inventing the wheel as well.

It can abstract differences between GE1/1 and GE2/1 (e.g. for the fiber mapping).

The DONE status must be read though the SCA just after the TTC hard reset, when it goes low.

A script in a higher level repo (cmsgemos/gempython/tests) is certainly the expected outcome of this RPC method.

Based on this (and the fact that there currently exists a functionality to achieve the desired result), this development should proceed after the RPC modules migrate to the templated API

cms-gem-daq-project / ctp7_modules