Add PVI integration

[GDYendell]

Closes #119

[codecov-commenter]

Codecov Report

Merging #132 (c12b33f) into main (b21cfa9) will increase coverage by 3.15%. Report is 5 commits behind head on main. The diff coverage is 98.21%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##             main     #132      +/-   ##
==========================================
+ Coverage   68.15%   71.31%   +3.15%     
==========================================
  Files          16       16              
  Lines         804      868      +64     
==========================================
+ Hits          548      619      +71     
+ Misses        256      249       -7     

Files	Coverage Δ
src/ibek/__main__.py	100.00% <100.00%> (ø)
src/ibek/gen_scripts.py	100.00% <100.00%> (ø)
src/ibek/globals.py	97.05% <100.00%> (+0.76%)	:arrow_up:
src/ibek/ioc_cmds/assets.py	18.18% <100.00%> (ø)
src/ibek/render_db.py	100.00% <100.00%> (ø)
src/ibek/runtime_cmds/commands.py	100.00% <100.00%> (ø)
src/ibek/support.py	98.79% <100.00%> (-0.06%)	:arrow_down:
src/ibek/support_cmds/commands.py	53.76% <100.00%> (+7.37%)	:arrow_up:
src/ibek/support_cmds/files.py	60.00% <100.00%> (+10.00%)	:arrow_up:
src/ibek/ioc_cmds/commands.py	50.72% <71.42%> (+2.33%)	:arrow_up:

:mega: Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

[gilesknap]

ouch!

If only I had done the pydantic conversion!

Do I need an Asyn definition? The top of my asynNDArrayDriver.pvi.device.yaml looks like this I also recall you Capitalising th asyn in this file before....

label: asynNDArrayDriver
parent: asynPortDriver
children:
  - type: Group
    name: ADSetup
    layout:

+ defs='/epics/ibek-defs/*.ibek.support.yaml'
+ ibek runtime generate /epics/ioc-adaravis/ioc/config/ioc.yaml /epics/ibek-defs/ADAravis.ibek.support.yaml /epics/ibek-defs/ADCore.ibek.support.yaml /epics/ibek-defs/asyn.ibek.support.yaml /epics/ibek-defs/autosave.ibek.support.yaml /epics/ibek-defs/epics.ibek.support.yaml /epics/ibek-defs/iocStats.ibek.support.yaml --out /tmp/st.cmd --db-out /tmp/ioc.subst
Traceback (most recent call last):

  File "/venv/bin/ibek", line 8, in <module>
    sys.exit(cli())

  File "/repos/ibek/src/ibek/runtime_cmds/commands.py", line 49, in generate
    pvi_index_entries, pvi_databases = generate_pvi(ioc_instance)

  File "/repos/ibek/src/ibek/runtime_cmds/commands.py", line 94, in generate_pvi
    device.deserialize_parents([PVI_DEFS])

  File "/repos/pvi/src/pvi/device.py", line 396, in deserialize_parents
    parent_parameters = find_components(self.parent, yaml_paths)

  File "/repos/pvi/src/pvi/device.py", line 437, in find_components
    list(find_components(device.parent, yaml_paths))

  File "/repos/pvi/src/pvi/device.py", line 437, in find_components
    list(find_components(device.parent, yaml_paths))

  File "/repos/pvi/src/pvi/device.py", line 434, in find_components
    device = Device.deserialize(device_yaml)

  File "/repos/pvi/src/pvi/device.py", line 389, in deserialize
    return deserialize(cls, YAML(typ="safe").load(serialized))

  File "/venv/lib/python3.10/site-packages/apischema/deserialization/__init__.py", line 887, in deserialize
    return deserialization_method(

  File "apischema/deserialization/methods.pyx", line 504, in apischema.deserialization.methods.ObjectMethod.deserialize

  File "apischema/deserialization/methods.pyx", line 505, in apischema.deserialization.methods.ObjectMethod.deserialize

  File "apischema/deserialization/methods.pyx", line 1077, in apischema.deserialization.methods.ObjectMethod_deserialize

apischema.validation.errors.ValidationError: ValidationError: [{'loc': ['children', 6, 'children'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'children'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'children'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'children'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'children'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'children'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'layout'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'layout'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'layout'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'layout'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'layout'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'layout'], 'err': 'unexpected property'}, {'loc': ['children', 6, 'macros'], 'err': 'missing property'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}, {'loc': ['children', 6, 'pv'], 'err': 'missing property'}, {'loc': ['children', 6, 'pv'], 'err': 'missing property'}, {'loc': ['children', 6, 'pv'], 'err': 'missing property'}, {'loc': ['children', 6, 'pv'], 'err': 'missing property'}, {'loc': ['children', 6, 'pv'], 'err': 'missing property'}, {'loc': ['children', 6, 'type'], 'err': "not one of ['SignalR'] (oneOf)"}, {'loc': ['children', 6, 'type'], 'err': "not one of ['SignalW'] (oneOf)"}, {'loc': ['children', 6, 'type'], 'err': "not one of ['SignalRW'] (oneOf)"}, {'loc': ['children', 6, 'type'], 'err': "not one of ['SignalX'] (oneOf)"}, {'loc': ['children', 6, 'type'], 'err': "not one of ['DeviceRef'] (oneOf)"}, {'loc': ['children', 6, 'type'], 'err': "not one of ['SignalRef'] (oneOf)"}, {'loc': ['children', 6, 'ui'], 'err': 'missing property'}]

[gilesknap]

One more question - do we have a schema for device.yaml. If so I can use vscode to debug the yaml itself.

UPDATE: I found the schema and all device YAMLS are passing.

[gilesknap]

ooops

[GDYendell]

If there is no schema error, it is probably the name case thing. Check label and name fields and make sure they start with a capital.

This may be the bit out of that mess that is useful:

{'loc': ['children', 6, 'name'], 'err': 'not matching pattern ([A-Z][a-z0-9]*)*$ (pattern)'}

So the name of Group 6 (not sure if 0-indexed) needs a capital?

[gilesknap]

OK that did it - no more schema errors - but I needed to change quite a few places. I assume we can fix the generator to not make this mistake (and maybe include the restriction in the schema).

I'm now getting

OSError: Cannot find AsynPortDriver.pvi.device.yaml in [PosixPath('/epics/pvi-defs')]

which is correct - will try and generate a AsynPortDriver

[GDYendell]

AsynPortDriver.pvi.device.yaml

This should start with a lower case to match the driver name. I generated the pvi yamls here.

[gilesknap]

Thanks - but that is the AsynPortDriver itself from Asyn I assume.

Your link does not have one of these and I can't convince PVI to build one. (not sure which DB I should use but its the .h file it doesn't like I think.

(venv) [hgv27681@pc0116 ~]$ pvi convert device /tmp /dls_sw/prod/R3.14.12.7/support/asyn/4-41/asyn/asynPortDriver/asynPortDriver.h --template /dls_sw/prod/R3.14.12.7/support/asyn/4-41/asyn/asynRecord/asynRecord.db Traceback (most recent call last):

  File "/scratch/hgv27681/work/venv/bin/pvi", line 8, in <module>
    sys.exit(app())
             ^^^^^

  File "/scratch/hgv27681/work/venv/lib/python3.11/site-packages/pvi/__main__.py", line 104, in device
    name, parent = extract_device_and_parent_class(h.read_text())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/scratch/hgv27681/work/venv/lib/python3.11/site-packages/pvi/_convert/utils.py", line 10, in extract_device_and_parent_class
    assert match, "Can't find classes"

AssertionError: Can't find classes

[gilesknap]

We have lift-off!!

I removed the ref to AsynPortDriver that I mentioned above https://github.com/epics-containers/ibek/pull/132#issuecomment-1802441854

It now works. and I get

[GDYendell]

@gilesknap I added the last tweaks, visible in the compare changes here.

I know you said it should avoid using if definitions because it is bad practice if it is None, but given the typer weirdness it really is a list, so I think this is the most correct check. I also added a comment in the code about this for clarity.

I will leave you to press the merge button in case there are any issues with this.

[gilesknap]

Yep I'm happy with the if definitions thing.