DMTF / Redfish-Service-Validator

The Redfish Service Validator is a Python3 tool for checking conformance of any "device" with a Redfish service interface against Redfish CSDL schema
Other
37 stars 33 forks source link

Changes to collection handling, notes/variable changes #552

Closed tomasg2012 closed 10 months ago

tomasg2012 commented 1 year ago

Program uses a Collection object to handle collections instead of the previous method.

Added some notes and changed variable names for clarity.

Fixes #497

mraineri commented 1 year ago

There seems to be a regression in this branch; I tried running it against the mockup built into the Redfish-Mockup-Server, and it's throwing an exception when processing "/redfish/v1/Registries/Base.1.5.0" (or right after finishing). Debug log file here: ConformanceLog_05_10_2023_102929.txt.

And this is the exception (doesn't show up in the log)

*** /redfish/v1/Registries/Base.1.5.0
     Type (MessageRegistryFile.v1_1_3.MessageRegistryFile), GET SUCCESS (time: 0:00:00.002545)
     PASS
Traceback (most recent call last):
  File "/share/Redfish/Redfish-Service-Validator/RedfishServiceValidator.py", line 10, in <module>
    status_code, lastResultsPage, exit_string = main()
  File "/share/Redfish/Redfish-Service-Validator/redfish_service_validator/RedfishServiceValidator.py", line 189, in main
    success, counts, results, xlinks, topobj = validateURITree(currentService, '/redfish/v1/', 'ServiceRoot', expectedJson=jsonData)
  File "/share/Redfish/Redfish-Service-Validator/redfish_service_validator/validateResource.py", line 367, in validateURITree
    returnVal = validateURITree(service, link_destination, uriName + ' -> ' + link.Name, parent=parent, allLinks=allLinks, inAnnotation=link.InAnnotation)
  File "/share/Redfish/Redfish-Service-Validator/redfish_service_validator/validateResource.py", line 367, in validateURITree
    returnVal = validateURITree(service, link_destination, uriName + ' -> ' + link.Name, parent=parent, allLinks=allLinks, inAnnotation=link.InAnnotation)
  File "/share/Redfish/Redfish-Service-Validator/redfish_service_validator/validateResource.py", line 315, in validateURITree
    for sub_obj in thisobj['Location'].Collection:
AttributeError: 'RedfishObjectCollection' object has no attribute 'Collection'. Did you mean: 'collection'?
mraineri commented 1 year ago

With the latest update, it looks like it's flagging all arrays of enumerations, even though they are being used properly. It seems like it's taking the entire array value and comparing it with the enum list.

For example, when running against public-rackmount1, I see this in the results for "/redfish/v1/Systems/437XR1138R2/Memory/DIMM1":

image

In the payload, this is the property usage:

    "MemoryMedia": [
        "DRAM"
    ],

Debug log attached:

ConformanceLog_06_10_2023_215941.txt

ConformanceHtmlLog_06_10_2023_215941.html.txt

tomasg2012 commented 1 year ago

Pushed some changes to revert some things, however still needs more testing.

mraineri commented 1 year ago

Running with the mockup packaged with the Redfish Mockup Server (it's the public-rackmount1 from 2022.3), I'm seeing one expected error turn into three errors.

image

/redfish/v1/AccountService/Accounts/1 is missing the "AccountTypes" property, and it's resulting in "failMandatoryExist" being counted twice instead of once, and a new "err.Collection(ManagerAccount.AccountTypes)" error.

ConformanceLog_06_30_2023_113202.txt

mraineri commented 1 year ago

Using the same mockup, I modified /redfish/v1/Managers/BMC/EthernetInterfaces/Dedicated to remove the array braces in IPv4Addresses, and it's not reporting an error with the property.

{
    "@odata.type": "#EthernetInterface.v1_5_1.EthernetInterface",
    "Id": "Dedicated",
    "Name": "Manager Ethernet Interface",
    "Description": "Management Network Interface",
    "Status": {
        "State": "Enabled",
        "Health": "OK"
    },
    "LinkStatus": "LinkUp",
    "InterfaceEnabled": true,
    "PermanentMACAddress": "23:11:8A:33:CF:EA",
    "MACAddress": "23:11:8A:33:CF:EA",
    "SpeedMbps": 100,
    "AutoNeg": true,
    "FullDuplex": true,
    "MTUSize": 1500,
    "HostName": "web483-bmc",
    "FQDN": "web483-bmc.dmtf.org",
    "MaxIPv6StaticAddresses": 1,
    "VLAN": {
        "VLANEnable": true,
        "VLANId": 101
    },
    "IPv4Addresses":  {    <--- This is defined in schema as an array of objects
        "Address": "192.168.0.10",
        "SubnetMask": "255.255.252.0",
        "AddressOrigin": "DHCP",
        "Gateway": "192.168.0.1",
        "Oem": {}
    },
    "IPv6AddressPolicyTable": [
        {
            "Prefix": "::1/128",
            "Precedence": 50,
            "Label": 0
        }
    ],
    "IPv6StaticAddresses": [
        {
            "Address": "fe80::1ec1:deff:fe6f:1e24",
            "PrefixLength": 16
        }
    ],
    "IPv6DefaultGateway": "fe80::1ec1:deff:fe6f:1e24",
    "IPv6Addresses": [
        {
            "Address": "fe80::1ec1:deff:fe6f:1e24",
            "PrefixLength": 64,
            "AddressOrigin": "SLAAC",
            "AddressState": "Preferred",
            "Oem": {}
        }
    ],
    "NameServers": [
        "names.dmtf.org"
    ],
    "@Redfish.Settings": {
        "@odata.type": "#Settings.v1_3_0.Settings",
        "SettingsObject": {
            "@odata.id": "/redfish/v1/Managers/BMC/EthernetInterfaces/Dedicated/SD"
        },
        "Time": "2012-03-07T14:44.30-05:00",
        "ETag": "84ffcbb050ddc7fa9cddb59014546e59",
        "Messages": [
            {
                "MessageId": "Base.1.0.SettingsFailed",
                "RelatedProperties": [
                    "#/IPv6Addresses/PrefixLength"
                ]
            }
        ]
    },
    "Oem": {},
    "@odata.id": "/redfish/v1/Managers/BMC/EthernetInterfaces/Dedicated",
    "@Redfish.Copyright": "Copyright 2014-2019 DMTF. For the full DMTF copyright policy, see http://www.dmtf.org/about/policies/copyright."
}
tomasg2012 commented 11 months ago

This still introduces some issues with Links, so continuing work...

mraineri commented 11 months ago

Testing with the latest changes and it's looking much better. Still seeing this issue though (where one error is now three errors):

https://user-images.githubusercontent.com/1505099/250131660-50528058-b763-4272-9c5e-c1a03e62c12e.png

mraineri commented 11 months ago

Looking at the latest changes and I'm now seeing Excerpt array properties flagged with missing @odata.id. For example (from public-rackmount1 in the mockup server):

image

mraineri commented 10 months ago

Testing looks good!