search

DMTF / Redfish-Service-Validator

The Redfish Service Validator is a Python3 tool for checking conformance of any "device" with a Redfish service interface against Redfish CSDL schema
Other
37 stars 33 forks source link

DSP2043 validation #589

Open commonism opened 1 month ago

commonism commented 1 month ago

Hi,

using the DSP8010_2024.1 & 1.2.6 Swordfish OpenAPI schemas I validated all DSP2043 mock files.

189 failed, 2864 passed, 1 warning in 62.05s (0:01:02)

failed.txt

FAILED tests/test_dsp2043.py::test_dsp2043_file[public-power-shelf::EventService/Subscriptions/1/index.json] - pydantic_core._pydantic_core.ValidationError: 1 validation error for EventDestination_v1_14_1_EventDestination

It's a mix of different issues:

autoExpand LogEntryCollections

>       r = type_.model_validate(data)
E       pydantic_core._pydantic_core.ValidationError: 10 validation errors for LogEntryCollection_LogEntryCollection
E       Members.0.`@odata.type`
E         Extra inputs are not permitted [type=extra_forbidden, input_value='#LogEntry.v1_16_1.LogEntry', input_type=str]
E           For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
E       Members.0.Id
E         Extra inputs are not permitted [type=extra_forbidden, input_value='1', input_type=str]
E           For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
E       Members.0.Name

missing properties

E       pydantic_core._pydantic_core.ValidationError: 3 validation errors for ComponentIntegrity_v1_2_3_ComponentIntegrity
E       `@odata.id`
E         Field required [type=missing, input_value={'@odata.type': '#Compone...ut/policies/copyright.'}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/missing
E       ComponentIntegrityTypeVersion
E         Field required [type=missing, input_value={'@odata.type': '#Compone...ut/policies/copyright.'}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/missing
E       Name
E         Field required [type=missing, input_value={'@odata.type': '#Compone...ut/policies/copyright.'}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/missing
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for Chassis_v1_25_1_Chassis
E       ChassisType
E         Field required [type=missing, input_value={'@odata.type': '#Chassis...ut/policies/copyright.'}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/missing

unknown properties

E       pydantic_core._pydantic_core.ValidationError: 2 validation errors for Thermal_v1_7_3_Thermal
E       Fans.0
E         Value error, unmatched property ReadingRPM [type=value_error, input_value={'@odata.id': '/redfish/v.../Chassis/CXL-Chassis'}]}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/value_error
E       Fans.1
E         Value error, unmatched property ReadingRPM [type=value_error, input_value={'@odata.id': '/redfish/v.../Chassis/CXL-Chassis'}]}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/value_error

pathes without PathItem definition

tests.test_dsp2043.RoutingError: /redfish/v1/UpdateService/ClientCertificates/1/RekeyActionInfo
tests.test_dsp2043.RoutingError: /redfish/v1/Managers/1/EthernetInterfaces/1/SD
tests.test_dsp2043.RoutingError: /redfish/v1/EventService/SubmitTestEventActionInfo

Focussing on a single entry …

E       pydantic_core._pydantic_core.ValidationError: 1 validation error for EventDestination_v1_14_1_EventDestination
E       SubscriptionType
E         Field required [type=missing, input_value={'@odata.type': '#EventDe...ut/policies/copyright.'}, input_type=dict]
E           For further information visit https://errors.pydantic.dev/2.5/v/missing

the definition is

    EventDestination_v1_14_1_EventDestination:
…
      properties:
…
        SubscriptionType:
          description: The subscription type for events.
          oneOf:
          - $ref: '#/components/schemas/EventDestination_v1_14_1_SubscriptionType'
          - enum:
            - 'null'
…
      required:
      - SubscriptionType

and the data 

# public-pdu/EventService/Subscriptions/1/index.json
{
    "@odata.type": "#EventDestination.v1_14_1.EventDestination",
    "Id": "1",
    "Name": "EventSubscription 1",
    "Destination": "http://www.dnsname.com/Destination1",
    "EventTypes": [
        "Alert"
    ],
    "Context": "WebUser3",
    "Protocol": "Redfish",
    "@odata.id": "/redfish/v1/EventService/Subscriptions/1",
    "@Redfish.Copyright": "Copyright 2014-2023 DMTF. For the full DMTF copyright policy, see http://www.dmtf.org/about/policies/copyright."
}

SubscriptionType is missing. Is this a problem with the DSP2043 data due to missing validation or the generated OpenAPI specification?

commonism commented 1 month ago

As you can see from the linked comits, I'm having issues downloading the dsp8010 zip from the github ci using the standard github runners.

It's slow and breaks, can you help with this?

--2024-05-27 15:40:03--  https://www.dmtf.org/sites/default/files/standards/documents/DSP8010_2024.1.zip
Resolving www.dmtf.org (www.dmtf.org)... 72.47.235.184
Connecting to www.dmtf.org (www.dmtf.org)|72.47.235.184|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 48279220 (46M) [application/zip]
Saving to: ‘DSP8010_2024.1.zip’
     0K .......... .......... .......... .......... ..........  0%  160K 4m55s
    50K .......... .......... .......... .......... ..........  0%  319K 3m41s
   100K .......... .......... .......... .......... ..........  0% 21.4K 14m38s
   150K .......... .......... .......... .......... ..........  0%  160K 12m12s
   200K .......... .......... .......... .......... ..........  0%  184M 9m45s
   250K .......... .......... .......... .......... ..........  0% 79.8K 9m45s
   300K .......... .......... .......... .......... ..........  0%  107K 9m23s
   350K .......... .......... .......... .......... ..........  0%  107K 9m7s
   400K .......... .......... .......... .......... ..........  0%  106K 8m55s
   450K .......... .......... .......... .......... ..........  1% 34.4K 10m16s
   500K .......... .......... .......... .......... ..........  1% 9.50K 16m46s
   550K .......... .......... .......... .......... ..........  1% 26.0K 17m50s
   600K .......... .......... .......... .......... ..........  1% 53.3K 17m34s
   650K .......... .......... .......... .......... ..........  1% 80.0K 16m59s
   700K .......... .......... .......... .......... ..........  1% 13.8K 19m34s
   750K .......... .......... .......... .......... ..........  1% 8.64K 23m55s
   800K .......... .......... .......... .......... ..........  1% 13.6K 25m50s
   850K .......... .......... .......... .......... ..........  1% 4.25K 34m27s
   900K .......... .......... .......... .......... ..........  2% 9.75K 36m45s
   950K .......... .......... .......... .......... ..........  2% 4.93K 42m41s
  1000K .......... .......... .......... .......... ..........  2% 5.96K 46m45s
  1050K .......... .......... .......... ........               2% 11.0K=67s
2024-05-27 15:41:11 (16.2 KB/s) - Connection closed at byte 1114112. Retrying.
--2024-05-27 15:41:12--  (try: 2)  https://www.dmtf.org/sites/default/files/standards/documents/DSP8010_2024.1.zip
Connecting to www.dmtf.org (www.dmtf.org)|72.47.235.184|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 48279220 (46M), 47165108 (45M) remaining [application/zip]
Saving to: ‘DSP8010_2024.1.zip’
        [ skipping 1050K ]
  1050K ,,,,,,,,,, ,,,,,,,,,, ,,,,,,,,,, ,,,,,,,,.. ..........  2% 32.3K 99m8s
  1100K .......... .......... .......... .......... ..........  2% 7.07K 1h46m
  1150K .......... .......... .......... .......... ..........  2% 27.1K 71m33s
  1200K .......... .......... .......... .......... ..........  2% 34.7K 56m13s
  1250K .......... .......... .......... .......... ..........  2% 62.4K 45m48s
  1300K .......... .......... .......... .......... ..........  2% 78.1K 38m53s
  1350K .......... .......... .......... .......... ..........  2% 77.6K 34m11s
  1400K .......... .......... .......... .......... ..........  3%  104K 30m27s
…
mraineri commented 1 month ago

There are certainly issues with mockups in DSP2043; we haven't run the service validator against all of the mockups yet, but the ones you've identified are certainly real issues we need to fix in DSP2043.

As far as why the 8010 ZIP file takes so long... That's a good question... I've noticed lately some of my other GitHub actions are struggling with downloading packages from other sources (like downloading from Microsoft), so it may not be unique to your action.