datahub-project / datahub

The Metadata Platform for your Data Stack
https://datahubproject.io
Apache License 2.0
9.85k stars 2.91k forks source link

Not possible to ingest MLmodels via OpenApi #6414

Closed patriciaramosf closed 1 year ago

patriciaramosf commented 1 year ago

Describe the bug There is a bug related to MLModel schema. The structure is ok, I also managed to ingest other entities as glossaryterms following this "structure or instructions" that appear in the swagger of datahub. The error is always related to the name of the aspect "MLModelProperties". I send the correct name but it is detected as if it was written with the first m in lowercase.

ERROR : "Request processing failed; nested exception is java.lang.RuntimeException: Unknown aspect mLModelProperties for entity MLModel"

REQUEST BODY: [ { "entityType": "MLModel", "entityUrn": "urn:li:mlModel:(urn:li:dataPlatform:science,sciencePatri,PROD)", "entityKeyAspect": { "__type": "MLModelKey", "platform":"urn:li:dataPlatform:science", "name":"sciencePatri", "origin":"PROD" }, "aspect": { "__type": "MLModelProperties", "customProperties": { "additionalProp1": "value1", "additionalProp2": "value2", "additionalProp3": "value3" }, "externalUrl": "url", "description":"description", "tags":["urn:li:tag:hola"] } } ]

To Reproduce Steps to reproduce the behavior:

  1. Go to 'https://demo.datahubproject.io/openapi/swagger-ui/index.html#
  2. Create the tags you are going to associate to your MLModel (tags are a required field)
  3. Create the platform entity where you are going to have the MLModel, or choose the urn of a platform that already exists
  4. Copy the structure attached above in to the POST /entities/v1 query
  5. Check the error in the console

Expected behavior The response should be 200 sucess and the mlModel should be created.

Screenshots If applicable, add screenshots to help explain your problem.

Captura de pantalla 2022-11-10 a las 10 16 49

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

patriciaramosf commented 1 year ago

This issue is still a bug. Tested in last version 10.0.0

Ido-Levi commented 1 year ago

Also managed to reproduce in 0.10.0, but weirdly enough I managed to create an empty MlModel entity without any aspects or ownership

patriciaramosf commented 1 year ago

Hi!!! Could you show me how you did it? Thanks!!

Ido-Levi commented 1 year ago

This is the body I used

[
  {
    "entityType": "MlModel",
    "entityUrn": "urn:li:mlModel:(urn:li:dataPlatform:testplatform,testtest,DEV)",
    "aspect": {
        "__type": "Ownership"
    }
  }
]

But I couldn't add any aspects to it or features, just an empty shell

patriciaramosf commented 1 year ago

Thanks!! But yes, still aspects are necessary.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 30 days since being marked as stale.