MarquezProject / marquez

Collect, aggregate, and visualize a data ecosystem's metadata
https://marquezproject.ai
Apache License 2.0
1.72k stars 309 forks source link

Marquez UI fails to render page for dataset when version API returns graph without Job #2475

Closed batwalrus76 closed 3 months ago

batwalrus76 commented 1 year ago

An anonymous function in Datasetinfo.tsx raises a TypeError if the jobVersion object (inside JOB object, within the lastestRun object) does not exist. The full application becomes blank and can only get it to render by manually putting in base view URL. I am encountering this with the versions 0.31.0 and 0.32.0 docker image.

Before The dataset node is clicked:

Screen Shot 2023-04-17 at 9 57 02 AM

After (with error in console):

Screen Shot 2023-04-17 at 9 58 09 AM

Return from Postman GET request to http://localhost:3000/api/v1/lineage/?nodeId=dataset:file:%2Ftmp%2Fdelta-table%2Fbronze

{ "graph": [ { "id": "dataset:file:/tmp/delta-table/bronze", "type": "DATASET", "data": { "type": "DATASET", "id": { "namespace": "file", "name": "/tmp/delta-table/bronze" }, "type": "DB_TABLE", "name": "/tmp/delta-table/bronze", "physicalName": "/tmp/delta-table/bronze", "createdAt": "2023-04-17T12:59:25.998Z", "updatedAt": "2023-04-17T13:00:07.077Z", "namespace": "file", "sourceName": "file", "fields": [ { "name": "id", "type": "string", "tags": [], "description": null }, { "name": "date", "type": "timestamp", "tags": [], "description": null }, { "name": "location", "type": "string", "tags": [], "description": null }, { "name": "page", "type": "string", "tags": [], "description": null }, { "name": "price", "type": "integer", "tags": [], "description": null } ], "tags": [], "lastModifiedAt": null, "description": null, "lastLifecycleState": "CREATE" }, "inEdges": [], "outEdges": [] } ] }

Note: The data for this namespace, comes from the OpenLineage SparkListener, pretty much when saving small dataframes to delta lake tables.

boring-cyborg[bot] commented 1 year ago

Thanks for opening your first issue in the Marquez project! Please be sure to follow the issue template!

phixMe commented 3 months ago

Is this still an issue in the new UI?

batwalrus76 commented 3 months ago

This has gone away in newer versions.