OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
We have Entities with properties that represent URLs. For example:
A Pipeline has Tasks, which have taskUrl linking to it,
A Dashboard has Charts which have chartUrl again, linking to it.
We currently inform that property during the ingestion process as a string, which contains the full URL. E.g., http://localhost:8080/my/task/url. However, the services (in this case, PipelineService or DashboardService), also have in their connection description a hostPort attribute.
We allow to update the connection URL of the services in the UI, but this change is not reflected properly in the tasks and charts URLs, as that information is not dynamic.
In this feature, we are going to revamp how we are ingesting and exploiting this information in the UI:
During the ingestion, we will just inform the suffix URL on taskUrl and chartUrl. E.g., /my/task/url.
In the UI, the link will be created dynamically from serviceUrl/taskUrl.
This way, all the changes and fixes done on the service side will be properly reflected in its children's attributes.
IMPORTANT: Not all services have a hostPort property. For example, PowerBI is a hosted service and we only give client/secret information to connect. Therefore, we will create the URL as {hostPort}/{urlSuffix} for those services that do have hostPort. Otherwise, just show the URL as it comes from the backend.
Feature
Describe the task
We have Entities with properties that represent URLs. For example:
Pipeline
hasTask
s, which havetaskUrl
linking to it,Dashboard
hasChart
s which havechartUrl
again, linking to it.We currently inform that property during the ingestion process as a
string
, which contains the full URL. E.g.,http://localhost:8080/my/task/url
. However, the services (in this case,PipelineService
orDashboardService
), also have in their connection description ahostPort
attribute.We allow to update the connection URL of the services in the UI, but this change is not reflected properly in the tasks and charts URLs, as that information is not dynamic.
In this feature, we are going to revamp how we are ingesting and exploiting this information in the UI:
taskUrl
andchartUrl
. E.g.,/my/task/url
.serviceUrl/taskUrl
.This way, all the changes and fixes done on the service side will be properly reflected in its children's attributes.
IMPORTANT: Not all services have a
hostPort
property. For example, PowerBI is a hosted service and we only give client/secret information to connect. Therefore, we will create the URL as{hostPort}/{urlSuffix}
for those services that do have hostPort. Otherwise, just show the URL as it comes from the backend.