Custom-attributes of asset being removed when asset changed via UI

tmberthold commented 1 month ago

Description - What happened? *

Original Issue: Situation in a data offer custom-attributess are used, everything runs fine and even broker shows attributes. Now, eg. due to a typo, description of the data offer has to be corrected. For this the customer ist using the Connector-UI edit function. The content of the custom-attributes is overwritten without fault message.

Expected Behavior *

Original Issue: Customer attributes are not supported for editing in UI, but there should be no overwritting of existing information.

Observed Behavior *

Existing custom attributes of an asset are deleted when the asset is edited via UI.

Steps to Reproduce

My steps to reproduce (TB):

Management-API: Create Asset using the Management-API adding additional custom attributes to the asset POST http://localhost:11002/api/management/v3/assets with POST-body (look for http://purl.org/dc/terms/test being custom attribute):

{
"@type": "https://w3id.org/edc/v0.0.1/ns/Asset",
"https://w3id.org/edc/v0.0.1/ns/properties": {

    "http://purl.org/dc/terms/test": "https://www.google.com",

    "https://w3id.org/edc/v0.0.1/ns/id": "123456789mds",
    "http://www.w3.org/ns/dcat#version": "1.0",
    "http://purl.org/dc/terms/language": "https://w3id.org/idsa/code/EN",
    "http://purl.org/dc/terms/title": "test-document",
    "http://purl.org/dc/terms/description": "my test document",
    "http://www.w3.org/ns/dcat#keyword": [
        "keyword1",
        "keyword2"
    ],
    "https://w3id.org/mobilitydcat-ap/mobilityTheme": {
        "https://w3id.org/mobilitydcat-ap/mobility-theme/data-content-category": "Roadworks and Road Conditions",
        "https://w3id.org/mobilitydcat-ap/mobility-theme/data-content-sub-category": "Road Conditions"
    },
    "https://w3id.org/mobilitydcat-ap/georeferencingMethod": "Lat/Lon",
    "https://w3id.org/mobilitydcat-ap/transportMode": "Road",
    "http://purl.org/dc/terms/rightsHolder": "my-sovereign-legal-name",
    "http://www.w3.org/ns/dcat#distribution": {
        "http://www.w3.org/ns/dcat#mediaType": "application/json",
        "https://w3id.org/mobilitydcat-ap/mobilityDataStandard": {
            "@id": "my-data-model-001",
            "https://w3id.org/mobilitydcat-ap/schema": {
                "http://www.w3.org/ns/dcat#downloadURL": [
                    "https://teamabc.departmentxyz.schema/a",
                    "https://teamabc.departmentxyz.schema/b"
                ],
                "http://www.w3.org/2000/01/rdf-schema#Literal": "These reference files are important"
            }
        },
        "http://www.w3.org/ns/adms#sample": [
            "https://teamabc.departmentxyz.sample/a",
            "https://teamabc.departmentxyz.sample/b"
        ],
        "http://purl.org/dc/terms/rights": {
            "http://www.w3.org/2000/01/rdf-schema#label": "Please cite the dataset as..."
        }
    },
    "http://purl.org/dc/terms/accrualPeriodicity": "every month",
    "http://purl.org/dc/terms/temporal": {
        "http://www.w3.org/ns/dcat#startDate": "2024-02-01",
        "http://www.w3.org/ns/dcat#endDate": "2024-02-10"
    },
    "http://purl.org/dc/terms/creator": {
        "http://xmlns.com/foaf/0.1/name": "My Org"
    },
    "http://purl.org/dc/terms/license": "https://creativecommons.org/licenses/by/4.0/",
    "http://www.w3.org/ns/dcat#landingPage": "https://mydepartment.myorg.com/my-offer",
    "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyMethod": "false",
    "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyPath": "false",
    "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyQueryParams": "false",
    "https://semantic.sovity.io/dcat-ext#httpDatasourceHintsProxyBody": "false",
    "http://purl.org/dc/terms/publisher": {
        "http://xmlns.com/foaf/0.1/homepage": "https://myorg.com/"
    }
},
"https://w3id.org/edc/v0.0.1/ns/privateProperties": {},
"https://w3id.org/edc/v0.0.1/ns/dataAddress": {
    "https://w3id.org/edc/v0.0.1/ns/type": "HttpData",
    "https://w3id.org/edc/v0.0.1/ns/baseUrl": "https://www.sovity.de"
}
}

UI: Validate in UI asset is created and custom attributes are available:

UI: Click on edit to edit the asset and change for example the asset-name and Update the asset.
Custom Attributes gone:

Context Information

TB: can confirm it using EDC CE 10.0.0 and UI 4.1.0.

Original Issue:

Portal: 2.3.0
Broker: 4.2.0
Logging House: 1.0.0-beta.3
EDC: 8.1.0

Relevant log output

No response

Screenshots

No response

ununhexium commented 1 month ago

Investigation results:

the way that we currently update assets, we only copy a fixed set of fields. Custom properties are not supported via the wrapper API, which is used by the UI.

Only CustomJson and CustomJsonLD are supported to store custom properties.

I will redirect to someone with product vision about this problem, because this looks like it's working as intended.

AbdullahMuk commented 1 month ago

@ununhexium I appreciate the clear assessment.

Given the user scenario AND your assessment, I envision that it is valuable to upgrade the API Wrapper to support Custom Properties.

@richardtreier @ununhexium some follow-up points for discussion:

Is that the right thing to do: upgrade the API Wrapper to support Custom Properties?
Assuming it is the right thing to do, please create a GH Issue for it that we will prioritize as per MDS 2.2 context (FYI, checked https://github.com/sovity/edc-ce/issues/177 for existing GH issue on this gap but couldn't find one).

ununhexium commented 1 month ago

The problem can be fixed on the UI side and the backend side.

On the frontend side: The asset is currently PUTed to the backend, which means a replacement of the entire object. The custom property that goes missing is stored in customJsonLdAsString.

To fix this specific problem:

[ ] Ensure that the customJsonLdAsString value is passed to the asset edition endpoint

In this example, it would be ... "customJsonLdAsString":"{\"http://purl.org/dc/terms/test\":\"https://google.com\"}" ...

And while checking this issue, also check that:

[ ] The customJsonAsString is supported
[ ] There no other missing value.

The list of fields to pass can be found in the UiAsset class.

In the long term, it would be better that the backend supports a PATCH method to only change the modified fields and preserve the rest. https://github.com/sovity/edc-ce/issues/1023

illfixit commented 1 month ago

Got help from @richardtreier and the FE part is almost ready, functions good with the existing BE https://github.com/sovity/edc-ui/pull/794

sovity / edc-ce