references in component not working

[leolivier]

Hi, I'm using the API to try to change some fields in specific type references between specific type components. The use case is as follows:

    Workspace businessAppsWS = client.workspace().getWorkspaceById(BUSINESS_APPS_WORKSPACE_ID);
    // iterate on each app id
    for (String businessAppId : businessAppsWS.getComponents()) {
    // get the app component
        Component businessApp = client.component().getComponentById(businessAppId);
        Component.References refs = businessApp.getReferences();
        if (refs == null) return; 
        ArrayList<Reference> inrefs = new ArrayList<Reference>();
        for (Reference inref : refs.getIn()) {
            if (inref.getTypeName() == "is_integrated_with") {
                // manage description
                inref.setDescription(html2md(inref.getDescription()));
                // manage other fields
                inref.setFields(manageFields(inref.getFields()));
            }
            inrefs.add(inref);
        }
        refs.setIn(inrefs);
        businessApp.setReferences (refs);

Although I have a lot of in and out references, refs is always null, e.g. for this app, I have incomingReferenceCount=21.0, outgoingReferenceCount=34.0 and references=null

"Component { id='03ee7819e9ec7bcfe05351ef'' batchId='null', name='CofaServe', model='024ae887a8e5a24df7c5767e', created=Mon Aug 10 15:39:48 CEST 2020, createdBy='8facd0732b08a69fec1488d4', lastUpdated=Sun Oct 25 18:25:18 CET 2020, version='null', _version=50, rootWorkspace='afc4ad7bb2f6466b90d37208', children=[], 
****references=null****, 
parent='null', type='Application', typeId='p1597085224242', description='xxxxx', 
_fields={a lot fields here, ardoq={entity-type=component, persistent={survey={f82d3a5801ae965d2798b0ab={valid=false}, 82bac7ab11ad8d027e40fb75={valid=true}, f20d16d00440c826dec8caba={valid=true}, 72b4b5d6892cda464e9dacda={valid=true}, e1e599986ab0c5ad8bc43af1={valid=true}, af172d334470ef3f82709df5={valid=true}, 5430985676a4fcdf955b2d8d={valid=true}, 5c39809fbc1c46f326e741b5={valid=false}, 8fed0905d56ce42941240105={valid=true}, 1fa7fa95d143dcba5de7bf5d={valid=true}}}, 
*** incomingReferenceCount=21.0 ***, 
*** outgoingReferenceCount=34.0 ***}, 
createdByName=xxx, functional_fit=null}}"

I looked at the code and I don't understand how references is supposed to be set as it is not part of the result of the API call and no special processing is made...

[kristianhelgesen]

Sorry for the late reply. The references only a part of the component payload if you provide an extra parameter includeReferences. I'm not sure whether this has always been like this, or if it is the result of an optimisation. Anyways, you can provide the parameter this way:

        Map<String,String> queryParams = new HashMap<String,String>();
        queryParams.put("includeReferences", "true");
        Component c = client.component().getComponentById(businessAppId, queryParams);

Generally, doing a roundtrip to the API for each component and reference isn't the most efficient approach. You could consider fetching the whole aggregated workspace with all components and references, and do all your processing, before submitting the necessary changes back.

        AggregatedWorkspace ws = client.workspace().getAggregatedWorkspace(BUSINESS_APPS_WORKSPACE_ID);

[leolivier]

@kristianhelgesen, thank you for your reply. I tried once the aggregated workspace but it took so much time and it downloaded so much data at once (16MB) that I preferred using the other way. I might reconsider this for next usages but as this one is a one shot script, I'll keep it like that. Also, I'm a bit afraid of the consequences of a mistake in the script leading to breaking all the workspace at once. When doing this component by component, I can do it one by one (I didn't provide the whole code, I'm asking if I want to continue only for next component or for all, so I test it on the first apps and stop if it breaks something) Btw, is there a way to go back to a previous version of the workspace if something goes wrong?

[leolivier]

Just tested with your includeReferences solution and indeed I got the references but the typeName is always null so I can't filter on it... Is there a way to do this?

[kristianhelgesen]

Hi There's no way to revert changes, unfortunately. You can find the mapping from reference type name to the reference type ID this way

        Model model = client.model().getModelById(businessAppsWS.getComponentModel());
        Map<String,Integer> referenceTypes = model.getReferenceTypes();

[leolivier]

Thank you @kristianhelgesen One (hopefully) last question: at the end of my loop above, I update the references in the component, then I update the component itself like this:

client.component().updateComponent(businessAppId, businessApp);

but the references are not updated. Should I update them one by one? There is no way (except the workspace aggrgated which really scares me as I cannot test in a sandbox envt) to manage it in bulk mode?

[kristianhelgesen]

Yes, you'll have to update them one by one. There's currently no possibility to batch update using our Java client. I'm not sure when we can have that ready. The aggregated workspace functionality is just a convenience method to read data, not to post the same type of payload back. The same goes for references in the component endpoint. Updates to references has to happen by submitting the references directly.

[leolivier]

Ok, thank you, I'll do it like that. I'm afraid it will last hours but there is no other choice... Closing the issue

[kristianhelgesen]

We do have a batch create API, if that's what you're after. https://github.com/ardoq/ardoq-java-client/tree/master/src/main/java/com/ardoq/batch

The assumption is that updates will be of a smaller volume, so the create API was prioritised. It's not well documented, unfortunately, but it's worth having a look at.