In Multi-threading scenario, batch call to S/4 On-Prem is failing

[I320857]

Issue Description

Application Background: CPR is a Web application built using CAP Java framework and running on BTP Cloud foundry environment. We use Price Calculation API from S/4 On-Prem to calculate prices in CPR.

Use Case Currently at CPR item level, there are 3 price fields. As part of new use case, we need to recalculate these prices whenever date is changed in CPR. So, we have created 3 Future tasks in After Save handler. Each of these tasks, will create a batch request using Generic Odata client with POST request headers for all items to fetch price responses for specific price field.

To avoid, HTTP Client Caching issue in Multi-Threading scenarios, we are waiting for each task to complete before submitting next task using separate executor service. Also, we have provided executor service 15 minutes of time to finish the tasks.

Issue When we have only 1 item with 3 prices to be calculated in 3 separate tasks, batch response is received for all 3 tasks. If we have 10 items with only 2 prices to be calculated using 2 separate tasks, batch response is received for all 2 tasks. But If we have 10 items with only 3 prices to be calculated using 3 separate tasks, the scenario is failing with 404 status code.

I have attached the logs. You can see the exception at the end. It has processed the first task response and at the second task response processing, it fails with error: com.sap.cloud.sdk.datamodel.odata.client.exception.ODataServiceErrorException: The HTTP response code (404) indicates an error. The OData service responded with an error message.

Code

package com.sap.procrenegotiation.cpr.simulation.service;

import java.io.IOException;
import java.math.BigDecimal;
import java.nio.charset.StandardCharsets;
import java.text.MessageFormat;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;

import org.apache.http.Header;
import org.json.JSONArray;
import org.json.JSONObject;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;

import com.sap.cds.services.EventContext;
import com.sap.cds.services.messages.Messages;
import com.sap.cds.services.runtime.RequestContextRunner;
import com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination;
import com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationAccessException;
import com.sap.cloud.sdk.cloudplatform.connectivity.exception.DestinationNotFoundException;
import com.sap.cloud.sdk.cloudplatform.thread.ThreadContextExecutor;
import com.sap.cloud.sdk.datamodel.odata.client.ODataProtocol;
import com.sap.cloud.sdk.datamodel.odata.client.exception.ODataResponseException;
import com.sap.cloud.sdk.datamodel.odata.client.expression.ODataResourcePath;
import com.sap.cloud.sdk.datamodel.odata.client.request.ODataEntityKey;
import com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestAction;
import com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestBatch;
import com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestResultGeneric;
import com.sap.cloud.sdk.datamodel.odata.client.request.ODataRequestResultMultipartGeneric;
import com.sap.procrenegotiation.cpr.client.SimulationHttpDestinationClient;
import com.sap.procrenegotiation.cpr.model.RenegotiationMessageCode;
import com.sap.procrenegotiation.cpr.service.NegotiationProjectHeaderService;
import com.sap.procrenegotiation.cpr.service.NegotiationProjectItemService;
import com.sap.procrenegotiation.cpr.simulation.dto.PriceCalculationResponseDto;
import com.sap.procrenegotiation.cpr.simulation.dto.SimulationHeaderRequestDto;
import com.sap.procrenegotiation.cpr.util.JsonUtils;
import com.sap.procrenegotiation.cpr.util.PriceCalculationUtil;

import cds.gen.negotiationservice.NegotiationProjectHeaders;
import cds.gen.negotiationservice.NegotiationProjectItems;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;

@RequiredArgsConstructor
@Slf4j
@Service
public class PriceCalculationService {

    @Autowired
    Messages messages;

    private final NegotiationProjectItemService negotiationProjectItemService;
    private final NegotiationProjectHeaderService negotiationProjectHeaderService;
    private final PriceCalculationUtil priceCalculationUtil;
    private final SimulationHttpDestinationClient client;

    private final static String HEADERS_POST_PATH = "/PurchasingDocument({0})/SAP__self.SimulatePricing";
    private final static String KEY_PROPERTY_NAME = "PurchasingDocument";
    private final static String ACTION_SEGMENT_NAME = "SAP__self.SimulatePricing";

    /*
     * This method is used to calculate the prices for the negotiation project items.
     * It fetches the non zero net price, simulated price and renegotiated price items from the database.
     * It then creates the payload for the price calculation and executes the batch request.
     * It then updates the recalculated amount in the database.
     */
    public void calculatePrices(EventContext context, NegotiationProjectHeaders header) {
        String projectUUID = header.getProjectUUID();
        boolean isActiveEntity = header.getIsActiveEntity();

        // Create synchronized list to store the request actions as this is thread safe
        List<ODataRequestAction> synchronizedNetPriceRequestActionList = Collections.synchronizedList(new ArrayList<>());
        List<ODataRequestAction> synchronizedSimulatedPriceRequestActionList = Collections.synchronizedList(new ArrayList<>());
        List<ODataRequestAction> synchronizedRenegotiatedPriceRequestActionList = Collections.synchronizedList(new ArrayList<>());

        RequestContextRunner cdsRuntime = context.getCdsRuntime().requestContext();
        ThreadContextExecutor threadContextExecutor = ThreadContextExecutor.fromNewContext();
        ExecutorService netPriceCalculationExecutorService = Executors.newFixedThreadPool(20);

        /*
         * Create a list of tasks to be executed by the executor service.
         * Each task is responsible for fetching the non zero net price, simulated price and renegotiated price items respectively.
         * It then creates the payload for the price calculation and executes the batch request only if non zero item prices is present.
         * Price Type distinguishes between Net Price, Simulated Price and Renegotiated Price.
         * Price Type 0: Net Price, Price Type 1: Simulated Price, Price Type 2: Renegotiated Price
         */
        try {
            Callable<ODataRequestResultMultipartGeneric> netPriceTask = () -> cdsRuntime.run(ctx -> {
                return threadContextExecutor.execute(() -> {
                    List<NegotiationProjectItems> nonZeroNetPriceItems = negotiationProjectItemService.getNonZeroNetPriceItems(projectUUID, isActiveEntity);
                    log.info("Non Zero Net Price Items: " + nonZeroNetPriceItems);
                    if(!nonZeroNetPriceItems.isEmpty()){
                        List<SimulationHeaderRequestDto> netPriceCalculationPayloadList = new ArrayList<>();
                        netPriceCalculationPayloadList = priceCalculationUtil.createSimulationRequestPayloadList(header, nonZeroNetPriceItems, 0);
                        return executeBatchRequest(header.getProjectUUID(), netPriceCalculationPayloadList, synchronizedNetPriceRequestActionList);
                    }
                    return null;
                });
            });

            Callable<ODataRequestResultMultipartGeneric> simulatedPriceTask = () -> cdsRuntime.run(ctx -> {
                return threadContextExecutor.execute(() -> {
                    List<NegotiationProjectItems> nonZeroSimulatedPriceItems = negotiationProjectItemService.getNonZeroSimulatedPriceItems(projectUUID, isActiveEntity);
                    log.info("Non Zero Simulated Price Items: " + nonZeroSimulatedPriceItems);
                    if(!nonZeroSimulatedPriceItems.isEmpty()){
                        List<SimulationHeaderRequestDto> simualatedPriceCalculationPayloadList = new ArrayList<>();
                        simualatedPriceCalculationPayloadList = priceCalculationUtil.createSimulationRequestPayloadList(header, nonZeroSimulatedPriceItems, 1);
                        return executeBatchRequest(header.getProjectUUID(), simualatedPriceCalculationPayloadList, synchronizedSimulatedPriceRequestActionList);
                    }
                    return null;
                });
            });

            Callable<ODataRequestResultMultipartGeneric> renegotiatedPriceTask = () -> cdsRuntime.run(ctx -> {
                return threadContextExecutor.execute(() -> {
                    List<NegotiationProjectItems> nonZeroRenegotiatedPriceItems = negotiationProjectItemService.getNonZeroRenegotiatedPriceItems(projectUUID, isActiveEntity);
                    log.info("Non Zero Renegotiated Price Items: " + nonZeroRenegotiatedPriceItems);
                    if(!nonZeroRenegotiatedPriceItems.isEmpty()){
                        List<SimulationHeaderRequestDto> renegotiatedPriceCalculationPayloadList = new ArrayList<>();
                        renegotiatedPriceCalculationPayloadList = priceCalculationUtil.createSimulationRequestPayloadList(header, nonZeroRenegotiatedPriceItems, 2);
                        return executeBatchRequest(header.getProjectUUID(), renegotiatedPriceCalculationPayloadList, synchronizedRenegotiatedPriceRequestActionList);
                    }
                    return null;
                });
            });

            log.info(Thread.currentThread().getName() + ": Submitting tasks to executor");
            Future<ODataRequestResultMultipartGeneric> netPriceFuture = netPriceCalculationExecutorService.submit(netPriceTask);
            log.info(Thread.currentThread().getName() + ": Submitted tasks to executor");

            awaitTerminationAfterShutdown(netPriceCalculationExecutorService);
            // If all the tasks are finished, then update the recalculated amount in the database
            if(netPriceFuture.isDone()){
                final ODataRequestResultMultipartGeneric netPriceBatchResult = netPriceFuture.get();
                try {
                    ExecutorService simulationPriceCalculationExecutorService = Executors.newFixedThreadPool(20);
                    Future<ODataRequestResultMultipartGeneric> simulatedPriceFuture = simulationPriceCalculationExecutorService.submit(simulatedPriceTask);
                    awaitTerminationAfterShutdown(simulationPriceCalculationExecutorService);
                    if(simulatedPriceFuture.isDone()){
                        final ODataRequestResultMultipartGeneric simulatedPriceBatchResult = simulatedPriceFuture.get();
                        try {
                            ExecutorService renegotiatedPriceCalculationExecutorService = Executors.newFixedThreadPool(20);
                            Future<ODataRequestResultMultipartGeneric> renegotiatedPriceFuture = renegotiatedPriceCalculationExecutorService.submit(renegotiatedPriceTask);
                            awaitTerminationAfterShutdown(renegotiatedPriceCalculationExecutorService);
                            if(renegotiatedPriceFuture.isDone()){
                                final ODataRequestResultMultipartGeneric renegotiatedPriceBatchResult = renegotiatedPriceFuture.get();
                                List<PriceCalculationResponseDto> netPriceCalculationResponseList = batchQueryResponse(header.getProjectUUID(), synchronizedNetPriceRequestActionList, netPriceBatchResult);
                                List<PriceCalculationResponseDto> simulatedPriceCalculationResponseList = batchQueryResponse(header.getProjectUUID(), synchronizedSimulatedPriceRequestActionList, simulatedPriceBatchResult);
                                List<PriceCalculationResponseDto> renegotiatedPriceCalculationResponseList = batchQueryResponse(header.getProjectUUID(), synchronizedRenegotiatedPriceRequestActionList, renegotiatedPriceBatchResult);
                                updateRecalculatedAmount(netPriceCalculationResponseList, 0);
                                updateRecalculatedAmount(simulatedPriceCalculationResponseList, 1);
                                updateRecalculatedAmount(renegotiatedPriceCalculationResponseList, 2);
                                negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM20.name());  
                            }                            
                        } catch (Exception e) {
                            negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM62.name());
                            log.info("Renegotiation task failed: " + e.getMessage());
                            e.printStackTrace();
                        }
                    }                    
                } catch (Exception e) {
                    negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM62.name());
                    log.info("Simulation task failed: " + e.getMessage());
                    e.printStackTrace();
                }
            }

        } catch (DestinationNotFoundException | DestinationAccessException e) {
            log.info("Exception occured: " + e.getMessage());
            negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM60.name());
        } catch (InterruptedException | ExecutionException e) {
            log.info("Exception occured: " + e.getMessage());
            negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM62.name());
        } catch (Exception e) {
            log.info("Exception occured: " + e.getMessage());
            negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM62.name());
        }
    }

    /*
     * This method is used to execute the batch request for the price calculation and returns the batch result.
     */
    private ODataRequestResultMultipartGeneric executeBatchRequest(String projectUUID, List<SimulationHeaderRequestDto> priceCalculationPayloadList, List<ODataRequestAction> synchronizedPriceRequestActionList)
    throws DestinationNotFoundException, DestinationAccessException {
        Header[] headers = null;
        HttpDestination simulationDestination = null;
        ODataRequestResultMultipartGeneric batchResult = null;

        headers = priceCalculationUtil.collectPostHeader(
        client.getHeaders(MessageFormat.format(HEADERS_POST_PATH, "'" + priceCalculationPayloadList.get(0).getPurchasingDocumentId() + "'")));
        simulationDestination = client.getSimulationDestination();

        if (simulationDestination != null) {
            log.info(Thread.currentThread().getName() + ": Simulation Destination not null");
            String servicePath = simulationDestination.getUri().getPath();

            ODataRequestBatch requestBatch = new ODataRequestBatch(servicePath, ODataProtocol.V4);

            addRequestBatchHeader(requestBatch, headers);

            addRequestActionToBatchRequest(servicePath, priceCalculationPayloadList, requestBatch, synchronizedPriceRequestActionList);

            batchResult = client.httpBatchPost(requestBatch, simulationDestination);
        }
        log.info(Thread.currentThread().getName() + "Price Calculation Completed");
        return batchResult;
    }

    /*
     * This method is used to add the headers to the batch request.
     */
    private void addRequestBatchHeader(ODataRequestBatch requestBatch, Header[] headers) {
        for(Header header: headers){
            requestBatch.addHeader(header.getName(), header.getValue());
          }
        requestBatch.addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
    }

    /*
     * This method is used to add the request action to the batch request.
     */
    private void addRequestActionToBatchRequest(String servicePath, List<SimulationHeaderRequestDto> priceCalculationPayloadList, ODataRequestBatch requestBatch, List<ODataRequestAction> synchronizedPriceRequestActionList){
        priceCalculationPayloadList.forEach(simulationHeaderRequestPayload -> {
            ODataEntityKey key = new ODataEntityKey(ODataProtocol.V4).addKeyProperty(KEY_PROPERTY_NAME, simulationHeaderRequestPayload.getPurchasingDocumentId());
            ODataResourcePath actionPath = new ODataResourcePath().addSegment(KEY_PROPERTY_NAME,key).addSegment(ACTION_SEGMENT_NAME);
            ODataRequestAction requestAction = new ODataRequestAction(servicePath, actionPath, JsonUtils.to(simulationHeaderRequestPayload), ODataProtocol.V4);
            synchronizedPriceRequestActionList.add(requestAction);
            requestBatch.beginChangeset().addAction(requestAction).endChangeset();
        });
    }

    /*
     * This method is used to query the response from the batch request and return the price calculation response list.
     * It then updates the database with the recalculated amount.
     * In case of failure, messagecode is updated in the database.
     * Price Calculation Ongoing Status is updated to false irrespective of success/failure.
     */
    private List<PriceCalculationResponseDto> batchQueryResponse(String projectUUID, List<ODataRequestAction> requestActionList, ODataRequestResultMultipartGeneric batchResult){
        List<PriceCalculationResponseDto> priceCalculationResponseList = new ArrayList<>();
        log.info("Request Action List: " + requestActionList);
        requestActionList.forEach(requestAction -> {
            log.info("Request Action: " + requestAction);
            ODataRequestResultGeneric queryResult = batchResult.getResult(requestAction);
            log.info("Query Result: " + queryResult);
            try {
                if(!client.isResponseSuccessful(queryResult.getHttpResponse())){
                    negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM61.name());
                    return;
                }else{
                    PriceCalculationResponseDto priceCalculationResponse = new PriceCalculationResponseDto();
                    priceCalculationResponse.setProjectUUID(projectUUID);
                    byte[] responseByte = queryResult.getHttpResponse().getEntity().getContent().readAllBytes();
                    JSONObject responseJson = new JSONObject(new String(responseByte,StandardCharsets.UTF_8));
                    JSONArray responseValueArray = responseJson.getJSONArray("value");

                    String centralPurchaseContract = responseValueArray.getJSONObject(0).get("PURCHASINGDOCUMENT").toString();
                    String centralPurchaseContractItem = responseValueArray.getJSONObject(0).get("PURCHASINGDOCUMENTITEM").toString();
                    priceCalculationResponse.setCentralPurchaseContract(centralPurchaseContract);
                    priceCalculationResponse.setCentralPurchaseContractItem(centralPurchaseContractItem);
                    log.info("Central Purchase Contract: " + centralPurchaseContract + " Central Purchase Contract Item: " + centralPurchaseContractItem);

                    JSONArray responsePricingTotalArray = responseValueArray.getJSONObject(0).getJSONArray("_PRICINGTOTAL");
                    BigDecimal netPriceAmount = responsePricingTotalArray.getJSONObject(0).getBigDecimal("NETPRICEAMOUNT");
                    BigDecimal effectiveAmount = responsePricingTotalArray.getJSONObject(0).getBigDecimal("EFFECTIVEAMOUNT");
                    BigDecimal conditionQuantity = responsePricingTotalArray.getJSONObject(0).getBigDecimal("CONDITIONQUANTITY");
                    if(netPriceAmount.equals(BigDecimal.ZERO)){
                        netPriceAmount = effectiveAmount.divide(conditionQuantity); 
                    }
                    priceCalculationResponse.setAmount(netPriceAmount);
                    log.info("Net Price Amount: " + netPriceAmount + " Effective Amount: " + effectiveAmount);
                    priceCalculationResponseList.add(priceCalculationResponse);
                }
            } catch (IOException | ODataResponseException e) {
                log.info("Exception occured: " + e.getMessage());
                negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM62.name());
                return;
            } catch (Exception e) {
                log.info("Exception occured: " + e.getMessage());
                negotiationProjectHeaderService.updatePriceCalculationStatus(projectUUID, false, RenegotiationMessageCode.RM62.name());
                return;
            }  
        });
        return priceCalculationResponseList;
    }

    private void updateRecalculatedAmount(List<PriceCalculationResponseDto> priceCalculationResponseList, int priceType){
        priceCalculationResponseList.forEach(priceCalculationResponse -> {
            long rowCount = negotiationProjectItemService.priceItemUpdate(priceCalculationResponse, priceType);
            log.info("Row Count: " + rowCount);
        });
    }

    private void awaitTerminationAfterShutdown(ExecutorService threadPool) throws InterruptedException {
        threadPool.shutdown();
        if (!threadPool.awaitTermination(3000, TimeUnit.SECONDS)) {
            threadPool.shutdownNow();
        }
    }    

}

Impact / Priority

We need to finish this task by July 12 as this will be our dev close for the upcoming Q3 release. This feature is of high prio for our release.

Error Message

service logs.txt

Maven Dependency Tree

[INFO] com.sap.procrenego:proc-renego-srv:jar:1.0.0-SNAPSHOT [INFO] +- io.opentelemetry.javaagent:opentelemetry-javaagent:jar:2.3.0:runtime [INFO] +- com.sap.xdsr:otel-agent-ext-java:jar:1.5.14:runtime [INFO] | +- com.google.auto.service:auto-service:jar:1.1.1:runtime [INFO] | | +- com.google.auto.service:auto-service-annotations:jar:1.1.1:runtime [INFO] | | - com.google.auto:auto-common:jar:1.2.1:runtime [INFO] | +- io.opentelemetry:opentelemetry-exporter-otlp:jar:1.31.0:compile [INFO] | | +- io.opentelemetry:opentelemetry-sdk-trace:jar:1.31.0:compile [INFO] | | +- io.opentelemetry:opentelemetry-sdk-metrics:jar:1.31.0:compile [INFO] | | | - io.opentelemetry:opentelemetry-extension-incubator:jar:1.31.0-alpha:runtime [INFO] | | +- io.opentelemetry:opentelemetry-exporter-sender-okhttp:jar:1.31.0:runtime [INFO] | | - io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:jar:1.31.0:runtime [INFO] | +- io.opentelemetry:opentelemetry-exporter-otlp-common:jar:1.31.0:compile [INFO] | +- io.opentelemetry.proto:opentelemetry-proto:jar:1.2.0-alpha:compile [INFO] | | - com.google.protobuf:protobuf-java:jar:3.23.4:compile [INFO] | +- io.opentelemetry.semconv:opentelemetry-semconv:jar:1.25.0-alpha:compile [INFO] | - io.opentelemetry.semconv:opentelemetry-semconv-incubating:jar:1.25.0-alpha:compile [INFO] +- com.sap.xdsr:calm-crun-client-api-java:jar:1.5.15:compile [INFO] | +- io.opentelemetry:opentelemetry-sdk:jar:1.31.0:compile [INFO] | | +- io.opentelemetry:opentelemetry-api:jar:1.31.0:compile [INFO] | | | - io.opentelemetry:opentelemetry-context:jar:1.31.0:compile [INFO] | | - io.opentelemetry:opentelemetry-sdk-common:jar:1.31.0:compile [INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.15.4:compile [INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.15.4:compile [INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.15.4:compile [INFO] | +- io.opentelemetry:opentelemetry-sdk-logs:jar:1.31.0:compile [INFO] | | - io.opentelemetry:opentelemetry-api-events:jar:1.31.0-alpha:runtime [INFO] | +- io.opentelemetry:opentelemetry-api-incubator:jar:1.38.0-alpha:compile [INFO] | +- org.bouncycastle:bcprov-jdk18on:jar:1.78.1:compile [INFO] | +- org.bouncycastle:bcpkix-jdk18on:jar:1.78.1:compile [INFO] | | - org.bouncycastle:bcutil-jdk18on:jar:1.78.1:compile [INFO] | +- com.squareup.okhttp3:okhttp:jar:4.12.0:compile [INFO] | | +- com.squareup.okio:okio:jar:3.6.0:compile [INFO] | | | - com.squareup.okio:okio-jvm:jar:3.6.0:compile [INFO] | | | - org.jetbrains.kotlin:kotlin-stdlib-common:jar:1.9.22:compile [INFO] | | - org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:1.9.22:compile [INFO] | | +- org.jetbrains.kotlin:kotlin-stdlib:jar:1.9.22:compile [INFO] | | - org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar:1.9.22:compile [INFO] | +- com.sap.xdsr:fesr-to-otel-java:jar:1.5.15:compile [INFO] | | - commons-io:commons-io:jar:2.16.1:compile [INFO] | - io.opentelemetry:opentelemetry-exporter-common:jar:1.31.0:compile [INFO] +- com.sap.cloud.servicesdk.xbem:emjapi-connector-sap-cp:jar:2.2.6:compile [INFO] | +- org.springframework.cloud:spring-cloud-cloudfoundry-connector:jar:2.0.7.RELEASE:compile [INFO] | | - org.springframework.cloud:spring-cloud-connectors-core:jar:2.0.7.RELEASE:compile [INFO] | - org.slf4j:slf4j-api:jar:2.0.12:compile [INFO] +- io.pivotal.cfenv:java-cfenv-boot:jar:3.1.3:compile [INFO] | +- io.pivotal.cfenv:java-cfenv:jar:3.1.3:compile [INFO] | | - com.cedarsoftware:json-io:jar:4.15.0:compile [INFO] | +- io.pivotal.cfenv:java-cfenv-jdbc:jar:3.1.3:compile [INFO] | - org.springframework.boot:spring-boot:jar:3.2.3:compile [INFO] +- com.sap.cloud.servicesdk.xbem:emjapi-core:jar:4.0.0:compile [INFO] | - jakarta.jms:jakarta.jms-api:jar:3.1.0:compile [INFO] +- com.sap.cloud.servicesdk.xbem:emjapi-extension-sap-cp-jms:jar:4.0.0:compile [INFO] | +- org.apache.qpid:qpid-jms-client:jar:2.2.0:compile [INFO] | | - org.apache.qpid:proton-j:jar:0.34.0:compile [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile [INFO] | +- commons-codec:commons-codec:jar:1.16.1:compile [INFO] | +- io.netty:netty-buffer:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-common:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-handler-proxy:jar:4.1.107.Final:compile [INFO] | | - io.netty:netty-codec-socks:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-transport:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.107.Final:compile [INFO] | | - io.netty:netty-transport-classes-epoll:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.107.Final:compile [INFO] | | - io.netty:netty-transport-classes-kqueue:jar:4.1.107.Final:compile [INFO] | - io.netty:netty-codec-http:jar:4.1.108.Final:compile [INFO] +- org.springframework.security:spring-security-config:jar:6.2.2:compile [INFO] | +- org.springframework:spring-aop:jar:6.1.4:compile [INFO] | +- org.springframework:spring-beans:jar:6.1.4:compile [INFO] | +- org.springframework:spring-context:jar:6.1.4:compile [INFO] | - org.springframework:spring-core:jar:6.1.4:compile [INFO] | - org.springframework:spring-jcl:jar:6.1.4:compile [INFO] +- org.springframework.security:spring-security-core:jar:6.2.3:compile [INFO] | +- org.springframework.security:spring-security-crypto:jar:6.2.2:compile [INFO] | +- org.springframework:spring-expression:jar:6.1.4:compile [INFO] | - io.micrometer:micrometer-observation:jar:1.12.3:compile [INFO] | - io.micrometer:micrometer-commons:jar:1.12.3:compile [INFO] +- org.springframework.boot:spring-boot-starter-aop:jar:3.2.3:compile [INFO] | +- org.springframework.boot:spring-boot-starter:jar:3.2.3:compile [INFO] | | +- org.springframework.boot:spring-boot-starter-logging:jar:3.2.3:compile [INFO] | | | +- org.apache.logging.log4j:log4j-to-slf4j:jar:2.21.1:compile [INFO] | | | | - org.apache.logging.log4j:log4j-api:jar:2.21.1:compile [INFO] | | | - org.slf4j:jul-to-slf4j:jar:2.0.12:compile [INFO] | | - jakarta.annotation:jakarta.annotation-api:jar:2.1.1:compile [INFO] | - org.aspectj:aspectjweaver:jar:1.9.21:compile [INFO] +- org.yaml:snakeyaml:jar:2.2:compile [INFO] +- org.springframework.boot:spring-boot-starter-actuator:jar:3.2.3:compile [INFO] | +- org.springframework.boot:spring-boot-actuator-autoconfigure:jar:3.2.3:compile [INFO] | | +- org.springframework.boot:spring-boot-actuator:jar:3.2.3:compile [INFO] | | - com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.15.4:compile [INFO] | - io.micrometer:micrometer-jakarta9:jar:1.12.3:compile [INFO] | - io.micrometer:micrometer-core:jar:1.12.3:compile [INFO] | +- org.hdrhistogram:HdrHistogram:jar:2.1.12:runtime [INFO] | - org.latencyutils:LatencyUtils:jar:2.0.3:runtime [INFO] +- com.sap.cds:cds-starter-spring-boot:jar:2.7.1:compile [INFO] | +- com.sap.cds:cds-services-api:jar:2.7.1:compile [INFO] | | +- com.sap.cds:cds4j-api:jar:2.7.1:compile [INFO] | | - com.sap.cloud.environment.servicebinding.api:java-core-api:jar:0.10.2:compile [INFO] | +- com.sap.cds:cds-services-impl:jar:2.7.1:runtime [INFO] | | +- com.sap.cds:cds-services-messaging:jar:2.7.1:runtime [INFO] | | +- com.sap.cloud.environment.servicebinding.api:java-access-api:jar:0.10.2:compile [INFO] | | - org.apache.commons:commons-csv:jar:1.10.0:runtime [INFO] | +- com.sap.cds:cds-feature-jdbc:jar:2.7.1:runtime [INFO] | | +- com.sap.cds:cds4j-runtime:jar:2.7.1:runtime [INFO] | | | - com.sap.cds:cds4j-jdbc-spi:jar:2.7.1:runtime [INFO] | | - com.sap.cds:cds4j-tx:jar:2.7.1:runtime [INFO] | +- com.sap.cds:cds-framework-spring-boot:jar:2.7.1:runtime [INFO] | +- org.springframework.boot:spring-boot-starter-web:jar:3.2.3:compile [INFO] | | +- org.springframework.boot:spring-boot-starter-tomcat:jar:3.2.3:compile [INFO] | | | +- org.apache.tomcat.embed:tomcat-embed-el:jar:10.1.19:compile [INFO] | | | - org.apache.tomcat.embed:tomcat-embed-websocket:jar:10.1.19:compile [INFO] | | - org.springframework:spring-webmvc:jar:6.1.4:compile [INFO] | - org.springframework.boot:spring-boot-starter-jdbc:jar:3.2.3:compile [INFO] | +- com.zaxxer:HikariCP:jar:5.0.1:compile [INFO] | - org.springframework:spring-jdbc:jar:6.1.4:compile [INFO] | - org.springframework:spring-tx:jar:6.1.4:compile [INFO] +- com.sap.cloud.sdk.s4hana:s4hana-connectivity:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:cloudplatform-core:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:auditlog:jar:4.27.0:compile [INFO] | | - com.sap.cloud.sdk.cloudplatform:tenant:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:cloudplatform-connectivity:jar:4.27.0:compile [INFO] | | +- com.sap.cloud.sdk.cloudplatform:resilience:jar:4.27.0:compile [INFO] | | - com.mikesamuel:json-sanitizer:jar:1.2.3:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:security:jar:4.27.0:compile [INFO] | | - com.auth0:java-jwt:jar:4.4.0:compile [INFO] | +- com.sap.cloud.sdk.s4hana:s4hana-core:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.datamodel:fluent-result:jar:4.27.0:compile [INFO] | +- org.slf4j:jcl-over-slf4j:jar:2.0.12:runtime [INFO] | +- io.vavr:vavr:jar:0.10.4:compile [INFO] | | - io.vavr:vavr-match:jar:0.10.4:compile [INFO] | +- com.google.guava:guava:jar:32.1.3-jre:compile [INFO] | | +- com.google.guava:failureaccess:jar:1.0.1:compile [INFO] | | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile [INFO] | | +- org.checkerframework:checker-qual:jar:3.37.0:compile [INFO] | | +- com.google.errorprone:error_prone_annotations:jar:2.21.1:compile [INFO] | | - com.google.j2objc:j2objc-annotations:jar:2.8:compile [INFO] | - org.apache.httpcomponents:httpcore:jar:4.4.16:compile [INFO] +- com.sap.cds:cds-starter-cloudfoundry:jar:2.7.1:compile [INFO] | +- com.sap.cds:cds-feature-hana:jar:2.7.1:compile [INFO] | | - com.sap.cloud.db.jdbc:ngdbc:jar:2.19.16:compile [INFO] | +- com.sap.cds:cds-feature-mt:jar:2.7.1:compile [INFO] | | +- com.sap.cloud.mt:multi-tenant-runtime:jar:2.7.1:compile [INFO] | | - com.sap.cloud.mt:multi-tenant-subscription:jar:2.7.1:compile [INFO] | | - com.sap.cloud.instancemanager:client:jar:3.14.0:compile [INFO] | | - com.sap.hcp.cf.logging:cf-java-logging-support-logback:jar:3.6.3:compile [INFO] | | - com.sap.hcp.cf.logging:cf-java-logging-support-core:jar:3.6.3:compile [INFO] | | - com.fasterxml.jackson.jr:jackson-jr-objects:jar:2.15.4:compile [INFO] | +- com.sap.cds:cds-feature-xsuaa:jar:2.7.1:compile [INFO] | - com.sap.cds:cds-feature-identity:jar:2.7.1:compile [INFO] +- com.sap.cds:cds-feature-cloudfoundry:jar:2.7.1:compile [INFO] | +- com.sap.cds:cds-services-utils:jar:2.7.1:compile [INFO] | | - com.sap.cloud.mt:tools:jar:2.7.1:compile [INFO] | - com.sap.cloud.environment.servicebinding:java-sap-vcap-services:jar:0.10.2:compile [INFO] +- com.sap.cloud.sdk.cloudplatform:scp-cf:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:auditlog-scp-cf:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:caching:jar:4.27.0:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:cloudplatform-core-scp-cf:jar:4.27.0:compile [INFO] | | - com.sap.cloud.environment.servicebinding.api:java-consumption-api:jar:0.10.1:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:cloudplatform-connectivity-scp-cf:jar:4.27.0:compile [INFO] | | +- com.sap.cloud.sdk.cloudplatform:cloudplatform-connectivity-scp:jar:4.27.0:compile [INFO] | | +- com.sap.cloud.sdk.cloudplatform:connectivity-oauth:jar:4.27.0:compile [INFO] | | - com.sap.cloud.sdk.frameworks:resilience4j:jar:4.27.0:compile [INFO] | | +- io.github.resilience4j:resilience4j-circuitbreaker:jar:1.7.1:compile [INFO] | | | - io.github.resilience4j:resilience4j-core:jar:1.7.1:compile [INFO] | | +- io.github.resilience4j:resilience4j-bulkhead:jar:1.7.1:compile [INFO] | | +- io.github.resilience4j:resilience4j-timelimiter:jar:1.7.1:compile [INFO] | | +- io.github.resilience4j:resilience4j-retry:jar:1.7.1:compile [INFO] | | +- io.github.resilience4j:resilience4j-ratelimiter:jar:1.7.1:compile [INFO] | | - javax.cache:cache-api:jar:1.1.1:compile [INFO] | +- com.sap.cloud.sdk.cloudplatform:security-scp-cf:jar:4.27.0:compile [INFO] | - com.sap.cloud.sdk.cloudplatform:tenant-scp-cf:jar:4.27.0:compile [INFO] +- com.sap.cloud.security.xsuaa:token-client:jar:3.3.0:compile [INFO] | +- com.sap.cloud.security:java-api:jar:3.3.0:compile [INFO] | +- org.json:json:jar:20231013:compile [INFO] | - com.github.ben-manes.caffeine:caffeine:jar:3.1.8:compile [INFO] +- com.sap.cds:cds-adapter-odata-v2:jar:2.7.1:runtime [INFO] | +- com.sap.cds:cds-adapter-api:jar:2.7.1:compile [INFO] | +- com.sap.cds:cds4j-core:jar:2.7.1:compile [INFO] | +- com.sap.cloud.mt:cds-mtx:jar:2.7.1:compile [INFO] | +- com.sap.cds.repackaged:odata-v2-lib:jar:2.7.1:runtime [INFO] | | - javax.ws.rs:javax.ws.rs-api:jar:2.1.1:runtime [INFO] | - org.apache.commons:commons-lang3:jar:3.13.0:compile [INFO] +- com.sap.cds:cds-adapter-odata-v4:jar:2.7.1:runtime [INFO] | - com.sap.cds.repackaged:odata-v4-lib:jar:2.7.1:runtime [INFO] | - com.fasterxml:aalto-xml:jar:1.3.2:runtime [INFO] +- com.sap.cds:cds-feature-auditlog-v2:jar:2.7.1:runtime [INFO] | +- com.sap.cds:cds-integration-cloud-sdk:jar:2.7.1:compile [INFO] | +- com.sap.cloud.sjb:xs-env:jar:1.49.0:runtime [INFO] | +- com.sap.cloud.security:java-security:jar:3.3.0:compile [INFO] | - com.sap.cloud.security:env:jar:3.3.0:compile [INFO] | - com.sap.cloud.environment.servicebinding:java-sap-service-operator:jar:0.10.1:compile [INFO] +- org.springframework.boot:spring-boot-starter-cache:jar:3.2.3:compile [INFO] | - org.springframework:spring-context-support:jar:6.1.4:compile [INFO] +- com.sap.cloud.sdk.datamodel:odata-client:jar:4.27.0:compile [INFO] +- org.springframework.security:spring-security-oauth2-client:jar:6.2.2:compile [INFO] | +- org.springframework.security:spring-security-oauth2-core:jar:6.2.2:compile [INFO] | - com.nimbusds:oauth2-oidc-sdk:jar:9.43.3:compile [INFO] | +- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile [INFO] | +- com.nimbusds:content-type:jar:2.2:compile [INFO] | +- com.nimbusds:lang-tag:jar:1.7:compile [INFO] | - com.nimbusds:nimbus-jose-jwt:jar:9.37.3:compile [INFO] +- com.sap.cloud.sdk.cloudplatform:servlet-jakarta:jar:4.27.0:compile [INFO] +- org.springframework.security:spring-security-web:jar:6.2.2:compile [INFO] | - org.springframework:spring-web:jar:6.1.6:compile [INFO] +- org.springframework.boot:spring-boot-starter-security:jar:3.2.3:compile [INFO] +- com.sap.cloud.security:resourceserver-security-spring-boot-starter:jar:3.3.0:compile [INFO] | +- com.sap.cloud.security:spring-security:jar:3.3.0:compile [INFO] | +- org.springframework.security:spring-security-oauth2-jose:jar:6.2.2:compile [INFO] | +- org.springframework.security:spring-security-oauth2-resource-server:jar:6.2.2:compile [INFO] | - org.springframework.boot:spring-boot-autoconfigure:jar:3.2.3:compile [INFO] +- org.apache.httpcomponents.client5:httpclient5:jar:5.2.3:compile [INFO] | +- org.apache.httpcomponents.core5:httpcore5:jar:5.2.4:compile [INFO] | - org.apache.httpcomponents.core5:httpcore5-h2:jar:5.2.4:compile [INFO] +- org.xerial:sqlite-jdbc:jar:3.43.2.2:compile [INFO] +- org.projectlombok:lombok:jar:1.18.30:compile [INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.15.4:compile [INFO] | - org.codehaus.woodstox:stax2-api:jar:4.2.1:compile [INFO] +- com.fasterxml.woodstox:woodstox-core:jar:6.4.0:compile [INFO] +- org.mapstruct:mapstruct:jar:1.5.5.Final:compile [INFO] +- com.google.code.gson:gson:jar:2.10.1:compile [INFO] +- net.minidev:json-smart:jar:2.5.0:compile [INFO] | - net.minidev:accessors-smart:jar:2.5.0:compile [INFO] | - org.ow2.asm:asm:jar:9.3:compile [INFO] +- org.apache.tomcat.embed:tomcat-embed-core:jar:10.1.19:compile [INFO] | - org.apache.tomcat:tomcat-annotations-api:jar:10.1.19:compile [INFO] +- org.springframework.boot:spring-boot-starter-test:jar:3.2.3:test [INFO] | +- org.springframework.boot:spring-boot-test:jar:3.2.3:test [INFO] | +- org.springframework.boot:spring-boot-test-autoconfigure:jar:3.2.3:test [INFO] | +- com.jayway.jsonpath:json-path:jar:2.9.0:test [INFO] | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:4.0.1:test [INFO] | | - jakarta.activation:jakarta.activation-api:jar:2.1.2:test [INFO] | +- org.assertj:assertj-core:jar:3.24.2:test [INFO] | | - net.bytebuddy:byte-buddy:jar:1.14.12:test [INFO] | +- org.awaitility:awaitility:jar:4.2.0:test [INFO] | +- org.hamcrest:hamcrest:jar:2.2:test [INFO] | +- org.junit.jupiter:junit-jupiter:jar:5.10.2:test [INFO] | | +- org.junit.jupiter:junit-jupiter-api:jar:5.10.2:test [INFO] | | | +- org.opentest4j:opentest4j:jar:1.3.0:test [INFO] | | | +- org.junit.platform:junit-platform-commons:jar:1.10.2:test [INFO] | | | - org.apiguardian:apiguardian-api:jar:1.1.2:test [INFO] | | +- org.junit.jupiter:junit-jupiter-params:jar:5.10.2:test [INFO] | | - org.junit.jupiter:junit-jupiter-engine:jar:5.10.2:test [INFO] | | - org.junit.platform:junit-platform-engine:jar:1.10.2:test [INFO] | +- org.mockito:mockito-core:jar:5.7.0:test [INFO] | | +- net.bytebuddy:byte-buddy-agent:jar:1.14.12:test [INFO] | | - org.objenesis:objenesis:jar:3.3:test [INFO] | +- org.mockito:mockito-junit-jupiter:jar:5.7.0:test [INFO] | +- org.skyscreamer:jsonassert:jar:1.5.1:test [INFO] | +- org.springframework:spring-test:jar:6.1.4:test [INFO] | - org.xmlunit:xmlunit-core:jar:2.9.1:test [INFO] +- ch.qos.logback:logback-classic:jar:1.4.14:compile [INFO] +- ch.qos.logback:logback-core:jar:1.4.14:compile [INFO] +- org.springframework.boot:spring-boot-starter-webflux:jar:3.2.3:compile [INFO] | +- org.springframework.boot:spring-boot-starter-json:jar:3.2.3:compile [INFO] | | +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.15.4:compile [INFO] | | - com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.15.4:compile [INFO] | +- org.springframework.boot:spring-boot-starter-reactor-netty:jar:3.2.3:compile [INFO] | - org.springframework:spring-webflux:jar:6.1.4:compile [INFO] +- io.netty:netty-handler:jar:4.1.108.Final:compile [INFO] | +- io.netty:netty-resolver:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-transport-native-unix-common:jar:4.1.107.Final:compile [INFO] | - io.netty:netty-codec:jar:4.1.107.Final:compile [INFO] +- io.netty:netty-codec-http2:jar:4.1.108.Final:compile [INFO] +- io.projectreactor.netty:reactor-netty-http:jar:1.1.16:compile [INFO] | +- io.netty:netty-resolver-dns:jar:4.1.107.Final:compile [INFO] | | - io.netty:netty-codec-dns:jar:4.1.107.Final:compile [INFO] | +- io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.107.Final:compile [INFO] | | - io.netty:netty-resolver-dns-classes-macos:jar:4.1.107.Final:compile [INFO] | +- io.projectreactor.netty:reactor-netty-core:jar:1.1.16:compile [INFO] | - io.projectreactor:reactor-core:jar:3.6.3:compile [INFO] | - org.reactivestreams:reactive-streams:jar:1.0.4:compile [INFO] - com.sap.jdsr:passport:jar:2.2.2:compile

Project Details

This is provided above.

• SDK Version: v4
• Link to GitHub repo: https://github.tools.sap/Procurement-Renegotiations/renegotiation-core
• Project type, for example:
  • [X] CAP Project
  • [ ] SDK Maven Archetype
  • [ ] None of the above:
• Platform:
  • [X] Cloud Foundry
  • [ ] Deploy with Confidence (Cloud Foundry)
  • [ ] None of the above:

---

Checklist

• [X] Checked out the documentation and Stack Overflow
• [X] Description provided with all relevant information
• [X] Exception and stack trace provided
• [ ] Attached debug logs
• [X] Attached dependency tree
• [X] Provided Cloud SDK version & link to relevant source code

[MatKuhr]

First of all, please note that the 404 you are receiving is the result of an individual item within the batch, not the overall batch request. So "only" some part of the batch failed. It seems likely that one of your requests inside the batch is referencing some entity or object that the server can't find, thus returning a 404 for that particular change set. Please double check the IDs you are using and how you are building the requests.

Finally, if you do believe your requests are correct please get in contact with the maintainers of the OData service and share with them the exact batch request body (which you can get via request.getBatchRequestBody()).

Some further remarks:

• You are using a no longer supported version of Cloud SDK, please upgrade to the latest version 5
• Is there any particular reason you are building these requests manually with the low-level API, and not using a generated client?
• I don't see the point of executing the requests in separate threads, if you anyhow await termination before running the next. You might as well run everything in the main thread.
  • In particular, I don't understand why you would create new empty thread contexts and request contexts..

Overall, your code seems overly complex. I'd recommend reducing it down to the simplest possible case, removing all threading and only adding it back in, if you actually do want to run things in parallel. And if you do, using simply Future runningTask = ThreadContextExecutors.submit(myTask); should suffice (see the docs).

[I320857]

Hi Matthias,

Added few clarifications on certain decisions:

1. Is there any particular reason you are building these requests manually with the low-level API, and not using a generated client? We went with low-level API as the metadata of S/4 Odata can keep changing and we would have had to update our resource folder regularly on every change. Also, S/4 release will be different than ours and in case of breaking changes in API, it can cause downtime for our users in BTP.

   2. I don't see the point of executing the requests in separate threads, if you anyhow await termination before running the next. You might as well run everything in the main thread. We have created 3 threads because we have 3 different kind of price calculation we had to do from S/4 side. For each kind of price calculation, we have a created a callable task to fetch results in a batch call. Initially, we had submitted all three tasks in one executor service together but then we realised there was an issue with csrf token handling due to HttpClientCache. To avoid that we are submitting tasks one after another. Also, for each task, a new ExecutorService instance has been created. This was done so that we thought we can give each task a maximum of 15 minutes for it to process in case of large number of items.

Thanks and Regards, Sandesh