Johannes-Schneider
commented
7 months ago Hi @VerletzterDigitusPedis,
thanks for reaching out to us.
What you are observing is part of a problem we (the Cloud SDK) decided not to tap into: User session management.
Explanation
Let me try to give a very brief summary of what is going on here in the background. Your S/4 system requires a CSRF token for any request that tries to manipulate data. Such a token can be requested by attaching an `x-csrf-token: fetch` header to a "pre-flight" request (i.e. a `HEAD` request against the service root path). Within the response, the S/4 system will return an `x-csrf-token` header with a value that is internally (within the S/4 system) matched to a user session. **Additionally** (and this is the most important part), the S/4 system also sends an HTTP cookie in its response. This cookie ends up being stored within the HTTP client instance that was used to send the request. Now, as you may know, the Cloud SDK implements a [cache for HTTP clients](https://sap.github.io/cloud-sdk/docs/java/features/connectivity/http-client#configuring-the-cache) to reduce the amount of time it takes for subsequent requests to be executed. This cache, by the way, is the reason why you are seeing the "every 5 minutes"-pattern: The default caching duration for our HTTP clients is exactly 5 minutes. Now in a single-threaded scenario, all of this comes down to a flow like this: ``` myService.createMyEntity(myEntity).executeRequest(myDestination) HttpClientAccessor.getHttpClient(myDestination) -> check if client is cached <- no -> create new http client <- new http client (using the returned HTTP client) -> check if csrf token header is present <- no -> fetch csrf token -> HEAD http://s4/myService/ x-csrf-token: fetch <- 200 x-csrf-token: some-token Set-Cookie: some-cookie (using the csrf token) -> POST http://s4/myService/myEntity x-csrf-token: some-token Cookie: some-cookie <- 200 --- (end of first OData request) myService.updateMyEntity(myEntity).executeRequest(myDestination) HttpClientAccessor.getHttpClient(myDestination) -> check if client is cached <- yes <- cached http client (using the returned HTTP client) -> check if csrf token header is present <- no -> fetch csrf token -> HEAD http://s4/myService/ x-csrf-token: fetch cookie: some-cookie <- 200 x-csrf-token: some-other-token Set-Cookie: some-other-cookie (using the csrf token) -> PATCH http://s4/myService/myEntity x-csrf-token: some-other-token Cookie: some-other-cookie <- 200 ``` This flow somewhat works as expected, but is of course suboptimal as we are running additional HEAD requests that are not necessary. In a multi-threaded scenario (like yours), the situation changes. If we are running multiple OData requests in parallel, there is a chance that we are fetching two (or more) CSRF tokens in parallel. This is a problem because the HTTP client instance is actually **shared** between all threads. As such, we end up overwriting the csrf-cookie in our shared HTTP client instance every time we are retrieving the csrf token from the S/4 system. By doing that, we end up with a mismatch between the `x-csrf-token` header and the csrf-cookie within our `POST`/`PATCH` request. This mismatch is causing the error you are seeing. **NOTE** I'm not an expert of how exactly all of this CSRF logic is implemented in the S/4 backend system. So please take my explanation with a grain of salt.Proposed Solution
To solve the issue, I recommend running the pre-flight (i.e. HEAD) request manually and just once.
Here is an example of how that could look:
final HttpClient httpClient = HttpClientAccessor.getHttpClient(destination);
final CsrfToken csrfToken = new DefaultCsrfTokenRetriever().retrieveCsrfToken(httpClient, SdkGroceryStoreService.DEFAULT_SERVICE_PATH);
// execute X times
ThreadContextExecutors.execute(() -> {
new DefaultSdkGroceryStoreService()
.deleteShelf(myShelf)
.withHeader(DefaultCsrfTokenRetriever.X_CSRF_TOKEN_HEADER_KEY, csrfToken.getToken())
.toRequest()
.execute(httpClient);
});Hope that makes sense!
Best regards, Johannes
VerletzterDigitusPedis
Hello everyone,
Issue Description
We are using the SAP Cloud SDK to connect with S/4HANA (on premise) in order to transfer business data from our custom coded source systems to S/4HANA. We are currently using version 5.2.0 of the Java SDK. For our VDM we are using the generator, so the dependency is
With the configuration
We use the SAP Cloud SDK in a multi-threaded way and sometimes receive errors where the CSRF token fetching fails. I would like to know if this is a problem with the SAP Cloud SDK itself or something that has to be configured in S/4HANA itself. To me it seems like multiple HEAD Requests at the same time seem to be the problem. We also noticed a pattern, where it typically happened every 5 minutes for a few requests. I also tried to reproduce it in an isolated test and I was at least somewhat successful. Here is the code:
This is a part of the output of the test code. This shows a similair pattern. A lot of requests fail at the same time, then everything seems to be fine for 5 minutes and the error continues.
I should add, that we also do load testing via K6 with virtual users and do not have the same problem there. Same with PI/CPI where we also use ODATA directly (although the PI handles the token fetching itself). Can you explain why and how it happens and maybe even the 5 minute pattern?
Impact / Priority
This affects our development and testing phase. We are discussing whether we should keep the CSRF mechanism at all.
Impact: Impaired
Timeline: Implmentation is scheduled to be finished in September but it already impairs our testing.
Error Message
If application logs are needed, then I can provide them as well. Given the amount of requests needed to reproduce this error, I omitted them for now. If you need a specific part, then just give me a heads up and I will try to add them.
Project Details
Checklist
Thank you for your valuable time and your support :)