gRPC client streaming does not work if an openidConnectClient is in use

[jonasrutishauser]

Describe the bug
If there is an openidConnectClient configured for JWT authentication and there is a gRPC service which uses client streaming (or bidirectional streaming) the client stream is always prematurely closed.

Steps to Reproduce
Using the grpc guide in the system server.xml the follwing should be added:

<server description="system service">

    <featureManager>
        <feature>restfulWS-3.1</feature>
        <!-- tag::grpc[] -->
        <feature>grpc-1.0</feature>
        <!-- end::grpc[] -->
        <feature>openidConnectClient-1.0</feature>
    </featureManager>

    <openidConnectClient id="test" inboundPropagation="required" issuerIdentifier="https://example.com/test" signatureAlgorithm="none"/>
[...]

In the test SystemServiceTest.java the inProcessChannel can be changed to inProcessChannel = ManagedChannelBuilder.forAddress("localhost", 9080).usePlaintext().build();. Now the method testGetClientStreamingProperties() should be changed to add a JWT token and wait some time between the properties like the following:

    public void testGetClientStreamingProperties() {
        [...]
        StreamObserver<SystemPropertyName> requestObserver =
            asyncStub.withCallCredentials(new CallCredentials() {
                @Override
                public void applyRequestMetadata(RequestInfo requestInfo, Executor appExecutor, MetadataApplier applier) {
                    Metadata metadata = new Metadata();
                    metadata.put(Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER), "Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJub25lIn0.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTcyNzYzODE2OSwiZXhwIjoxOTI3NjQxNzY5LCJncm91cHMiOlsidGVzdCJdLCJpc3MiOiJodHRwczovL2V4YW1wbGUuY29tL3Rlc3QifQ.");
                    applier.apply(metadata);
                }
            }).getClientStreamingProperties(responseObserver);
        List<String> keys = System.getProperties().stringPropertyNames().stream()
                                  .filter(k -> k.startsWith("user."))
                                  .collect(Collectors.toList());
        keys.stream()
            .map(k -> SystemPropertyName.newBuilder().setPropertyName(k).build())
            .peek(k -> {
                try {
                    Thread.sleep(100);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                }
            })
            .forEach(requestObserver::onNext);
        [...]
    }

If the test method is now run while the liberty server is running there is mostly only one property written to the server console instead of all:

[29.09.2024, 22:16:19:156 CEST] 000000f8 io.openliberty.guides.system.SystemService                   I client streaming received property: user.name
[29.09.2024, 22:16:19:157 CEST] 000000f8 io.openliberty.guides.system.SystemService                   I client streaming was completed!

Expected behavior
The stream is only closed when the client calls onCompleted() (or onError()).

Diagnostic information:

• OpenLiberty Version: * - 24.0.0.9
• Affected feature(s): grpc-1.0, openidConnectClient-1.0
• Java Version: all
• server.xml configuration: see above

Additional context
I think the problem is that the openidConnectClient feature stores all post requests (even if they are not complete or huge) which is a problem for client streaming services (we had to change one of our rest service to use PUT instead of POST to avoid out of memory problems, but gRPC always uses POST).

[jonasrutishauser]

I think the biggest problem is the following line: https://github.com/OpenLiberty/open-liberty/blob/1333c9d1605843b19c30c496cc647e879eb3d087/dev/com.ibm.ws.security.openidconnect.client/src/com/ibm/ws/security/openidconnect/client/internal/OidcClientImpl.java#L310

[ayoho]

Hi @jonasrutishauser. It's likely that this a related problem to https://github.com/OpenLiberty/open-liberty/issues/29902. The Security Sign-On team will need to investigate.