VertexAI Flake: generateContentStream_withPlainText_nonEmptyCandidateList

[burkedavison]

From recent nightly build failure: https://fusion2.corp.google.com/invocations/11fc84ca-231a-4d90-a7cf-860ef92bb920/targets/cloud-devrel%2Fclient-libraries%2Fjava%2Fgoogle-cloud-java%2Fnightly%2Fgraalvm-sub-jobs%2Fnative-17%2Fgraalvm-native-17-13;config=default/log

Passed the next night, so this is a flake.

Failures (1):
  JUnit Vintage:ITGenerativeModelIntegrationTest:generateContentStream_withPlainText_nonEmptyCandidateList
    MethodSource [className = 'com.google.cloud.vertexai.it.ITGenerativeModelIntegrationTest', methodName = 'generateContentStream_withPlainText_nonEmptyCandidateList', methodParameterTypes = '']
    => com.google.api.gax.rpc.FailedPreconditionException: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: The output data is not valid json. Original output: {
    "predictions": [[NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN]
    ]
}.
       com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:102)
       com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
       com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
       com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
       com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82)
       [...]
       Suppressed: java.lang.RuntimeException: Asynchronous task failed
         com.google.api.gax.rpc.ServerStreamIterator.hasNext(ServerStreamIterator.java:105)
         com.google.cloud.vertexai.generativeai.ResponseStreamIteratorWithHistory.hasNext(ResponseStreamIteratorWithHistory.java:37)
         com.google.cloud.vertexai.it.ITGenerativeModelIntegrationTest.assertNonEmptyAndLogTextContentOfResponseStream(ITGenerativeModelIntegrationTest.java:122)
         com.google.cloud.vertexai.it.ITGenerativeModelIntegrationTest.generateContentStream_withPlainText_nonEmptyCandidateList(ITGenerativeModelIntegrationTest.java:218)
         java.base@17.0.8/java.lang.reflect.Method.invoke([Method.java:568](https://cs.corp.google.com/#search/&sq=package:%5Epiper$%20project:openjdk21&q=java/lang/reflect/Method.java:568))
     Caused by: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: The output data is not valid json. Original output: {
    "predictions": [[NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN]
    ]
}.
       io.grpc.Status.asRuntimeException(Status.java:533)
       [...]

[lqiu96]

@ZhenyiQ FYI

[suztomo]

@jaycee-li Would you check this issue too?

[jaycee-li]

This is a flaky test resulting from a backend error. No action is needed as it has only happened once.

[suztomo]

Thank you