latacora / unsiemly

A Clojure abstraction for sending structured data to SIEMs
Eclipse Public License 1.0
3 stars 0 forks source link

Test GCP client creation #9

Open lvh opened 7 years ago

lvh commented 7 years ago

If this is done in an environment with no GCP creds, project ID... configured, you get the following exception when evaluating (LoggingOptions/getDefaultInstance):

1. Unhandled java.lang.IllegalArgumentException
   A project ID is required for this service but could not be determined from
   the builder or the environment. Please set a project ID using the builder.

Per the exception, I tried:

(-> (LoggingOptions/newBuilder) (.setProjectId project-id) (.build)

The source code confirms that (.build (LoggingOptions/newBuilder)) is what the default implementation ((LoggingOptions/getDefaultInstance)) does anyway, so I was hopeful this would work. It does evaluate to a LoggingOptions instance, but when you call getService you get a totally inscrutable error message:

(-> (LoggingOptions/newBuilder)
    (.setProjectId "my fake project id")
    (.build)
    (.getService))
1. Unhandled java.lang.NoSuchMethodError
   com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;J)V

          Semaphore64.java:   86  com.google.api.gax.batching.Semaphore64/notNegative
          Semaphore64.java:   46  com.google.api.gax.batching.Semaphore64/<init>
       FlowController.java:  146  com.google.api.gax.batching.FlowController/<init>
       BatcherFactory.java:   79  com.google.api.gax.grpc.BatcherFactory/<init>
 BatchingCallSettings.java:   56  com.google.api.gax.grpc.BatchingCallSettings/create
        UnaryCallable.java:  215  com.google.api.gax.grpc.UnaryCallable/create
        LoggingClient.java:  159  com.google.cloud.logging.v2.LoggingClient/<init>
        LoggingClient.java:  136  com.google.cloud.logging.v2.LoggingClient/create
       GrpcLoggingRpc.java:  131  com.google.cloud.logging.spi.v2.GrpcLoggingRpc/<init>
       LoggingOptions.java:   64  com.google.cloud.logging.LoggingOptions$DefaultLoggingRpcFactory/create
       LoggingOptions.java:   58  com.google.cloud.logging.LoggingOptions$DefaultLoggingRpcFactory/create
       ServiceOptions.java:  445  com.google.cloud.ServiceOptions/getRpc
       LoggingOptions.java:  134  com.google.cloud.logging.LoggingOptions/getLoggingRpcV2
          LoggingImpl.java:  108  com.google.cloud.logging.LoggingImpl/<init>
       LoggingOptions.java:   46  com.google.cloud.logging.LoggingOptions$DefaultLoggingFactory/create
       LoggingOptions.java:   41  com.google.cloud.logging.LoggingOptions$DefaultLoggingFactory/create
       ServiceOptions.java:  432  com.google.cloud.ServiceOptions/getService

Maybe I'm doing something wrong, but an error message that bad seems like it should probably be fixed upstream :)

As a consequence of all of this, I can't really test this part of the code. Suggestions include setting whatever JVM properties or whatever the code wants to successfully create that instance.

danielcompton commented 7 years ago

That is a terrible error message agreed, and should definitely be fixed. You may know this already, but one way to configure GCP libraries is with application default credentials. I suspect that error may be coming because that project doesn't exist? Although I don't really know.

lvh commented 7 years ago

@danielcompton Yep, the error happens in dev with no GCP creds set in the environment. I don't know if it works with a valid project but otherwise no creds set -- I presume there are no preauth APIs for it to hit.

lvh commented 6 years ago

@danielcompton Do you happen to know if the application default credentials are enabled by default if you don't specify other credentials? https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-core/src/main/java/com/google/cloud/ServiceOptions.java#L190 suggests yes, but I'm still getting the same error in prod.

danielcompton commented 6 years ago

GCS works for me without any explicit settings, this returns a com.google.cloud.storage.Storage.

(let [options (StorageOptions/getDefaultInstance)
            service (.getService options)]
        (assoc this :service service))

Your update to build-client looks fine to me. This seems a bit suspicious though: https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-logging/src/main/java/com/google/cloud/logging/spi/v2/GrpcLoggingRpc.java#L91. I got a bit lost in the layers of abstractions to figure out if the GCS client was doing something different.

lvh commented 6 years ago

Just to follow up: I'm now more likely to believe you're right and that it is a logging-specific issue. Storage client works fine.