search

googleapis / google-api-java-client

Google APIs Client Library for Java
Apache License 2.0
1.34k stars 699 forks source link

Admin Directory. "IllegalArgumentException: key emails" random error. #2335

Closed roma2341 closed 8 months ago

roma2341 commented 1 year ago

Environment details

  1. Specify the API at the beginning of the title. For example, "BigQuery: ..."). General, Core, and Other are also allowed as types
  2. OS type and version: Ubuntu 16.04.2 LTS (Xenial Xerus)
  3. Java version: 15
  4. version(s): google-api-client(2.2.0) google-api-services-admin-directory(directory_v1-rev20230316-2.0.0)

Steps to reproduce

  1. Randomly use api to suspend\unsuspend gsuite users.
  2. After several days it start to throw "IllegalArgumentException: key emails" and suspend/unsuspend become completely broken until application restart.

Code example

@Service
@Slf4j
public class GoogleAuthAdminSDKService {

    String serviceAccountUserEmail = "SECRET";
    boolean allowGsuiteUsersDeletion = true;

    @Autowired
    GsonFactory JSON_FACTORY;

    @Autowired
    NetHttpTransport HTTP_TRANSPORT;

    static final String APPLICATION_NAME = "Google Calendar API Java Quickstart";

    private volatile GoogleCredential credential;

    public GoogleAuthAdminSDKService() throws IOException {
        credential = GoogleCredential.fromStream(new ClassPathResource("secret.json").getInputStream());
        credential = credential.createScoped(Collections.singletonList(DirectoryScopes.ADMIN_DIRECTORY_USER));
        //invalidateCredentialsIfRequired();
    }

    @SneakyThrows
    public void invalidateCredentialsIfRequired(){
        if(credential.getExpirationTimeMilliseconds() == null) {
            log.info("Google Admin token refreshed, because expiration time is NULL");
            credential.refreshToken();
            return;
        }
        if (credential.getExpirationTimeMilliseconds() < System.currentTimeMillis()) {
            // Access token has expired, refresh it using the refresh token
            log.info("Google Admin token refreshed, because TOKEN IS EXPIRED");
            credential.refreshToken();
        }
    }

    public boolean validUser(User user) {
        if(user.getPrimaryEmail() == null || user.getPrimaryEmail().isBlank())
            return false;
        return user.getName() != null;
    }

    public GoogleCredential getGCredentialForAdminSDK() throws IOException {
        return getGCredential(serviceAccountUserEmail, Collections.singletonList(DirectoryScopes.ADMIN_DIRECTORY_USER));
    }

    private synchronized GoogleCredential getGCredential(String userEmail, Collection<String> scopes) {
        //invalidateCredentialsIfRequired();
        PrivateKey privateKey = credential.getServiceAccountPrivateKey();

        return new GoogleCredential.Builder()
                .setTransport(HTTP_TRANSPORT)
                .setJsonFactory(JSON_FACTORY)
                .setServiceAccountId(credential.getServiceAccountId())
                .setTokenServerEncodedUrl(credential.getTokenServerEncodedUrl())
                .setServiceAccountPrivateKeyId(credential.getServiceAccountPrivateKeyId())
                .setServiceAccountUser(userEmail)
                .setServiceAccountScopes(scopes)
                .setServiceAccountPrivateKey(privateKey)
                .setServiceAccountProjectId(credential.getServiceAccountProjectId())
                .build();
    }

    public User createGmailAccount(User user) throws InvalidAttributesException, IOException {

        if(!validUser(user))
            throw new InvalidAttributesException("Invalid user name or email!!!");

        var adminSdkCred = getGCredentialForAdminSDK();

        Directory service = new Directory.Builder(HTTP_TRANSPORT, JSON_FACTORY, adminSdkCred)
                .setApplicationName(APPLICATION_NAME)
                .build();

        service.users().insert(user).execute();
        return user;
    }

    @SneakyThrows
    public void suspendGsuiteAccount(String userEmail,String whoAskToSuspend){
        if(!allowGsuiteUsersDeletion){
            String message = "(" + whoAskToSuspend + ")An attempt was made to suspend the gsuite account, " +
                    "this operation is disabled in the current environment. Account Name:" + userEmail;
            log.info(message);
            return;
        }
        try{
            var adminSdkCred = getGCredentialForAdminSDK();

            Directory service = new Directory.Builder(HTTP_TRANSPORT, JSON_FACTORY, adminSdkCred)
                    .setApplicationName(APPLICATION_NAME)
                    .build();

            var gUser = service.users().get(userEmail).execute();
            if(BooleanUtils.isTrue(gUser.getSuspended())){
                throw new RuntimeException("Error. Account is already suspended");
            }
            gUser.setSuspended(true);
            gUser.setSuspensionReason(whoAskToSuspend);
            service.users().patch(userEmail,gUser).execute();
            String gsuiteTerminationMessage = "(" + whoAskToSuspend + ")" + "Gsuite account was suspended:" + userEmail;
            log.info(gsuiteTerminationMessage);
        }
        catch (Exception e) {
            throw e;
        }
    }

    @SneakyThrows
    public void restoreSuspendedAccount(String userEmail,String whoAskToSuspend){
        if(!allowGsuiteUsersDeletion){
            String message = "(" + whoAskToSuspend + ")An attempt was made to restore the suspended gsuite account, " +
                    "this operation is disabled in the current environment. Account Name:" + userEmail;
            log.info(message);
            return;
        }
        try{
            var adminSdkCred = getGCredentialForAdminSDK();

            Directory service = new Directory.Builder(HTTP_TRANSPORT, JSON_FACTORY, adminSdkCred)
                    .setApplicationName(APPLICATION_NAME)
                    .build();

            var gUser = service.users().get(userEmail).execute();
            if(BooleanUtils.isFalse(gUser.getSuspended())){
                throw new RuntimeException("Error. Account is not suspended");
            }
            gUser.setSuspended(false);
            service.users().patch(userEmail,gUser).execute();
            String gsuiteTerminationMessage = "(" + whoAskToSuspend + ")" + "Suspended gsuite account was restored:" + userEmail;
            log.info(gsuiteTerminationMessage);
        }
        catch (Exception e){
            throw e;
        }
    }

    public void deleteGmailAccount(String userEmail, String whoAskToRemoveUser) throws IOException {
        if(!allowGsuiteUsersDeletion){
            String message = "(" + whoAskToRemoveUser + ")An attempt was made to delete the gsuite account, " +
                    "this operation is disabled in the current environment. Account Name:" + userEmail;
            log.info(message);
            return;
        }

        try{
            var adminSdkCred = getGCredentialForAdminSDK();

            Directory service = new Directory.Builder(HTTP_TRANSPORT, JSON_FACTORY, adminSdkCred)
                    .setApplicationName(APPLICATION_NAME)
                    .build();

            service.users().delete(userEmail).execute();
            String gsuiteTerminationMessage = "(" + whoAskToRemoveUser + ")" + "Gsuite account was removed:" + userEmail;
            log.info(gsuiteTerminationMessage);
        }
        catch (Exception e){
            throw e;
        }
    }
}

Stack trace

com.shs.exception.PublicCrmRuntimeException: java.lang.IllegalArgumentException: key emails
    at com.shs.crm.modules.integration.google.gsuite.GoogleAdminSDKController.restoreSuspendedGsuiteAccount(GoogleAdminSDKController.java:112)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
    at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:289)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:106)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at com.shs.crm.config.security.MultyReadBodyFilter.doFilter(MultyReadBodyFilter.java:24)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at com.shs.crm.config.security.JWTProxyTokenAuthFilter.doFilter(JWTProxyTokenAuthFilter.java:37)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at com.shs.config.security.BasicJWTTokenAuthFilter.doFilterInternal(BasicJWTTokenAuthFilter.java:55)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.web.servlet.resource.ResourceUrlEncodingFilter.doFilter(ResourceUrlEncodingFilter.java:67)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
    at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
    at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
    at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275)
    at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79)
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134)
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131)
    at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
    at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255)
    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79)
    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100)
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)
    at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
    at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2019)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1558)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1449)
    at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
    at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.IllegalArgumentException: key emails
    at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:900)
    at com.google.api.client.json.JsonParser.parse(JsonParser.java:360)
    at com.google.api.client.json.JsonParser.parse(JsonParser.java:335)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:79)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:73)
    at com.google.api.client.http.HttpResponse.parseAs(HttpResponse.java:460)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:603)
    at com.shs.crm.modules.integration.google.auth.GoogleAuthAdminSDKService.restoreSuspendedAccount(GoogleAuthAdminSDKService.java:172)
    at com.shs.crm.modules.integration.google.gsuite.GoogleAdminSDKController.restoreSuspendedGsuiteAccount(GoogleAdminSDKController.java:104)
    ... 95 more
Caused by: java.lang.IllegalArgumentException: key emails, field private java.lang.Object com.google.api.services.directory.model.User.emails
    at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:900)
    at com.google.api.client.json.JsonParser.parse(JsonParser.java:451)
    at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:787)
    ... 103 more
Caused by: java.lang.IllegalArgumentException: expected collection or array type but got class java.lang.Object
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:167)
    at com.google.api.client.util.Preconditions.checkArgument(Preconditions.java:67)
    at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:724)
    ... 105 more

External references such as API reference guides

Any additional information below

I frequently encounter this issue, which seems to occur randomly. The first consistent instance was when it occurred within the @scheduled task while adding email synchronization logic (gmail api). However, when I moved the code out of the @scheduled task, it worked fine. Unfortunately, the problem resurfaced when I added logic involving Admin SDK API calls, and now I'm unsure how to resolve it. The issue persists randomly and only goes away after restarting the application. We encountered issues when users of our CRM attempted to read their mail using the Gmail API. Unexpectedly, the API started throwing exceptions randomly, requiring us to restart the application to resolve the problem. Google libraries is barely usable because of this randomness, we are already thinking about migrating to other service provider. we use deprecated com.google.api.client.googleapis.auth.oauth2.GoogleCredential. could the problem be due to this? I tried to write test that suspends/unsuspends a lot of users, do it in parallel, but can't reproduce issue manually.

Possible relates to this issue

diegomarquezp commented 1 year ago

Hello @roma2341 , thanks for bringing this up. The latest version available for Admin Directory is in https://github.com/googleapis/google-api-java-client-services/tree/main/clients/google-api-services-admin/directory_v1/2.0.0 - I'm not aware of any fixes of this kind lately but it may be better to have the latest just in case.

The problem seems to occur when parsing the response from the authentication server - I'll take a look to that logic.

Regarding the replication example, how are the methods of the example code called? A full reproducer would be great to have.

edit: also, feel free to reach out to our support hub if you have a plan for faster response times

roma2341 commented 1 year ago

Here is their previous response, related to a problem with the Google API's (maybe it can help you, for me this answer was expected and useless):

Initially i created previous issue on Github: https://github.com/googleapis/google-api-java-client-services/issues/14823 But after a while i contacted google support:

Details of the google support response

Hello Anna, Just to set the right expectations, we don't have the proper tools and documentation to work with stack traces, however I went ahead as a best effort, and I was able to determine the following: For Gmail: The Java program threw an IllegalArgumentException with the message "key messages". The exception was thrown in the JsonParser class of the Google HTTP client library, which was being used by a Google Gmail service in the GoogleGmailService class. The stack trace also shows that the read method of the GmailBoxRESTController class was being executed when the exception occurred. It's hard to say exactly what's causing the exception without more context, but it looks like the JsonParser is failing to parse a JSON response from the Gmail API because it's missing a required key called "messages". It's possible that there's an issue with the request being sent to the Gmail API. For Calendar: The stack trace shows the sequence of method calls that led to the exception being thrown. At the top of the stack trace is the location where the exception was thrown, which appears to be in a JSON parser (com.google.api.client.json.JsonParser). The exception seems to indicate that there is a problem with a key named "items". It is possible that this key is missing, misspelled, or has an incorrect format. The code is likely attempting to parse a JSON response from an API call, and the response is not in the expected format. To resolve this issue, you should check the code that is making the API call and ensure that the key "items" is spelled correctly and is present in the JSON response. You should also ensure that the response is in the expected format. For Directory: The error message indicates that there is an illegal argument being passed into a method in the Java program. Specifically, it is indicating that there is an issue with the "emails" field of a user object in the Google API client for the Directory API. It appears that the value being passed in for the "emails" field is not of the expected type, which is causing the error. It is possible that the value being passed in is null, or is not in the correct format. To resolve this error, you will need to ensure that the value being passed in for the "emails" field is of the correct type and format, according to the requirements of the Directory API. You may need to consult the documentation to determine what the correct format is for this field. Just to set the right expectations again, please note the Google Workspace API team's primary focus is to guarantee the Google Workspace APIs functionality, and we don't have the actual ability to support app implementation, or code development. If all of the above doesn't solve this behavior, as I was requiring in my last email, please let me know about your desired outcome and an actual screenshot of any error message you may receive, or if you could record your screen while performing the steps you have mentioned and provide a more detailed explanation I would highly appreciate it, as this information stands to be mandatory for further investigation and if there's the case, to create an internal consult case with the higher-tier engineers. I will remain at your disposal in case you have any other doubts, inquiries or updates to share with me, one email away, so please do not hesitate to reach out to me, as it will be my pleasure to continue working with you. Have a lovely rest of your day! With many thanks and warm regards, Iulian Google Workspace Support

Unfortunately, the previous solution provided did not resolve the issue entirely. The perplexing aspect is that it intermittently functions without any problems before abruptly ceasing to work. If the data were incorrect, one would expect random failures, yet it consistently operates without issues for 2-3 days before stopping, sometimes even after just 5-6 hours. Initially, we suspected potential API limitations or insufficient RAM, but after conducting a benchmark, it performed smoothly. We even attempted to address the RAM concern by increasing its capacity, but that did not yield any improvement either. Since the option to suspend/unsuspend users is only available in the production environment, I cannot simply rewrite the code until it functions correctly. It's reminiscent of the previous issue we encountered with the Gmail API, which was somehow resolved several months ago by adding numerous synchronized keywords in certain places and rewriting other segments and updated google libraries. However, I am dissatisfied with the fact that the API client does not provide clear error messages, making it challenging to explain to clients why the Google integrations are not functioning as expected.

We utilize the GoogleAuthAdminSDKService in our Spring @RestController. While it may seem like a threading issue, I have thoroughly examined our codebase and verified that we are not sharing anything non-threadsafe in relation to the Google libraries. However, while working on other tasks involving Google APIs, I observed that the Gmail API, which is executed through @Scheduled tasks, frequently throws the error "Illegal argument: key: emails (or other)". It's important to note that we do not utilize @Scheduled tasks in our main product.

In our implementation, we retrieve the User using the Directory API. We make use of the setSuspended(true/false) method to modify the suspension status, and then we apply the patch using the same object. It's worth mentioning that we do not make any modifications to the "email" field during this process.

roma2341 commented 1 year ago

I also tried to enable logging:

@PostConstruct
    private void enableLogging() {
          Logger logger = Logger.getLogger(HttpTransport.class.getName());
          logger.setLevel(Level.ALL);
          logger.addHandler(new Handler() {

            @Override
            public void close() throws SecurityException {
            }

            @Override
            public void flush() {
            }

            @Override
            public void publish(LogRecord record) {
                log.info(record.getMessage());
            }
          });
        }

Just before encountering the error, we logged this JSON data (however, when attempting to parse it using com.google.api.client.json.JsonParser, everything appeared to be ok).":

`{
  "kind": "admin#directory#user",
  "id": "NOT_MATTER",
  "etag": "\"NOT_MATTER\"",
  "primaryEmail": "NOT_MATTER@NOT_MATTER",
  "name": {
    "givenName": "NOT_MATTER",
    "familyName": "NOT_MATTER NOT_MATTER",
    "fullName": "NOT_MATTER NOT_MATTER  NOT_MATTER"
  },
  "isAdmin": false,
  "isDelegatedAdmin": false,
  "lastLoginTime": "2023-04-18T16:25:34.000Z",
  "creationTime": "2022-10-24T02:33:56.000Z",
  "agreedToTerms": true,
  "suspended": true,
  "suspensionReason": "ADMIN",
  "archived": false,
  "changePasswordAtNextLogin": false,
  "ipWhitelisted": false,
  "emails": [
    {
      "address": "NOT_MATTER@NOT_MATTER.com",
      "primary": true
    }
  ],
  "languages": [
    {
      "languageCode": "en",
      "preference": "preferred"
    }
  ],
  "customerId": "NOT_MATTER",
  "orgUnitPath": "/NOT_MATTER",
  "isMailboxSetup": true,
  "isEnrolledIn2Sv": false,
  "isEnforcedIn2Sv": false,
  "includeInGlobalAddressList": true
}`

As i see from stacktrace it fails on this line:

var gUser = service.users().get(userEmail).execute();
diegomarquezp commented 1 year ago

Thanks for bringing up the other instances of this error. I take back my suspicion on the authentication logic.

It seems like the Json parser gets values of unexpected types. For example emails is expected to be an array of objects as the one you posted in your last response, but the error in the first comment is that it expected collection or array type but got class java.lang.Object.

While I can't imagine why all your clients would start failing similarly with unexpected value types, I can see that the error message itself is not helpful at all (e.g. from the other issue Caused by: java.lang.IllegalArgumentException: expected collection or array type but got java.util.List<com.google.api.client.googleapis.json.GoogleJsonError$ErrorInfo>) (edit: note that we have JsonErrors that should be explained in the exception instead of just saying they are unexpected)

The error occurs here: https://github.com/googleapis/google-http-java-client/blob/223dfef05114bd64acaba5424ee6d6c44f9223b9/google-http-client/src/main/java/com/google/api/client/json/JsonParser.java#L887-L901

Which is caused by an argument check to confirm that emails (in this issue) is actually an array https://github.com/googleapis/google-http-java-client/blob/223dfef05114bd64acaba5424ee6d6c44f9223b9/google-http-client/src/main/java/com/google/api/client/json/JsonParser.java#L724-L729

cc: @blakeli0 - why would we using a custom JsonParser instead of something like gson in the first place?

roma2341 commented 1 year ago

I will attempt to update all Google libraries as I suspect that the issue may be caused by conflicts between different versions of the libraries or dependencies. I will let you know in a couple of days whether it helped, but if it is indeed due to library versions, here are all the versions of libraries we are using now (I will update the libraries and recheck to see if the bug is resolved. However, I have doubts that the versions are the problem because only the admin-directory doesn't work sometime):

crm-service dependencies

4.0.0 com.shs microservice-project 0.4 ../../ crm-service jar 0.5.1 crm-service experior-crm com.shs common 0.4 com.shs crm-client 0.4 com.shs tinyurl-client 1.0-SNAPSHOT com.shs zoom-api-client 1.0-SNAPSHOT com.shs ipipeline-client 1.0.0 com.shs clidies-parser-service 0.4 compile org.codehaus.mojo jaxb2-maven-plugin org.springframework.boot spring-boot-starter-data-jpa org.freemarker freemarker no.api.freemarker freemarker-java8 2.1.0 org.springframework.boot spring-boot-starter-freemarker org.springframework.boot spring-boot-starter-integration org.springframework.boot spring-boot-starter org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-tomcat org.springframework.boot spring-boot-starter-undertow org.flywaydb flyway-core mysql mysql-connector-java runtime org.springframework.boot spring-boot-starter-test test io.projectreactor reactor-test test org.springframework.boot spring-boot-starter-aop com.auth0 java-jwt 4.0.0 commons-io commons-io 2.11.0 org.springframework.boot spring-boot-starter-security org.springframework.security spring-security-oauth2-client org.springframework.boot spring-boot-starter-actuator org.springframework.security spring-security-test test org.modelmapper modelmapper 3.1.0 org.springframework.boot spring-boot-starter-mail org.springframework.data spring-data-envers org.springframework.boot spring-boot-starter-amqp org.hibernate hibernate-jpamodelgen org.hibernate hibernate-envers org.springdoc springdoc-openapi-ui 1.6.11 org.springdoc springdoc-openapi-data-rest 1.6.11 org.telegram telegrambots 6.1.0 com.github.javafaker javafaker 1.0.2 com.twelvemonkeys.imageio imageio-core 3.8.3 org.apache.poi poi 5.2.3 org.jsoup jsoup 1.15.3 org.apache.poi poi-ooxml 5.2.3 org.jodconverter jodconverter-local 4.4.4 org.springframework.boot spring-boot-starter-oauth2-client org.springframework.boot spring-boot-starter-validation org.mnode.ical4j ical4j 3.2.5 org.apache.commons commons-text 1.10.0 com.google.api-client google-api-client 2.2.0 com.google.apis google-api-services-calendar v3-rev20230317-2.0.0 com.google.apis google-api-services-gmail v1-rev20230206-2.0.0 com.google.apis google-api-services-tasks v1-rev20210709-2.0.0 com.google.apis google-api-services-admin-directory directory_v1-rev20230316-2.0.0 com.google.firebase firebase-admin 8.2.0 com.google.apis google-api-services-fcm v1-rev20221003-2.0.0 com.google.maps google-maps-services 2.1.0 com.google.cloud google-cloud-texttospeech 2.3.0 org.springframework spring-webmvc org.reflections reflections 0.10.2 com.googlecode.libphonenumber libphonenumber 8.12.47 com.googlecode.libphonenumber geocoder 2.183 net.iakovlev timeshape 2021c.13 org.quartz-scheduler quartz org.quartz-scheduler quartz-jobs com.cronutils cron-utils 9.1.7 org.springframework spring-context-support com.twilio.sdk twilio 8.29.1 com.wildbit.java postmark 1.8.1 org.slf4j slf4j-log4j12 org.apache.tika tika-core 2.4.0 org.apache.pdfbox pdfbox 2.0.26 org.apache.pdfbox jbig2-imageio 3.0.4 com.github.jai-imageio jai-imageio-core 1.4.0 com.twelvemonkeys.imageio imageio-jpeg 3.8.3 com.github.jai-imageio jai-imageio-jpeg2000 1.4.0 org.springframework spring-test org.apache.commons commons-lang3 org.springframework.integration spring-integration-ftp 5.5.11 org.springframework.integration spring-integration-sftp 5.5.11 com.opencsv opencsv 5.7.0 name.neuhalfen.projects.crypto.bouncycastle.openpgp bouncy-gpg 2.3.0 com.vladmihalcea hibernate-types-52 2.18.0 net.lingala.zip4j zip4j 2.11.2 com.onelogin java-saml 2.9.0 org.springframework.boot spring-boot-maven-plugin maven-war-plugin false org.flywaydb flyway-core 7.15.0 spring-snapshots Spring Snapshots https://repo.spring.io/snapshot true spring-milestones Spring Milestones https://repo.spring.io/milestone oss-snapshot-local OSS Snapshot local https://oss.jfrog.org/artifactory/oss-snapshot-local spring-snapshots Spring Snapshots https://repo.spring.io/snapshot true spring-milestones Spring Milestones https://repo.spring.io/milestone Codehaus Mojo Codehaus Mojo https://repo1.maven.org/maven2/org/codehaus/mojo/

Common dependencies

4.0.0 com.shs microservice-project 0.4 common jar common org.springframework.boot spring-boot-starter-data-jpa org.springframework.boot spring-boot-starter-integration org.springframework.boot spring-boot-starter org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-test test io.projectreactor reactor-test test org.springframework.boot spring-boot-starter-aop ch.qos.logback logback-core com.auth0 java-jwt 4.0.0 commons-io commons-io 2.11.0 com.google.guava guava 31.1-jre org.springframework.boot spring-boot-starter-security org.springframework.security spring-security-test test org.hibernate hibernate-jpamodelgen com.fasterxml.jackson.datatype jackson-datatype-hibernate5 org.hibernate hibernate-envers org.apache.poi poi 5.2.3 org.apache.poi poi-ooxml 5.2.3 org.springframework.boot spring-boot-starter-validation com.fasterxml.jackson.dataformat jackson-dataformat-xml org.apache.commons commons-text 1.10.0 org.springframework spring-webmvc org.reflections reflections 0.10.2 joda-time joda-time 2.11.2 org.jsoup jsoup 1.15.3 com.cronutils cron-utils 9.2.0 com.esotericsoftware kryo 5.3.0 commons-beanutils commons-beanutils 1.9.4 io.vavr vavr 0.10.4 com.github.gavlyukovskiy p6spy-spring-boot-starter 1.8.1 com.github.dpaukov combinatoricslib3 3.3.3 common-lib

roma2341 commented 1 year ago

after I updated the libraries the error remained.

Updated versions

google-api-services-admin-directory: directory_v1-rev20230516-2.0.0 firebase-admin: 9.2.0 google-api-services-tasks: v1-rev20230401-2.0.0 google-api-services-fcm: v1-rev20230609-2.0.0 google-cloud-texttospeech: 2.21.0

java.lang.IllegalArgumentExcep (114).txt

roma2341 commented 1 year ago

Google support replied that they do not support debugging and code implementation.

suztomo commented 11 months ago

@roma2341 We couldn't reproduce the issue so far. It's very likely that when the error happens, you get some strange response of an object as "emails". Is it possible to shorten the reproducing example, preferably just one main function?

zhumin8 commented 8 months ago

Closing as we couldn't reproduce this. Please reopen if this is still an issue and preferably you could produce a shorten repro as mentioned in above comment.