Gravitee APIM gateway 3.17.1.0 not accepting API keys

[hamartin]

1. 1. :collision: Describe the bug After upgrading GraviteeIO APIM from 3.13.x to 3.16.x, our single host Gravitee system started to always respond with 401, API Key is not valid or is expired / revoked on all APIs on the system.

After wiping the host and reinstalling the system from scratch with GraviteeIO APIM 3.17.1.0, the error still occurs on the 1 test API created for testing if the problem still exist on the system.

1. 1. :sunrise_over_mountains: To Reproduce Install packages mentioned further down and start the service.
2. 1. :rainbow: Expected behaviour The gateway should return the output coming from the endpoint.
3. 1. Current behaviour The gateway always responds with

 [root@<hostname>
 logs]# curl -XGET https://
 <fqdn>
 :8082/ 
 <context path>
 / -H "X-Gravitee-Api-Key: 
 <api key found in portal after request is accepted>
 " 

 {"message":"API Key is not valid or is expired / revoked.","http_status_code":401} 

 ``` ## :movie_camera: Useful information ``` bash [root@ 
 <hostname>
 logs]# rpm -qa | egrep -i "elasticsearch|mongo|gravitee|jdk" copy-jdk-configs-3.3-10.el7_5.noarch elasticsearch-oss-7.10.2-1.x86_64 graviteeio-apim-rest-api-3x-3.17.1-0.noarch graviteeio-apim-management-ui-3x-3.17.1-0.noarch mongodb-org-mongos-3.6.23-1.el7.x86_64 mongodb-org-server-3.6.23-1.el7.x86_64 java-11-openjdk-headless-11.0.15.0.9-2.0.1.el7_9.x86_64 graviteeio-apim-gateway-3x-3.17.1-0.noarch graviteeio-apim-portal-ui-3x-3.17.1-0.noarch mongodb-org-tools-3.6.23-1.el7.x86_64 mongodb-org-3.6.23-1.el7.x86_64 java-11-openjdk-11.0.15.0.9-2.0.1.el7_9.x86_64 java-11-openjdk-devel-11.0.15.0.9-2.0.1.el7_9.x86_64 mongodb-org-shell-3.6.23-1.el7.x86_64 [root@ 
 <hostname>
 logs]# ``` I have also tried wiping the Mongo data in /var/lib/mongo and installing Mongo 4.4, with no change in behaviour. Before wiping the system, the host was running the non OSS Elasticsearch version, again with the same behaviour. When looking through the logs in systemd, the log files for the Gravitee services, mongo and Elasticsearch pluss the SELinux audit logs. The only noticeable error/warning message from my point of view is the section mentioning the ApiKeyEnumMode which I do not remember having seen before the update. ``` verbatim 09:26:36.764 <span class="error">[gravitee-listener-81]</span> WARN o.h.v.i.p.j.JavaBeanExecutable - HV000254: Missing parameter metadata for ApiKeyModeEnum(String, int, String), which declares implicit or synthetic parameters. Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag. ``` ## Screenshot, video, logs, other supporting material /opt/graviteeio/apim/rest-api/logs/gravitee.log ``` bash [root@ 
 <hostname>
 logs]# pwd /opt/graviteeio/apim/rest-api/logs [root@ 
 <hostname>
 logs]# tail gravitee.log 09:23:13.312 <span class="error">[graviteeio-node]</span> INFO i.g.p.a.i.AlertTriggerProviderManagerImpl - There is no trigger provider to start 09:23:13.312 <span class="error">[graviteeio-node]</span> INFO i.g.r.a.s.node.GraviteeApisNode - Starting component: AlertEventProducerManager 09:23:13.312 <span class="error">[graviteeio-node]</span> INFO i.g.common.service.AbstractService - Initializing service Alert Engine - Event producer 09:23:13.312 <span class="error">[graviteeio-node]</span> INFO i.g.p.a.i.AlertEventProducerManagerImpl - There is no event producer to start 09:23:13.318 <span class="error">[graviteeio-node]</span> INFO i.g.r.a.s.node.GraviteeApisNode - Gravitee.io - Rest APIs id<span class="error">[89ecf386-2c1f-40c9-acf3-862c1f20c90d]</span> version<span class="error">[3.17.1]</span> pid<span class="error">[18562]</span> build[68973#34d1d7a0b5](#34d1d7a0b5) jvm<span class="error">[Red Hat, Inc./OpenJDK 64-Bit Server VM/11.0.15+9-LTS]</span> started in 6569 ms. 09:23:22.326 <span class="error">[gravitee-listener-48]</span> INFO i.g.c.event.impl.EventManagerImpl - Register new listener GraviteeCorsConfiguration for event type class io.gravitee.rest.api.model.parameters.Key 09:25:58.092 <span class="error">[gravitee-listener-49]</span> INFO i.g.c.event.impl.EventManagerImpl - Register new listener GraviteeCorsConfiguration for event type class io.gravitee.rest.api.model.parameters.Key 09:26:36.764 <span class="error">[gravitee-listener-81]</span> WARN o.h.v.i.p.j.JavaBeanExecutable - HV000254: Missing parameter metadata for ApiKeyModeEnum(String, int, String), which declares implicit or synthetic parameters. Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag. 09:30:00.007 <span class="error">[sync-1]</span> INFO i.g.r.api.services.sync.ApiManager - Deployment of ApiEntity

 {id='91c4e6a9-87a3-4f1e-84e6-a987a39f1ea1', crossId='null', name='test', version='0.1', description='Returns static dummy data.', proxy=io.gravitee.definition.model.Proxy@f857ac3, paths=null, createdAt=Tue Apr 26 09:29:58 CEST 2022, updatedAt=Tue Apr 26 09:29:59 CEST 2022, visibility=PRIVATE, state=STARTED, primaryOwner=io.gravitee.rest.api.model.PrimaryOwnerEntity@5e8171c2, tags=[], category=null, groups=null, pathMappings=[], lifecycleState=CREATED, workflowState=null, disableMembershipNotifications=false, graviteeDefinitionVersion=null, flowMode=null} 

 09:30:00.041 <span class="error">[sync-1]</span> INFO i.g.r.a.s.d.DynamicPropertiesService - Dynamic properties service is disabled for: test <span class="error">[0.1]</span> ``` /opt/graviteeio/apim/gateway/logs/gravitee.log ``` bash [root@ 
 <hostname>
 logs]# pwd /opt/graviteeio/apim/gateway/logs [root@ 
 <hostname>
 logs]# tail gravitee.log 09:24:22.008 <span class="error">[vert.x-worker-thread-1]</span> [] INFO i.g.common.service.AbstractService - Initializing service io.gravitee.reporter.elasticsearch.ElasticsearchReporter 09:24:22.008 <span class="error">[vert.x-worker-thread-1]</span> [] INFO i.g.r.e.ElasticsearchReporter - Starting Elastic reporter engine... 09:24:22.373 <span class="error">[vert.x-worker-thread-1]</span> [] INFO i.g.r.file.vertx.VertxFileWriter - Initializing file reporter to write into file: /opt/graviteeio/apim/graviteeio-apim-gateway-3.17.1/metrics/health-check-2022_04_26.json 09:24:22.374 <span class="error">[vert.x-worker-thread-1]</span> [] INFO i.g.r.file.vertx.VertxFileWriter - Initializing file reporter to write into file: /opt/graviteeio/apim/graviteeio-apim-gateway-3.17.1/metrics/request-2022_04_26.json 09:24:22.375 <span class="error">[vert.x-worker-thread-1]</span> [] INFO i.g.r.file.vertx.VertxFileWriter - Initializing file reporter to write into file: /opt/graviteeio/apim/graviteeio-apim-gateway-3.17.1/metrics/log-2022_04_26.json 09:24:22.375 <span class="error">[vert.x-worker-thread-1]</span> [] INFO i.g.r.file.vertx.VertxFileWriter - Initializing file reporter to write into file: /opt/graviteeio/apim/graviteeio-apim-gateway-3.17.1/metrics/node-2022_04_26.json 09:24:22.384 <span class="error">[vert.x-eventloop-thread-0]</span> [] INFO i.g.reporter.file.FileReporter - File reporter successfully started 09:24:22.415 <span class="error">[vert.x-eventloop-thread-0]</span> [] INFO i.g.r.e.ElasticsearchReporter - Starting Elastic reporter engine... DONE 09:24:22.415 <span class="error">[vert.x-eventloop-thread-0]</span> [] INFO i.g.r.e.ElasticsearchReporter - Index mapping template successfully defined 09:30:01.997 <span class="error">[gio.sync-6]</span> <span class="error">[91c4e6a9-87a3-4f1e-84e6-a987a39f1ea1]</span> INFO i.g.g.h.a.m.impl.ApiManagerImpl - API id<span class="error">[91c4e6a9-87a3-4f1e-84e6-a987a39f1ea1]</span> name<span class="error">[test]</span> version<span class="error">[0.1]</span> has been deployed <span class="error">[root@apitest01 logs]</span># ``` ## :computer: Desktop: - Environment: APIM 3.17.1.0 - OS: Oracle Linux 7.9 - Browser: Not relevant ## :warning: Potential impacts ****Which other features may be impacted by this fix. This could be populated after fix**** ****What are the impacted versions?**** Version >= 3.16.x? ## :link: Dependencies Link a story or other related things... 
 </hostname> 
 </hostname> 
 </hostname> 
 </hostname> 
 </hostname> 
 </hostname> 
 </api> 
 </context> 
 </fqdn>
 </hostname>

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[masterbox]

Any news on this ? We upgraded to 3.18.0 one of our gateway and get the same behavior.

[mouligno]

Can you check with latest versions ? any stack trace/warning during installation ?

[gaetanmaisse]

I'm closing the issue as we weren't able to reproduce the issue on our side and we did get feedback for a couple of months. Feels free to reopen it if needed.