search

wso2 / api-manager

All issues, tasks, improvements and new features of WSO2 API Manager
Apache License 2.0
34 stars 8 forks source link

Tenant artifact synchronization with sync retrieval mode breaks the tenant context creation #1048

Open lakshithagunasekara opened 1 year ago

lakshithagunasekara commented 1 year ago

Description

When we specify the sync mode for data retrieval mode, tenant APIs deployment will break the tenant loading and will stuck in an infinite loop with tenant context creation. This does not happen in the default mode which is async.

Need to investigate it and fix.

Steps to Reproduce

  1. Change the data_retrieval_mode to sync with following config for the gateway.

    [apim.sync_runtime_artifacts.gateway]
gateway_labels =["Default"]
data_retrieval_mode = "sync"

  2. Create a tenant

  3. Create a tenant API

  4. Restart the server and load the tenant by login as the tenant user to publisher

Affected Component

APIM

Version

4.1.0

Environment Details (with versions)

No response

Relevant Log Output

TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,140]  INFO {org.wso2.carbon.identity.tenant.resource.manager.TenantAwareAxis2ConfigurationContextObserver} - Loading configuration context for tenant domain: bal.com
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,140]  WARN {org.wso2.carbon.identity.tenant.resource.manager.TenantAwareAxis2ConfigurationContextObserver} - Configuration store does not contain  publisher resource type. Super tenant configurations will be used for the tenant domain: bal.com
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,140]  INFO {org.wso2.carbon.core.multitenancy.TenantAxisConfigurator} - Creating tenant AxisConfiguration for tenant: bal.com[1]
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,155]  INFO {org.wso2.carbon.identity.user.store.configuration.deployer.UserStoreConfigurationDeployer} - User Store Configuration Deployer initiated.
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,155]  INFO {org.wso2.carbon.identity.user.store.configuration.deployer.UserStoreConfigurationDeployer} - User Store Configuration Deployer initiated.
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,918]  INFO {org.wso2.carbon.core.internal.permission.update.PermissionUpdater} - Permission cache updated for tenant 1
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,925]  INFO {org.wso2.carbon.apimgt.gateway.internal.TenantServiceCreator} - Initializing APIM TenantServiceCreator for the tenant domain : bal.com
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,927]  INFO {org.wso2.carbon.mediation.initializer.multitenancy.TenantServiceBusInitializer} - Initializing the ESB Configuration for the tenant domain : bal.com
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,928]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : main was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : fault was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.SynapseControllerFactory} - Using Synapse home : /Users/lakshithagunasekara/Projects/JFR/Setup/tenantInfinite/wso2am-4.1.0/repository/tenants/1
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.SynapseControllerFactory} - Using synapse.xml location : /Users/lakshithagunasekara/Projects/JFR/Setup/tenantInfinite/wso2am-4.1.0/repository/tenants/1/synapse-configs/default
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.SynapseControllerFactory} - Using server name : synapse.bal.com
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.SynapseControllerFactory} - The timeout handler will run every : 15s
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.Axis2SynapseController} - Initializing Synapse at : Mon Nov 28 14:46:22 IST 2022
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.wso2.carbon.mediation.initializer.CarbonSynapseController} - Loading the mediation configuration from the file system
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.config.xml.MultiXMLConfigurationBuilder} - Building synapse configuration from the synapse artifact repository at : /Users/lakshithagunasekara/Projects/JFR/Setup/tenantInfinite/wso2am-4.1.0/repository/tenants/1/synapse-configs/default
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,929]  INFO {org.apache.synapse.config.xml.XMLConfigurationBuilder} - Generating the Synapse configuration model by parsing the XML configuration
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,930]  INFO {org.wso2.carbon.mediation.ntask.NTaskTaskManager} - Initialized task manager. Tenant [1]
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,931]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : fault was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,932]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : main was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,946]  WARN {org.wso2.carbon.apimgt.impl.factory.KeyManagerHolder} - Key Manager Resident Key Manager already initialized in tenant bal.com
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,948]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : main was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,952]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _cors_request_handler_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,960]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _throttle_out_handler_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,961]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : dispatchSeq was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,962]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : outDispatchSeq was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,967]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _threat_fault_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,968]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _production_key_error_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,969]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _resource_mismatch_handler_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,986]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : fault was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:22,994]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _opa_policy_failure_handler_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,001]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _backend_failure_handler_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,006]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : webhooksFaultSequence was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,010]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _graphql_failure_handler was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,012]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _api_blocked_sequence_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,020]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _auth_failure_handler_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Sequence : _sandbox_key_error_ was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Inbound Endpoint : SecureWebhookServer was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Inbound Endpoint : WebhookServer was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Inbound Endpoint : SecureWebSocketEP was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Inbound Endpoint : WebSocketInboundEndpoint was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.apache.synapse.config.SynapseConfigurationBuilder} - Loaded Synapse configuration from the artifact repository at : /Users/lakshithagunasekara/Projects/JFR/Setup/tenantInfinite/wso2am-4.1.0/repository/tenants/1/synapse-configs/default
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Local entry : SERVER_HOST was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,022]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} - Local entry : SERVER_IP was added to the Synapse configuration successfully
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.Axis2SynapseController} - Loading mediator extensions...
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.Axis2SynapseController} - Deploying the Synapse service...
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.Axis2SynapseController} - Deploying Proxy services...
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.Axis2SynapseController} - Deploying EventSources...
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Destroying Inbound Endpoint: WebSocketInboundEndpoint
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Destroying Inbound Endpoint: SecureWebSocketEP
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Destroying Inbound Endpoint: WebhookServer
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Destroying Inbound Endpoint: SecureWebhookServer
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,023]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Initializing Inbound Endpoint: WebSocketInboundEndpoint
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.wso2.carbon.inbound.endpoint.protocol.websocket.management.WebsocketEndpointManager} - Netty Listener already started on port 9099
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Initializing Inbound Endpoint: SecureWebSocketEP
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.wso2.carbon.inbound.endpoint.protocol.websocket.management.WebsocketEndpointManager} - Netty Listener already started on port 8099
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Initializing Inbound Endpoint: WebhookServer
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.wso2.carbon.inbound.endpoint.protocol.http.management.HTTPEndpointManager} - Listener is already started for port : 9021
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.apache.synapse.inbound.InboundEndpoint} - Initializing Inbound Endpoint: SecureWebhookServer
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.wso2.carbon.inbound.endpoint.protocol.http.management.HTTPEndpointManager} - Listener is already started for port : 8021
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,024]  INFO {org.apache.synapse.ServerManager} - Server ready for processing...
TID: [-1234] [internal/data/v1] [2022-11-28 14:46:23,059]  INFO {org.wso2.carbon.apimgt.impl.importexport.utils.CommonUtil} - Creation of folder is successful. Directory Name : PizzaShackAPI-1.0.0
TID: [-1234] [internal/data/v1] [2022-11-28 14:46:23,059]  INFO {org.wso2.carbon.apimgt.impl.importexport.utils.CommonUtil} - Creation of folder is successful. Directory Name : Definitions
TID: [-1234] [api/am/publisher] [2022-11-28 14:46:23,095]  INFO {org.wso2.carbon.apimgt.gateway.InMemoryAPIDeployer} - Deploying synapse artifacts of PizzaShackAPI

Related Issues

No response

Suggested Labels

No response

BLasan commented 1 year ago

Hi,

After rebasing the latest code base of the product-apim, there were failures when loading the publisher portal along with some errors printed in the logs. This was reproduced before rebasing the latest one and there were some errors getting printed on the console when logging in to the publisher portal via tenant credentials.

Thanks, Benura

BLasan commented 1 year ago

Hi,

When trying to log in to the publisher portal via tenant credentials, the following error will show up.


[2023-01-25 10:37:26,927] ERROR - SystemScopesIssuer Error while getting scopes of application RESTAPIScopes config does not exist for tenant test.com
org.wso2.carbon.apimgt.api.APIManagementException: RESTAPIScopes config does not exist for tenant test.com
    at org.wso2.carbon.apimgt.impl.utils.SystemScopeUtils.getTenantRESTAPIScopesConfig_aroundBody16(SystemScopeUtils.java:237) ~[org.wso2.carbon.apimgt.impl_9.28.25.jar:?]
    at org.wso2.carbon.apimgt.impl.utils.SystemScopeUtils.getTenantRESTAPIScopesConfig(SystemScopeUtils.java:1) ~[org.wso2.carbon.apimgt.impl_9.28.25.jar:?]
    at org.wso2.carbon.apimgt.impl.utils.SystemScopeUtils.getRESTAPIScopesForTenant_aroundBody12(SystemScopeUtils.java:173) ~[org.wso2.carbon.apimgt.impl_9.28.25.jar:?]
    at org.wso2.carbon.apimgt.impl.utils.SystemScopeUtils.getRESTAPIScopesForTenant(SystemScopeUtils.java:1) ~[org.wso2.carbon.apimgt.impl_9.28.25.jar:?]
    at org.wso2.carbon.apimgt.impl.issuers.SystemScopesIssuer.getAppScopes_aroundBody38(SystemScopesIssuer.java:712) ~[org.wso2.carbon.apimgt.impl_9.28.25.jar:?]
    at org.wso2.carbon.apimgt.impl.issuers.SystemScopesIssuer.getAppScopes(SystemScopesIssuer.java:1) ~[org.wso2.carbon.apimgt.impl_9.28.25.jar:?]

Is this an expected behavior? However, we can login to the publisher portal and create an API with this error log in the console.

Thanks, Benura

Tharsanan1 commented 1 year ago

We will deprecate the data_retrieval_mode = "sync" configuration and remove it from the next release.

YasasRangika commented 6 months ago

Need to verify if https://github.com/wso2/api-manager/issues/1048#issuecomment-1411418439 is applicable in the APIM-4.3.0 release.