search

wso2 / api-manager

All issues, tasks, improvements and new features of WSO2 API Manager
Apache License 2.0
34 stars 8 forks source link

API-M 4.4.0 Migration Effort #3139

Open tharikaGitHub opened 2 weeks ago

tharikaGitHub commented 2 weeks ago

Description

This issue is created to track the tasks for APIM 4.4.0 migration.

Version combinations to test :

  1. APIM 4.2.0 - APIM 4.4.0
  2. APIM 4.3.0 - APIM 4.4.0

Affected Component

APIM

Version

4.4.0

Related Issues

No response

Suggested Labels

No response

tharikaGitHub commented 1 week ago

Update

  1. An APIM 4.2.0 setup was done with the data population.
  2. Created the MYSQL script for direct migration from APIM 4.2.0 to APIM 4.4.0.
  3. Tested the APIM 4.4.0 setup after the migration. This was tested in an APIM 4.4.0-alpha pack.
  4. The following issues were encountered.

Issue 1 : Error during startup

The following exception was thrown during startup when trying to load an API Product. The issue happens after the fix https://github.com/wso2/carbon-apimgt/pull/12416. This is the same for APIM 4.3.0 as well. This is because the Product versioning concept was introduced from APIM 4.3.0 onwards and due to that, the formation of the product context has changed. Previously the context did not have the version appended. But now it has.

[2024-10-03 18:18:22,858] ERROR - Framework FrameworkEvent ERROR
java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 15
    at java.lang.String.checkBoundsBeginEnd(String.java:3319) ~[?:?]
    at java.lang.String.substring(String.java:1874) ~[?:?]
    at org.wso2.carbon.apimgt.gateway.internal.DataHolder.addAPIMetaData_aroundBody46(DataHolder.java:199) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.internal.DataHolder.addAPIMetaData(DataHolder.java:1) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.listeners.GatewayStartupListener.lambda$1_aroundBody48(GatewayStartupListener.java:236) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.listeners.GatewayStartupListener.lambda$1(GatewayStartupListener.java:1) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?]
    at org.wso2.carbon.apimgt.gateway.listeners.GatewayStartupListener.retrieveAllAPIMetadata_aroundBody10(GatewayStartupListener.java:236) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.listeners.GatewayStartupListener.retrieveAllAPIMetadata(GatewayStartupListener.java:1) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.listeners.GatewayStartupListener.completedServerStartup_aroundBody8(GatewayStartupListener.java:196) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.listeners.GatewayStartupListener.completedServerStartup(GatewayStartupListener.java:1) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.notifyAfter(CarbonCoreServiceComponent.java:264) ~[?:?]
    at org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.completeInitialization(StartupFinalizerServiceComponent.java:214) ~[?:?]
    at org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.serviceChanged(StartupFinalizerServiceComponent.java:309) ~[?:?]

Issue 2 : Error when loading the shared scopes page in Publisher

The following error was encountered when loading the shared scopes page in Publisher portal when there are shared scopes used by one or several APIs and API Products. The issue is a result of the fix https://github.com/wso2/carbon-apimgt/pull/12452. API Product url mappings were not considered.

java.lang.NullPointerException: null
    at org.wso2.carbon.apimgt.impl.dao.ApiMgtDAO.getSharedScopeUsage_aroundBody830(ApiMgtDAO.java:17125) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.impl.dao.ApiMgtDAO.getSharedScopeUsage(ApiMgtDAO.java:1) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.impl.APIProviderImpl.getSharedScopeUsage_aroundBody394(APIProviderImpl.java:5172) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.impl.APIProviderImpl.getSharedScopeUsage(APIProviderImpl.java:1) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.impl.UserAwareAPIProvider.getSharedScopeUsage(UserAwareAPIProvider.java:1) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.impl.ScopesApiServiceImpl.getSharedScopeUsages(ScopesApiServiceImpl.java:184) ~[?:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.ScopesApi.getSharedScopeUsages(ScopesApi.java:106) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]

Issue 3 : No APIs listed when creating an API Product

When trying to create a new API Product or when trying to edit resources of an API Product, the UI does not list any API. The message says No REST APIs are created yet. The reason is, for migrated APIs, the gatewayType and gatewayVendor fields are null when retrieving all APIs to the UI. We check the gatewayType when displaying the APIs for API Product resource editing UI. Since gatewayType is null in the search result set, no APIs are displayed.

Issue 4 : Error when loading an API which has maxTps defined.

In Publisher, the following exception is thrown when loading an API which has maxTps values defined in Runtime page.

[2024-10-04 14:50:40,688] ERROR - GlobalThrowableMapper An unknown exception has been captured by the global exception mapper.
java.lang.NullPointerException: null
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.common.mappings.APIMappingUtil.setMaxTpsFromModelToApiDTO(APIMappingUtil.java:1676) ~[org.wso2.carbon.apimgt.rest.api.publisher.v1.common_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.common.mappings.APIMappingUtil.fromAPItoDTO(APIMappingUtil.java:1447) ~[org.wso2.carbon.apimgt.rest.api.publisher.v1.common_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.common.mappings.APIMappingUtil.fromAPItoDTO(APIMappingUtil.java:1059) ~[org.wso2.carbon.apimgt.rest.api.publisher.v1.common_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.impl.ApisApiServiceImpl.getAPIByID(ApisApiServiceImpl.java:3701) ~[?:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.impl.ApisApiServiceImpl.getAPI(ApisApiServiceImpl.java:244) ~[?:?]
    at org.wso2.carbon.apimgt.rest.api.publisher.v1.ApisApi.getAPI(ApisApi.java:678) ~[?:?]

Issue 5 : Error when trying to invoke an API

The following error is encountered when trying to invoke an API for both tenant and super tenant. The reason is the change in the Product versioning concept.

[2024-10-04 20:40:33,944] ERROR - ServerWorker Error processing GET request for : /t/adpsample.com/pizzashack/1.0.0/menu.
java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 15
    at java.lang.String.checkBoundsBeginEnd(String.java:3319) ~[?:?]
    at java.lang.String.substring(String.java:1874) ~[?:?]
    at org.wso2.carbon.apimgt.keymgt.model.impl.SubscriptionDataStoreImpl.getAllAPIsByContextList_aroundBody92(SubscriptionDataStoreImpl.java:779) ~[org.wso2.carbon.apimgt.keymgt_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.keymgt.model.impl.SubscriptionDataStoreImpl.getAllAPIsByContextList(SubscriptionDataStoreImpl.java:1) ~[org.wso2.carbon.apimgt.keymgt_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.handlers.Utils.getSelectedAPIList_aroundBody58(Utils.java:700) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.handlers.Utils.getSelectedAPIList(Utils.java:1) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.handlers.DefaultAPIHandler.handleRequestInFlow_aroundBody0(DefaultAPIHandler.java:77) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.gateway.handlers.DefaultAPIHandler.handleRequestInFlow(DefaultAPIHandler.java:1) ~[org.wso2.carbon.apimgt.gateway_9.30.10.jar:?]
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.invokeHandlers(Axis2SynapseEnvironment.java:1202) ~[synapse-core_4.0.0.wso2v128.jar:4.0.0-wso2v128]

Issue 6 : Error when loading the Scope Assignments page in Admin portal

The following warnings are thrown when loading the Scope Assignments page in the Admin portal. This is due to the missing scopes coming from new features introduced in APIM 4.3.0 and APIM 4.4.0. This behaviour is the same for APIM 4.3.0 migrated setup as well.

[2024-10-04 20:34:19,154]  WARN - SystemScopesMappingUtil The scope apim:gateway_policy_manage does not exist in tenant.conf
[2024-10-04 20:34:19,154]  WARN - SystemScopesMappingUtil The scope apim:gateway_policy_view does not exist in tenant.conf
[2024-10-04 20:34:19,155]  WARN - SystemScopesMappingUtil The scope apim:api_provider_change does not exist in tenant.conf
[2024-10-04 20:34:19,155]  WARN - SystemScopesMappingUtil The scope apim:llm_provider_manage does not exist in tenant.conf
[2024-10-04 20:34:19,155]  WARN - SystemScopesMappingUtil The scope apim:llm_provider_read does not exist in tenant.conf

Issue 7 : Error when loading the Change API Provider page in the Admin portal

The following error is thrown when loading the Change API Provider page in Admin portal. The reason is the artifacts become null for documents. This behaviour is the same for a non migrated default APIM 4.4.0 pack as well when an API has a document attached. The same issue persists for APIM 4.3.0.

[2024-10-04 20:32:40,009] ERROR - GlobalThrowableMapper An unknown exception has been captured by the global exception mapper.
java.lang.NullPointerException: null
    at org.wso2.carbon.apimgt.persistence.RegistryPersistenceImpl.searchContentForAdmin(RegistryPersistenceImpl.java:3971) ~[org.wso2.carbon.apimgt.persistence_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.impl.APIAdminImpl.searchPaginatedApis_aroundBody154(APIAdminImpl.java:1528) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.impl.APIAdminImpl.searchPaginatedApis(APIAdminImpl.java:1) ~[org.wso2.carbon.apimgt.impl_9.30.10.jar:?]
    at org.wso2.carbon.apimgt.rest.api.admin.v1.impl.ApisApiServiceImpl.getAllAPIs(ApisApiServiceImpl.java:69) ~[?:?]
    at org.wso2.carbon.apimgt.rest.api.admin.v1.ApisApi.getAllAPIs(ApisApi.java:54) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]

Thanks, Tharika.

tharikaGitHub commented 1 week ago

Update

The fixes and comments regarding each of the above errors are as follows.

Main fix PR : https://github.com/wso2/carbon-apimgt/pull/12635

Issue 1 : Error during startup

This issue was fixed with the code https://github.com/wso2/carbon-apimgt/pull/12635/files#diff-2b416597458d994d9b8a486b337595f4b0a5381b3e76d55ed712642a1eddd94dR198-R202

Issue 2 : Error when loading the shared scopes page in Publisher

This issue was fixed with the code https://github.com/wso2/carbon-apimgt/pull/12635/files#diff-ac8fc1b7b2d10089084f9f842ad03ac096b64f8c555798aa18a926e0ec663a4cR17149-R17155

Issue 3 : No APIs listed when creating an API Product

This issue was fixed with the code https://github.com/wso2/carbon-apimgt/pull/12635/files#diff-f366598ae991e829fe8295cfee12176f8a8fa05a0644057f687491a411216935R177

Issue 4 : Error when loading an API which has maxTps defined

This issue was fixed with https://github.com/wso2/carbon-apimgt/pull/12635#issuecomment-2394929573

Issue 5 : Error when trying to invoke an API

This issue was fixed with the code https://github.com/wso2/carbon-apimgt/pull/12635/files#diff-47915adaff738f6d7b3bccea8d0456399f7ef09e62e7522543d5923246dcf363R778-R783

Issue 6 : Error when loading the Scope Assignments page in Admin portal

To fix this issue we have to manually add the scope mappings to the tenant-con from Admin Portal. This has to be done for all tenants. However since this warning is harmless and if required, the scopes can be added manually based on the usage of the new feature, no further action is required.

But we can look into some on the fly tenant-conf changes if needed.

Issue 7 : Error when loading the Change API Provider page in the Admin portal

This issue is fixed from the code https://github.com/wso2/carbon-apimgt/pull/12635/files#diff-e193f41af36b63c9f226e35c981eeae30ee45850c880876c502785b547dbe00cR3971-R3974. Although the initial loading issue is fixed, there is another issue that happens for a migrated setup.

No APIs are listed after loading the Change API Provider page. Even if a new API is created, no APIs are listed here. This issue does not occur in a normal non migrated APIM 4.4.0 setup. When searching for content for Admin, only the previously migrated document resources are returned from the persistence layer. No new resource is returned.

I have attempted reindexing as well, but the issue persists. This is the same for APIM 4.3.0.

This issue needs to be fixed.

Thanks, Tharika.

Avishka-Shamendra commented 1 week ago

Postgres

Hi all,

Environment Details

Testing

01 - Error when loading the Change API Provider page in the Admin portal

The same error observed with MySQL migration was observed with Postgres as well

Thank you Avishka

Avishka-Shamendra commented 1 week ago

DB2

Hi All,

I have attempted to migrate a 4.2.0.116-4.4.0 pack using DB2. However, following error was observed while deploying Sample PizzaShack API on 4.2.0.

[2024-10-09 13:48:26,993] ERROR - GatewayArtifactsMgtDAO Failed to Add Artifact to Database
com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-798, SQLSTATE=428C9, SQLERRMC=TIME_STAMP, DRIVER=4.27.25
    at com.ibm.db2.jcc.am.b7.a(b7.java:810) ~[db2jcc4.jar:?]
    at com.ibm.db2.jcc.am.b7.a(b7.java:66) ~[db2jcc4.jar:?]

Observed the following diff between apimgt/db2.sql the 4.2.0 latest and 4.3.0 latest. This is the cause of above error log

CREATE TABLE AM_GW_API_ARTIFACTS (
  API_ID varchar(255) NOT NULL,
  REVISION_ID varchar(255) NOT NULL,
  ARTIFACT blob,
-  TIME_STAMP TIMESTAMP NOT NULL GENERATED ALWAYS FOR EACH ROW ON UPDATE AS ROW CHANGE TIMESTAMP,
+  TIME_STAMP TIMESTAMP NOT NULL GENERATED BY DEFAULT FOR EACH ROW ON UPDATE AS ROW CHANGE TIMESTAMP,
  PRIMARY KEY (REVISION_ID, API_ID),
  FOREIGN KEY (API_ID) REFERENCES AM_GW_PUBLISHED_API_DETAILS (API_ID) ON DELETE NO ACTION 
-  ON UPDATE RESTRICT
+  ON UPDATE NO ACTION
);

Will proceed with the migration after correcting the 4.2.0 DB2 script

Thank you Avishka

Avishka-Shamendra commented 6 days ago

MSSQL

Hi all,

Environment Details

Testing

Issues

01 - Error when loading the Change API Provider page in the Admin portal

The same error observed with MySQL migration was observed with Postgres as well

Thank you Avishka

tharikaGitHub commented 5 days ago

Update

This is an update regarding the follow up issue under Issue 7 reported in comment [1] and is reproduced in every migration DB testing above.

This is an issue in the Change API Provider feature when multiple (more than 10) documents, and API Products are available in the system. This is not related to migration. I have created a public issue[2] to track it.

[1] https://github.com/wso2/api-manager/issues/3139#issuecomment-2394924817 [2] https://github.com/wso2/api-manager/issues/3241

Thanks, Tharika.

Avishka-Shamendra commented 5 days ago

DB2

Hi all,

Environment Details

Testing

Issues

01 - Error when loading the Change API Provider page in the Admin portal

The same error observed with MySQL migration was observed with Postgres as well

Thank you Avishka

tharikaGitHub commented 4 days ago

Update

The issue[1] was encountered during migration testing for all DB versions tested so far. This is a bug in APIM versions 4.2.0, 4.3.0 and also 4.4.0. Since the Jmeter script used to execute the data population has the same event order as mentioned in [1], the issue happens in the source version (APIM 4.2.0) itself. Hence this is not an issue triggered from migration. Created the public issue [1] for this.

[1] https://github.com/wso2/api-manager/issues/3247

Thanks, Tharika.

tharikaGitHub commented 4 days ago

Update

I have tested the APIM 4.3.0 to APIM 4.4.0 migration for MySQL. We had to do few changes to the data population jmeter script. The same jmeter change applies to APIM 4.2.0. The manual tests pass with the fixes introduced in [1].

[1] https://github.com/wso2/api-manager/issues/3139#issuecomment-2394924817

Thanks, Tharika.

tharikaGitHub commented 2 days ago

Update

Added the initial changes for the existing migration docs with PR [1].

[1] https://github.com/wso2-enterprise/migration-docs/pull/265

Thanks, Tharika.

Avishka-Shamendra commented 2 days ago

Oracle

Hi all,

Environment Details

Testing

Issues

01 - Error when loading the Change API Provider page in the Admin portal

The same error observed with MySQL migration was observed with Postgres as well

Thank you Avishka

Avishka-Shamendra commented 2 days ago

Update - Issue

Notice the following behavior on Oracle migrated pack, observed the same in the Postgres Migrated pack as well

Use tenant admin@adpexample.com

  1. In the migrated set up, deploy and publish the pizzashack API if you have not done so
  2. Go to ADPAPIProduct :1.0.0 -> Resources -> Edit Resources -> Add 'menu' from pizzashack API -> Select 'Save and Deploy'
  3. Go to dev portal and subscribe to ADPAPIProduct :1.0.0
  4. Try to invoke resource 'menu'.

Following error could be observed

[2024-10-13 22:10:59,717] ERROR - APIAuthenticationHandler API authentication failure due to Unclassified Authentication Failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Unclassified Authentication Failure
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.isAuthenticate_aroundBody64(APIAuthenticationHandler.java:616) ~[org.wso2.carbon.apimgt.gateway_9.30.43.SNAPSHOT.jar:?]

Thank you Avishka

tharikaGitHub commented 2 days ago

Update

The data populator scripts for APIM 4.2.0 and 4.3.0 are added via PR [1].

[1] https://github.com/wso2-enterprise/apim-migration-resources/pull/487

Thanks, Tharika.

Avishka-Shamendra commented 2 days ago

Update

The 4.2.0-4.4.0 db migration scripts have been added via PR [1]

[1] https://github.com/wso2-enterprise/migration-docs/pull/266

Thank you Avishka

Avishka-Shamendra commented 2 days ago

Update - Oracle

Env Details

Thank you Avishka

Avishka-Shamendra commented 2 days ago

Update - MSSQL

Env Details

Thank you Avishka

Avishka-Shamendra commented 1 day ago

Update - DB2

Env Details

Thank you Avishka

Avishka-Shamendra commented 1 day ago

Update - Postgres

Env Details

Thank you Avishka

Avishka-Shamendra commented 1 day ago

Update

The 4.3.0-4.4.0 db migration scripts have been added via PR [1]

[1] - https://github.com/wso2-enterprise/migration-docs/pull/268

Thank you Avishka

Avishka-Shamendra commented 1 day ago

Hi all,

The issue reported https://github.com/wso2/api-manager/issues/3139#issuecomment-2409047946 is not related to migration but rather to the payload used to create the API Product in the JMeter data populator script. The script did not account for the versioning support for products introduced in version 4.2.0, and it failed to include the version in the payload when creating the API Product. The JMeter script was fixed with [1], and this error has not been observed in migrations from 4.3.0 to 4.4.0 following the fix.

[1] - https://github.com/wso2-enterprise/apim-migration-resources/pull/487

Thank you Avishka