tharindu1st
commented
2 years ago can you share the data source configuration in deployment.toml file hiding sensitive information.
Open TomasTokaMrazek opened 2 years ago
tharindu1st
commented
2 years ago can you share the data source configuration in deployment.toml file hiding sensitive information.
TomasTokaMrazek
commented
2 years ago PG driver version: 42.2.23
There is nothing interesting, no custom values are used.
[database.apim_db]
type = "postgre"
url = "jdbc:postgresql://{{ .Values.wso2.deployment.am.database.hostname }}:{{ .Values.wso2.deployment.am.database.port }}/wso2_am400_apim_db?ssl=true&sslmode=require"
username = "{{ .Values.wso2.deployment.am.database.username }}"
password = "{{ .Values.wso2.deployment.am.database.password }}"
[database.shared_db]
type = "postgre"
url = "jdbc:postgresql://{{ .Values.wso2.deployment.am.database.hostname }}:{{ .Values.wso2.deployment.am.database.port }}/wso2_am400_shared_db?ssl=true&sslmode=require"
username = "{{ .Values.wso2.deployment.am.database.username }}"
password = "{{ .Values.wso2.deployment.am.database.password }}"Here's master-datasources.xml file.
<datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration">
<providers>
<provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider>
</providers>
<datasources>
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used for registry and user manager</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:h2:./repository/database/WSO2CARBON_DB;DB_CLOSE_ON_EXIT=FALSE</url>
<username>wso2carbon</username>
<password>wso2carbon</password>
<driverClassName>org.h2.Driver</driverClassName>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>true</defaultAutoCommit>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<maxActive>50</maxActive>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_SHARED_DB</name>
<description>Shared Database for user and registry data</description>
<jndiConfig>
<name>jdbc/SHARED_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:postgresql://redacted.postgres.database.azure.com:5432/wso2_am400_shared_db?ssl=true&sslmode=require</url>
<username>redacted</username>
<password>redacted</password>
<driverClassName>org.postgresql.Driver</driverClassName>
<validationQuery>SELECT 1; COMMIT</validationQuery>
<testOnBorrow>true</testOnBorrow>
<maxWait>60000</maxWait>
<defaultAutoCommit>true</defaultAutoCommit>
<validationInterval>30000</validationInterval>
<maxActive>50</maxActive>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2AM_DB</name>
<description>The datasource used for API Manager database</description>
<jndiConfig>
<name>jdbc/WSO2AM_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:postgresql://redacted.postgres.database.azure.com:5432/wso2_am400_apim_db?ssl=true&sslmode=require</url>
<username>redacted</username>
<password>redacted</password>
<driverClassName>org.postgresql.Driver</driverClassName>
<validationQuery>SELECT 1; COMMIT</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>true</defaultAutoCommit>
<testOnBorrow>true</testOnBorrow>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2_MB_STORE_DB</name>
<description>The datasource used for message broker database</description>
<jndiConfig>
<name>WSO2MBStoreDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<url>jdbc:h2:./repository/database/WSO2MB_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000;MVCC=TRUE</url>
<username>wso2carbon</username>
<password>wso2carbon</password>
<driverClassName>org.h2.Driver</driverClassName>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<maxWait>60000</maxWait>
<defaultAutoCommit>false</defaultAutoCommit>
<testOnBorrow>true</testOnBorrow>
<maxActive>50</maxActive>
</configuration>
</definition>
</datasource>
</datasources>
</datasources-configuration>I tried to set maxActive to 200 for shared db, just to see how many connections are actually there. It spawned new connections gradually for about 5 seconds, the final number was 134 connections at 60 visible APIs. Eventually everything cleared up all at once in 30 - 60 seconds, might be related to maxWait.
Here's a csv from pg_stat_activity table.
pg_stat_activity_202111161112.csv
I can simulate it every time, so feel free to ask whatever you need.
Description:
Our Devportal contains 56 published APIs. When a user with permission to all APIs logs into devportal and displays them at once (100 per page), devprotal stops working due to pool exhaustion to shared_db. Default size of 50 connections is instantaneously depleted. Connections wait idle in transaction on
SELECT UM_ID FROM UM_PERMISSION WHERE UM_RESOURCE_ID=$1 AND UM_ACTION=$2 AND UM_TENANT_ID=$3.My guess is that the transaction is released after the whole process completes. In other worlds for every API there is at least one transaction called from devportal, which is released after the whole operation is done. This appears to be after maxWait 60 seconds, when the rest of JDBC calls fail. Full stack trace is below.
Affected Product Version:
4.0.0
Environment details (with versions):
Stack Trace