search

quanted / cts_app

cts repo
3 stars 3 forks source link

BioTransformer log4j vulnerabilities #188

Open popenc opened 2 years ago

popenc commented 2 years ago

Prisma cloud scan is showing critical and high level security vulnerabilities:

Vulnerability Filename CVE-2019-17571 log4j_log4j:1.2.17 CVE-2018-7489 com.fasterxml.jackson.core_jackson-databind:2.4.2 CVE-2020-35491 com.fasterxml.jackson.core_jackson-databind:2.4.2 CVE-2020-35490 com.fasterxml.jackson.core_jackson-databind:2.4.2 CVE-2019-10086 commons-beanutils_commons-beanutils:1.9.3

popenc commented 2 years ago

Biotransformer currently shut down on GDIT dev and qed.epa.gov until above vulnerabilities are resolved.

popenc commented 2 years ago

Siyang Tian from BioTransformer has responded and will resolve the vulnerabilities.

Le0nT1 commented 2 years ago

Hi, This is Siyang. Can you please try the BioTransformer3.0_20220504.jar which can be found at: https://bitbucket.org/wishartlab/biotransformer3.0jar/src/master/ Do you think it's better to work on this issue on this GitHub discussion thread or through the original emails?

popenc commented 2 years ago

Hey Siyang, thank you for the update. I will try the new jar by the end of this week.

Tracking the discussion on here would be great.

popenc commented 2 years ago

Pulled down the latest at https://bitbucket.org/wishartlab/biotransformer3.0jar/src/master/ and re-ran Prisma image scans in our Gitlab pipeline. Still getting the same flagged vulnerabilities at the moment:

  1. CVE-2019-17571
    • log4j_log4j
    • v1.2.17
  2. CVE-2018-7489
    • com.fasterxml.jackson.core_jackson-databind
    • v2.4.2
  3. CVE-2020-35491
    • com.fasterxml.jackson.core_jackson-databind
    • v2.4.2
  4. CVE-2020-35490
    • com.fasterxml.jackson.core_jackson-databind
    • v2.4.2
  5. CVE-2019-10086
    • commons-beanutils_commons-beanutils
    • v1.9.3

I will open the jar and looks further into where these are being flagged. Looks like the pipeline is scanning the latest image with the new jar.

popenc commented 2 years ago

I've attached a document that tracks where the vulnerable versions are located in BioTransformer3.0_20220504.jar.

Tracking Vulnerable Version Locations in BioTransformer.docx

Le0nT1 commented 2 years ago

I have updated the jar file to BioTransformer3.0_20220615.jar and checked the Dependency Hierarchy. The only two versions of log4j that can be used in this release should be 2.17.2 and 2.17.1 and there should be no v1.2.17 being used. Please let me know if the issue still exists. If so, any suggestion that can help solve the problem will be appreciated.

Thanks

popenc commented 2 years ago

@Le0nT1, thanks for the update! I will test this out by the end of this week.

popenc commented 2 years ago

@Le0nT1, I ran our code with BioTransformer3.0_20220615.jar and the log4j vulnerability showing up in the Prisma cloud scan is resolved now.

There a some remaining CVEs from two libraries. I think the way to resolve them would be to update their versions in the pom.xml files and to rebuild the jar file, although I'm not very familiar with Java. Below are the CVEs from the two libraries, their current versions with the CVEs, the upgraded version to resolve the vulnerabilities, and the locations I'm seeing the vulnerable versions.

  1. CVE-2018-7489, CVE-2020-35491, CVE-2020-35490

    • Affected library and version: com.fasterxml.jackson.core_jackson-databind v2.4.2
    • Resolved by updating to >= v2.9.10.8
    • Locations I've found this library referenced in the jar file:
      1. CypReact.iml
      2. cyProduct.iml
      3. meta-inf\maven\com.fasterxml.jackson.core\jackson-databind\pom.properties
      4. meta-inf\maven\com.fasterxml.jackson.core\jackson-databind\pom.xml
      5. meta-inf\maven\com.fasterxml.jackson.core\jackson-core\pom.properties
      6. meta-inf\maven\com.fasterxml.jackson.core\jackson-core\pom.xml

  2. CVE-2019-10086

Let me know if this helps, or if there's anything I can do to assist. As a side note, we've been using biotransformer with an API wrapper (https://github.com/quanted/bt_api) for a while now (it's a great tool and we're huge fans) and our current EPA API/App scans do not throw any high or critical level vulnerabilities. The above are being flagged when running Prisma cloud scanning, which scans the contents of the code itself and not just what can be hit from a server.

popenc commented 2 years ago

Updates with log4j resolutions added to dev and epa staging.

NOTE: If biotransformer CVEs are still being flagged in Prisma via the Gitlab pipeline, this along with OPERAWS can be deployed on a separate backend server that goes through the routine manual app scans, which do pass.

Le0nT1 commented 2 years ago

Hi, I have been working on other developments during the past month. I will start looking into the remaining CVEs in August.

popenc commented 2 years ago

Hey @Le0nT1, thanks for the update, and apologies if you're getting a ton of emails as I fumble with adding/editing comments on here lol.

popenc commented 2 years ago

@Le0nT1, I have a question about a connection time out I'm seeing with the latest biotransformer jar: BioTransformer3.0_20220615.jar

This is only occurring on our EPA AWS server, where we need firewall rules for anything outbound. On our dev server we don't get this issue.

I'm getting this series of errors (the data still comes back, but it takes a very long time with these time outs, just wondering if you'd know what's going on here):

Processing molecule with SMILES: CC(=O)OC1=CC=CC=C1C(O)=O com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:828) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:683) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:230) at biotransformer.dbrelevant.RetriveFromDB.isEndogenous(RetriveFromDB.java:233) at biotransformer.utils.filterCertainClasses.FilterCertainClasses.retriveFromDB(FilterCertainClasses.java:35) at biotransformer.railsappspecific.Cyp450BTransformer_rails.predictCyp450BiotransformationChainByMode(Cyp450BTransformer_rails.java:461) at executable.BiotransformerExecutable.main(BiotransformerExecutable.java:818) Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89) at com.mysql.cj.NativeSession.connect(NativeSession.java:120) at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:948) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) ... 9 more Caused by: java.net.ConnectException: Connection timed out at java.base/sun.nio.ch.Net.connect0(Native Method) at java.base/sun.nio.ch.Net.connect(Net.java:579) at java.base/sun.nio.ch.Net.connect(Net.java:568) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:585) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:156) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63) ... 12 more com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:828) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:683) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:230) at biotransformer.dbrelevant.RetriveFromDB.isEndogenous(RetriveFromDB.java:233) at biotransformer.utils.filterCertainClasses.FilterCertainClasses.retriveFromDB(FilterCertainClasses.java:35) at biotransformer.railsappspecific.Cyp450BTransformer_rails.predictCyp450BiotransformationChainByMode(Cyp450BTransformer_rails.java:480) at biotransformer.railsappspecific.Cyp450BTransformer_rails.predictCyp450BiotransformationChainByMode(Cyp450BTransformer_rails.java:466) at executable.BiotransformerExecutable.main(BiotransformerExecutable.java:818) Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89) at com.mysql.cj.NativeSession.connect(NativeSession.java:120) at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:948) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) ... 10 more Caused by: java.net.ConnectException: Connection timed out at java.base/sun.nio.ch.Net.connect0(Native Method) at java.base/sun.nio.ch.Net.connect(Net.java:579) at java.base/sun.nio.ch.Net.connect(Net.java:568) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:585) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:156) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63) ... 13 more CYP450 num of iterations done: 1 Substrates for next iteration: 3 Unique Biotransformations: 3 Unique metabolites: 3 The results were saved to the following file: /src/temp/77f8e514-3d1d-49b9-9bf6-addad3439c15.csv

Successfully completed metabolism prediction for 1 out of 1 molecule(s). Total time consumption: 260020

Le0nT1 commented 2 years ago

@Le0nT1, I have a question about a connection time out I'm seeing with the latest biotransformer jar: BioTransformer3.0_20220615.jar

This is only occurring on our EPA AWS server, where we need firewall rules for anything outbound. On our dev server we don't get this issue.

I'm getting this series of errors (the data still comes back, but it takes a very long time with these time outs, just wondering if you'd know what's going on here):

Processing molecule with SMILES: CC(=O)OC1=CC=CC=C1C(O)=O com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:828) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:683) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:230) at biotransformer.dbrelevant.RetriveFromDB.isEndogenous(RetriveFromDB.java:233) at biotransformer.utils.filterCertainClasses.FilterCertainClasses.retriveFromDB(FilterCertainClasses.java:35) at biotransformer.railsappspecific.Cyp450BTransformer_rails.predictCyp450BiotransformationChainByMode(Cyp450BTransformer_rails.java:461) at executable.BiotransformerExecutable.main(BiotransformerExecutable.java:818) Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89) at com.mysql.cj.NativeSession.connect(NativeSession.java:120) at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:948) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) ... 9 more Caused by: java.net.ConnectException: Connection timed out at java.base/sun.nio.ch.Net.connect0(Native Method) at java.base/sun.nio.ch.Net.connect(Net.java:579) at java.base/sun.nio.ch.Net.connect(Net.java:568) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:585) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:156) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63) ... 12 more com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:828) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:683) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:230) at biotransformer.dbrelevant.RetriveFromDB.isEndogenous(RetriveFromDB.java:233) at biotransformer.utils.filterCertainClasses.FilterCertainClasses.retriveFromDB(FilterCertainClasses.java:35) at biotransformer.railsappspecific.Cyp450BTransformer_rails.predictCyp450BiotransformationChainByMode(Cyp450BTransformer_rails.java:480) at biotransformer.railsappspecific.Cyp450BTransformer_rails.predictCyp450BiotransformationChainByMode(Cyp450BTransformer_rails.java:466) at executable.BiotransformerExecutable.main(BiotransformerExecutable.java:818) Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89) at com.mysql.cj.NativeSession.connect(NativeSession.java:120) at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:948) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) ... 10 more Caused by: java.net.ConnectException: Connection timed out at java.base/sun.nio.ch.Net.connect0(Native Method) at java.base/sun.nio.ch.Net.connect(Net.java:579) at java.base/sun.nio.ch.Net.connect(Net.java:568) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:585) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:156) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63) ... 13 more CYP450 num of iterations done: 1 Substrates for next iteration: 3 Unique Biotransformations: 3 Unique metabolites: 3 The results were saved to the following file: /src/temp/77f8e514-3d1d-49b9-9bf6-addad3439c15.csv

Successfully completed metabolism prediction for 1 out of 1 molecule(s). Total time consumption: 260020

It's because BioTransformer will try to query HMDB5.0 for some endogenous molecules. You can disable it using parameter -useDB false or wait till I push an update later. In the new version, the useDB feature will be disabled by dafault and won't be used till the user uses -useDB true.

popenc commented 2 years ago

Awesome, thanks for the info!

popenc commented 1 year ago

Prisma image scans are back online and have been reapplied to the cts_kube stack. Getting the same CVEs listed previously in https://github.com/quanted/cts_app/issues/188#issuecomment-1161668106