Critical and High vulnerabilities reported in hudi jar

[shivangigarg1]

Tips before filing an issue

• Have you gone through our FAQs? yes

• Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

• If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced

Twistlock scan tool reported multiple critical and high vulnerabilities issue for hudi jar Tried upgrading fasterxml.version in pom.xml and build it. But, the final package still gets the old version of jackson-databind - 2.4.0 and is not picking the right one.

Are there any pointers resolve these vulnerabilities?

Environment Description

• Hudi version : 0.14.0

• Spark version : 3.4

• Hive version : 2.3.1

• Hadoop version : 2.10.1

[danny0405]

good point, can you fire a JIRA issue and let's trace the issues.

[ad1happy2go]

Thanks a lot for raising this @shivangigarg1 . Created a tracking jira for this so we don't miss it - https://issues.apache.org/jira/browse/HUDI-7521

Can you provide the updated pom (after that also it is picking old version) and which profile you are trying to build?

[shivangigarg1]

@ad1happy2go Tried multiple options

1. Updated fasterxml to latest version - 2.16.1 https://github.com/apache/hudi/blob/7c55ac35ba11ed00151bf9d536aecdb5d83af33f/pom.xml#L98
2. From the build logs it was indicated that 2.4.0 version files are added as overlapping resource from htrace-core and hbase-client so tried after removing them from shaded plugin - https://github.com/apache/hudi/blob/7c55ac35ba11ed00151bf9d536aecdb5d83af33f/pom.xml#L459C29-L459C30 But nothing helped

And I am trying to build hudi-spark-bundle

[CTTY]

The older jackson 2.4.0 may come from htrace and it should be under a shaded classpath (org/apache/htrace/shaded/fasterxml/jackson). Thus, it should only be used by htrace. Hudi itself uses a much newer version of jackson.

This got flagged probably because there are pom files of older jackson got bundled into the jar, you can use vim to open the jar and find the pom file, and then press enter to verify this.

To prevent Hudi from getting flagged like this in the future, we can filter these pom files out from bundle jars. I'll post a PR later

[shivangigarg1]

@CTTY when can we expect the PR ?

[CTTY]

@shivangigarg1 After checking the code base, I don't think a patch makes sense to the opensource Hudi as the latest Hudi switched to use native HFile reader and doesn't depend on htrace anymore

[shivangi24]

Hello @CTTY Is the opensource Hudi upgraded to 2.5.x version of hbase, as when I check for hbase dependency in 2.4.x version it has dependency on htrace-core and untill we are using 2.4.x version of hbase will continue getting fasterxml vulnerabilities.

[kishoreraj05]

@ad1happy2go can you please update the ticket to fix the below reported vulnerability also? https://nvd.nist.gov/vuln/detail/CVE-2018-1282 The fixes for this vulnerability is to upgrade the hive-jdbc version. `${hive.groupid}

hive-jdbc

    <version>${hive.version}</version>
    <scope>provided</scope>`