Frightening amount of security issues in latest v4.x and v5.x

[kramer65]

I just ran Trivy (a vulnerability scanner) against our docker image with Cantaloupe v4.1.11 (the latest v4 version). The results are a little frightening:

Total: 242 (HIGH: 205, CRITICAL: 37) Full results of the scan: trivy-cantaloupe-v4.1.11.txt

I also tried v5.0.5, which gives me a slightly better, but still shocking:

Total: 97 (HIGH: 78, CRITICAL: 19) Full results of the scan: trivy-cantaloupe-v5.0.5.txt

What can we do to mitigate these vulnerabilities? Does anybody else face these issues?

[mrghosti3]

This could be mitigated by updating dependencies. But for now this project is not receiving updates (refer to discussion #613).

[hadro]

@glenrobson is out of the office this week, but per the last comment in the linked thread just above, the first thing he'll be doing to help support this project upon his return is working on dependency updates -- stay tuned!

[glenrobson]

As Josh mentions I'm looking at updating the library versions. I'm making some progress but come across the following issue:

package javax.media.jai does not exist

I've got the docker version working with the same pom file but testing it outside of docker I get this missing package issue. Pointers welcome!

[mrghosti3]

Searching for the mentioned lib mostly brings up this package: https://mvnrepository.com/artifact/Javax.media/jai_core

Maybe it got a name change since then?

[glenrobson]

Thanks, @mrghosti3 thats definitely an option but the fact it works using the docker file makes me think its coming in from somewhere else but I haven't spotted it yet...

[mrghosti3]

Then could it be cache issue? Is the container version compiled during image build?

[glenrobson]

I don't think so as the container builds cantaloupe with these lines:

RUN mvn --quiet dependency:resolve
ENTRYPOINT mvn --batch-mode test -Pfreedeps

and when I run them locally I get:

 mvn dependency:resolve
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building Cantaloupe 5.0.5
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:resolve (default-cli) @ cantaloupe ---
[INFO] 
[INFO] The following files have been resolved:
[INFO]    javax.xml.bind:jaxb-api:jar:2.4.0-b180830.0359:compile
[INFO]    org.eclipse.jetty.websocket:websocket-client:jar:9.4.50.v20221201:test
[INFO]    commons-net:commons-net:jar:3.9.0:test
....
[INFO]    com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.11.0:compile
[INFO]    javax.media:jai_codec:jar:1.1.3:compile
[INFO]    org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar:1.6.20:compile
[INFO]    org.apache.pdfbox:jbig2-imageio:jar:3.0.4:compile
[INFO]    software.amazon.awssdk:metrics-spi:jar:2.15.28:compile
...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.440 s
[INFO] Finished at: 2023-08-09T14:24:17+01:00
[INFO] Final Memory: 22M/88M
[INFO] ------------------------------------------------------------------------

which interestingly includes:

[INFO]    javax.media:jai_codec:jar:1.1.3:compile

which is the missing library...

Also just updated my version of mvn to the latest but that didn't fix it...

[glenrobson]

It looks like it was some sort of local maven cache issue. I went back to 1.3.2 of it.geosolutions.imageio-ext which seems to be the latest released version (even though 1.4.7 is available) and ran:

 mvn dependency:purge-local-repository

and that solved the missing jai files. Thanks @mrghosti3!

[mrghosti3]

No problem. Looking forward to a merge.

[glenrobson]

Just making notes before I drop off.

• seleniumhq cantaloupe using 2.53.0 but it looks like there is now a version 4 which might require some substantial re-writing of the tests.

Remaining security issues and JAR locations:

trivy -q fs . |awk '{ print $2}' |grep o |grep -v pom | while read line; do echo "$line"; mvn  dependency:tree -Dincludes="$line" |grep -A 3 edu.illinois|grep -v BUILD|grep -v "\[INFO\] -"|sed 's/\[INFO\]/ - /g' ; echo ''; done 
ch.qos.logback:logback-core
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- ch.qos.logback:logback-core:jar:1.2.8:compile

com.fasterxml.jackson.core:jackson-databind
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- com.fasterxml.jackson.core:jackson-databind:jar:2.11.0:compile

com.google.guava:guava
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.seleniumhq.selenium:selenium-api:jar:2.53.0:test
 -     \- com.google.guava:guava:jar:19.0:compile

io.netty:netty-handler
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- io.lettuce:lettuce-core:jar:6.2.4.RELEASE:compile
 -     \- io.netty:netty-handler:jar:4.1.91.Final:compile

org.codehaus.plexus:plexus-archiver
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.apache.maven.plugins:maven-assembly-plugin:maven-plugin:3.6.0:compile
 -     \- org.codehaus.plexus:plexus-archiver:jar:4.7.1:compile

org.eclipse.jetty.http2:http2-server
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.eclipse.jetty.http2:http2-server:jar:9.4.34.v20201102:compile

org.eclipse.jetty:jetty-http
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.eclipse.jetty:jetty-http:jar:9.4.34.v20201102:compile

org.eclipse.jetty:jetty-io
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.eclipse.jetty:jetty-io:jar:9.4.34.v20201102:compile

org.eclipse.jetty:jetty-server
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.eclipse.jetty:jetty-server:jar:9.4.34.v20201102:compile

org.eclipse.jetty:jetty-util
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.eclipse.jetty:jetty-util:jar:9.4.34.v20201102:compile

org.yaml:snakeyaml
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.11.0:compile
 -     \- org.yaml:snakeyaml:jar:1.26:compile

[glenrobson]

Currently on the following issue with:

<dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.198</version><!-- latest 2.2.220-->
            <scope>test</scope>
</dependency>

Versions 1.4.198 and up cause:

org.h2.jdbc.JdbcSQLFeatureNotSupportedException: Feature not supported: "Stream setter is not yet closed." [50100-198]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:504)
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:427)
    at org.h2.message.DbException.get(DbException.java:205)
    at org.h2.message.DbException.get(DbException.java:181)
    at org.h2.message.DbException.getUnsupportedException(DbException.java:269)
    at org.h2.jdbc.JdbcLob.checkReadable(JdbcLob.java:123)
    at org.h2.jdbc.JdbcLob.getBinaryStream(JdbcLob.java:154)
    at org.h2.jdbc.JdbcBlob.getBinaryStream(JdbcBlob.java:155)
    at org.h2.jdbc.JdbcPreparedStatement.setBlob(JdbcPreparedStatement.java:852)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.setBlob(HikariProxyPreparedStatement.java)
    at edu.illinois.library.cantaloupe.cache.JdbcCache$ImageBlobOutputStream.<init>(JdbcCache.java:87)
    at edu.illinois.library.cantaloupe.cache.JdbcCache.newDerivativeImageOutputStream(JdbcCache.java:391)
    at edu.illinois.library.cantaloupe.cache.JdbcCacheTest.seed(JdbcCacheTest.java:113)
    at edu.illinois.library.cantaloupe.cache.JdbcCacheTest.setUp(JdbcCacheTest.java:55)

Which looks to be a change that was made between 1.4.197 and 198 where it will throw the above exception if the setBlob method is called before the blob is populated with the data. Switching this to be written in the close function seems to fix this issue: https://github.com/glenrobson/cantaloupe/pull/1/commits/85a993a84d5841a151ff0b232841265723f3528a.

[glenrobson]

Next issue:

org.h2.jdbc.JdbcSQLNonTransientException: General error: "java.lang.NullPointerException"; SQL statement:
SELECT image FROM deriv WHERE operations = ? AND last_accessed >= ? [50000-198]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:502)
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:427)
    at org.h2.message.DbException.get(DbException.java:194)
    at org.h2.message.DbException.convert(DbException.java:347)
    at org.h2.command.Command.executeQuery(Command.java:219)
    at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:115)
    at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
    at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
    at edu.illinois.library.cantaloupe.cache.JdbcCache.newDerivativeImageInputStream(JdbcCache.java:374)
    at edu.illinois.library.cantaloupe.cache.AbstractCacheTest.lambda$1(AbstractCacheTest.java:189)
    at edu.illinois.library.cantaloupe.test.ConcurrentReaderWriter.lambda$1(ConcurrentReaderWriter.java:53)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
    at org.h2.store.LobStorageMap.copyLob(LobStorageMap.java:255)
    at org.h2.value.ValueLobDb.copyToResult(ValueLobDb.java:554)
    at org.h2.value.ValueLobDb.copyToResult(ValueLobDb.java:40)
    at org.h2.result.LocalResultImpl.cloneLobs(LocalResultImpl.java:280)
    at org.h2.result.LocalResultImpl.addRow(LocalResultImpl.java:315)
    at org.h2.command.dml.Select.queryFlat(Select.java:722)
    at org.h2.command.dml.Select.queryWithoutCache(Select.java:895)
    at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(Query.java:151)
    at org.h2.command.dml.Query.query(Query.java:435)
    at org.h2.command.dml.Query.query(Query.java:397)
    at org.h2.command.CommandContainer.query(CommandContainer.java:145)
    at org.h2.command.Command.executeQuery(Command.java:202)
    ... 7 more

Updating to:

<dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>2.2.220</version>
            <scope>test</scope>
        </dependency>

Seemed to fix this.

[glenrobson]

Currently at: [ERROR] Tests run: 3932, Failures: 1, Errors: 16, Skipped: 27

First:

cantaloupe_1  | [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.575 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.codec.tiff.TIFFMetadataTest
cantaloupe_1  | [ERROR] edu.illinois.library.cantaloupe.processor.codec.tiff.TIFFMetadataTest.testGetXMP  Time elapsed: 0.662 s  <<< ERROR!
cantaloupe_1  | org.apache.jena.riot.RiotException: [line: 6, col: 57] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1  |     at edu.illinois.library.cantaloupe.processor.codec.tiff.TIFFMetadataTest.testGetXMP(TIFFMetadataTest.java:83)
cantaloupe_1  |

It looks like the test file (src/test/resources/images/tif-xmp.tif) contains the following XMP data:

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
   <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
      <rdf:Description rdf:about=""
            xmlns:aux="http://ns.adobe.com/exif/1.0/aux/"
            xmlns:xmp="http://ns.adobe.com/xap/1.0/"
            xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
            xmlns:dc="http://purl.org/dc/elements/1.1/">
         <aux:Lens> 5.4-10.8mm</aux:Lens>
         <aux:FlashCompensation>0/1</aux:FlashCompensation>
         <aux:Firmware>Firmware Version 1.00</aux:Firmware>
         <aux:OwnerName>Alex Dolski</aux:OwnerName>
         <xmp:CreateDate>2002-07-14T09:01:42</xmp:CreateDate>
         <xmp:ModifyDate>2002-07-14T09:01:42</xmp:ModifyDate>
         <xmp:CreatorTool>Photos 1.5</xmp:CreatorTool>
         <photoshop:DateCreated>2002-07-14T09:01:42</photoshop:DateCreated>
         <dc:subject>
            <rdf:Bag>
               <rdf:li>Green Bay / De Pere</rdf:li>
               <rdf:li>Instructional Services Building</rdf:li>
               <rdf:li>UWGB</rdf:li>
            </rdf:Bag>
         </dc:subject>
      </rdf:Description>
   </rdf:RDF>
</x:xmpmeta>

Which has an empty rdf:about="". To pass it seems it needs to have a http/https URI. Version 3.13.1 of Jena allows an empty about property. It also works up to 3.17.0 but it looks like this version also has issues:

org.apache.commons:commons-compress
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.apache.jena:jena-core:jar:3.17.0:compile
 -     \- org.apache.jena:jena-base:jar:3.17.0:compile
 -        \- org.apache.commons:commons-compress:jar:1.20:compile

org.apache.jena:jena-core
 -  edu.illinois.library.cantaloupe:cantaloupe:jar:5.0.5
 -  \- org.apache.jena:jena-core:jar:3.17.0:compile

It looks like jena 4.8 brought in changes to the way IRI was structured so its no longer allowed to have about="". To fix this I added a base URI param to the test:

model.read(new StringReader(rdf), "file://" + srcFile.getParent().toAbsolutePath(), "RDF/XML");

Which produces the following about which is no longer empty:

<rdf:Description rdf:about="file:///Users/gmr/development/cantaloupe/src/test/resources/images">

This will cause problems if the test fixture is given a absolute IRI but will work for the current fixture.

If people are using the XMP headers for their images they will have to make sure that it has a populated about attribute.

Include in RELEASE_NOTES for the new version.

[glenrobson]

2nd:

cantaloupe_1  | [INFO] Running edu.illinois.library.cantaloupe.processor.TurboJpegProcessorTest
cantaloupe_1  | jpg : /home/cantaloupe/src/test/resources/images/jpg-xmp-extended.jpg
cantaloupe_1  | [ERROR] Tests run: 38, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 2.099 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.TurboJpegProcessorTest
cantaloupe_1  | [ERROR] edu.illinois.library.cantaloupe.processor.TurboJpegProcessorTest.testReadInfoOnAllFixtures  Time elapsed: 0.715 s  <<< ERROR!
cantaloupe_1  | org.apache.jena.riot.RiotException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1  |     at edu.illinois.library.cantaloupe.processor.TurboJpegProcessorTest.testReadInfoOnAllFixtures(TurboJpegProcessorTest.java:12)
cantaloupe_1  |

Similar problem but this one is embedded in edu.illinois.library.cantaloupe.processor.codec.jpeg.Util.readModel(Util.java:125) where the model is read from a JPEG file. The change adds a baseURI if the about is empty:

private static Model readModel(String rdfXML) {
        Model model = ModelFactory.createDefaultModel();
        String base = null;
        if (rdfXML.indexOf("rdf:about=''") != -1 || rdfXML.indexOf("rdf:about=\"\"") != -1) {
            // Version 4.8+ of jena requires a rdf:about link to not be empty
            base = "http://example.com";
        }
        try (StringReader reader = new StringReader(rdfXML)) {
            model.read(reader, base, "RDF/XML");
        }
        return model;
    }

Changed this to a more robust test which can handle spaces around the about attribute. Now it checks the exception and tries it again adding a Base URI:

Model model = ModelFactory.createDefaultModel();
        String base = null;
        try (StringReader reader = new StringReader(rdfXML)) {
            model.read(reader, base, "RDF/XML");
        } catch (RiotException exception) {
            if (exception.getMessage().indexOf("Base URI is null, but there are relative URIs to resolve") != -1) {
                // Version 4.8+ of jena requires a rdf:about link to not be empty
                try (StringReader reader = new StringReader(rdfXML)) {
                    model.read(reader, "http://example.com", "RDF/XML");
                }    
            } else {
                throw exception;
            }
        }
        return model;

[glenrobson]

3rd:

cantaloupe_1 | [ERROR] Tests run: 30, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.126 s <<< FAILURE! - in edu.illinois.library.cantaloupe.image.MetadataTest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.image.MetadataTest.testGetXMPModelWithPresentXMPData Time elapsed: 0.06 s <<< FAILURE!
cantaloupe_1 | org.opentest4j.AssertionFailedError: expected: <12> but was: <0>
cantaloupe_1 | at edu.illinois.library.cantaloupe.image.MetadataTest.testGetXMPModelWithPresentXMPData(MetadataTest.java:258)
cantaloupe_1 | 

Same fix as above in edu.illinois.library.cantaloupe.image.Metadata.loadXMP():

try (StringReader reader = new StringReader(xmp.get())) {
                xmpModel.read(reader, base, "RDF/XML");
            } catch (RiotException e) {
                if (e.getMessage().indexOf("Base URI is null, but there are relative URIs to resolve") != -1) {
                    // Version 4.8+ of jena requires a rdf:about link to not be empty
                    try (StringReader reader = new StringReader(xmp.get())) {
                        xmpModel.read(reader, "http://example.com", "RDF/XML");
                    } catch (RiotException exception) {
                        LOGGER.info("loadXMP(): {}", exception.getMessage());
                    }    
                } else {
                    LOGGER.info("loadXMP(): {}", e.getMessage());
                    throw e;
                }
            } catch (NullPointerException e) {
                // The XMP string may be invalid RDF/XML, or there may be a bug
                // in Jena (that would be the NPE). Not much we can do.
                LOGGER.info("loadXMP(): {}", e.getMessage());
            }

[glenrobson]

4th:

cantaloupe_1 | [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.445 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadataTest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadataTest.testGetXMPWithExtendedXMP Time elapsed: 0.871 s <<< ERROR!
cantaloupe_1 | org.apache.jena.riot.RiotException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadataTest.testGetXMPWithExtendedXMP(JPEGMetadataTest.java:72)
cantaloupe_1 |

Fixed by the JPG metadata fix.

[glenrobson]

5th:

cantaloupe_1 | [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.455 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.codec.png.PNGMetadataTest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.processor.codec.png.PNGMetadataTest.testGetXMP Time elapsed: 0.87 s <<< ERROR!
cantaloupe_1 | org.apache.jena.riot.RiotException: [line: 9, col: 59] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.codec.png.PNGMetadataTest.testGetXMP(PNGMetadataTest.java:50)
cantaloupe_1 |

Fixed by adding a baseURI:

@Test
    void testGetXMP() throws IOException {
        final String fixtureName = "png-xmp.png";
        final String rdf = getInstance(fixtureName).getXMP().orElseThrow();
        final Model model = ModelFactory.createDefaultModel();
        model.read(new StringReader(rdf), "file://" + TestUtil.getImage(fixtureName).getParent().toAbsolutePath(), "RDF/XML");
    }

[glenrobson]

6th:

cantaloupe_1 | [ERROR] Tests run: 41, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 6.729 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.Java2dProcessorTest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.processor.Java2dProcessorTest.testReadInfoOnAllFixtures Time elapsed: 0.073 s <<< ERROR!
cantaloupe_1 | com.fasterxml.jackson.databind.JsonMappingException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <> (through reference chain: edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadata["xmp"])
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.Java2dProcessorTest.testReadInfoOnAllFixtures(Java2dProcessorTest.java:19)
cantaloupe_1 | Caused by: org.apache.jena.riot.RiotException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.Java2dProcessorTest.testReadInfoOnAllFixtures(Java2dProcessorTest.java:19)
cantaloupe_1 |

Fixed by previous fixes.

[glenrobson]

7th:

cantaloupe_1 | [ERROR] Tests run: 19, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.449 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadataReaderTest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadataReaderTest.testGetXMPWithExtendedXMPImage Time elapsed: 0.602 s <<< ERROR!
cantaloupe_1 | org.apache.jena.riot.RiotException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadataReaderTest.testGetXMPWithExtendedXMPImage(JPEGMetadataReaderTest.java:183)

Fixed by previous fixes

[glenrobson]

8th:

cantaloupe_1 | [ERROR] Tests run: 9, Failures: 0, Errors: 9, Skipped: 0, Time elapsed: 6.681 s <<< FAILURE! - in edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testOPTIONSWhenEnabled Time elapsed: 5.095 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testOPTIONSWhenDisabled Time elapsed: 0.141 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testOverlaysSection Time elapsed: 0.155 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testSourceSection Time elapsed: 0.099 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testEndpointsSection Time elapsed: 0.086 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testServerSection Time elapsed: 0.108 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testApplicationSection Time elapsed: 0.109 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testProcessorsSection Time elapsed: 0.106 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.testCachesSection Time elapsed: 0.099 s <<< ERROR!
cantaloupe_1 | java.lang.NoSuchFieldError: DISMISS_AND_NOTIFY
cantaloupe_1 | at edu.illinois.library.cantaloupe.resource.admin.AdminResourceUITest.setUp(AdminResourceUITest.java:47)
cantaloupe_1 |

Fixed by reverting htmlunit-driver to 2.21 rather than pinning it to selenium version in https://github.com/glenrobson/cantaloupe/pull/1/commits/9fdabbaf4cb36ff6ed783ffee221d98b94fc54ff

[glenrobson]

9th:

cantaloupe_1 | [ERROR] Tests run: 40, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 6.245 s <<< FAILURE! - in edu.illinois.library.cantaloupe.processor.JaiProcessorTest
cantaloupe_1 | [ERROR] edu.illinois.library.cantaloupe.processor.JaiProcessorTest.testReadInfoOnAllFixtures Time elapsed: 0.183 s <<< ERROR!
cantaloupe_1 | com.fasterxml.jackson.databind.JsonMappingException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <> (through reference chain: edu.illinois.library.cantaloupe.processor.codec.jpeg.JPEGMetadata["xmp"])
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.JaiProcessorTest.testReadInfoOnAllFixtures(JaiProcessorTest.java:10)
cantaloupe_1 | Caused by: org.apache.jena.riot.RiotException: [line: 4, col: 49] {E211} Base URI is null, but there are relative URIs to resolve.: <>
cantaloupe_1 | at edu.illinois.library.cantaloupe.processor.JaiProcessorTest.testReadInfoOnAllFixtures(JaiProcessorTest.java:10)
cantaloupe_1 |

Fixed by previous fixes.

[glenrobson]

Came across an issue with the TurboJPEG tests:

edu.illinois.library.cantaloupe.processor.codec.jpeg.TurboJPEGImageWriterTest.testWriteWithCompressedTurboJPEGImage  Time elapsed: 0.084 s  <<< ERROR!
java.lang.IllegalStateException: No JPEG image is associated with this instance
    at edu.illinois.library.cantaloupe.processor.codec.jpeg.TurboJPEGImageWriterTest.testWriteWithCompressedTurboJPEGImage(TurboJPEGImageWriterTest.java:137)

Turned out to be an issue with the version of TurboJPEG. I couldn't get version 3 to work but I installed 2.1.5.1 by running:

export JAVA_HOME=/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home
cmake -DWITH_JAVA=1 libjpeg-turbo-2.1.5.1 -DJAVA_INCLUDE_PATH=$JAVA_HOME/include -DJAVA_INCLUDE_PATH2=$JAVA_HOME/include/darwin -DJAVA_AWT_INCLUDE_PATH=$JAVA_HOME/include
 sudo make install

[glenrobson]

Down to three errors:

cantaloupe_1  | [ERROR] Errors: 
cantaloupe_1  | [ERROR]   MetadataTest.testGetOrientationWithMalformedXMP:249 ? Riot [line: 1, col: 1 ] ...
cantaloupe_1  | [ERROR]   MetadataTest.testToMap:429 ? Riot [line: 1, col: 10] The prefix "rdf" for elem...
cantaloupe_1  | [ERROR]   EncodeTest.testToMap:145 ? Riot [line: 1, col: 10] The prefix "rdf" for elemen...
cantaloupe_1  | [INFO] 
cantaloupe_1  | [ERROR] Tests run: 3932, Failures: 0, Errors: 3, Skipped: 27

Second and third were issues where the RDF namespace wasn't defnied in the XMP so I added it the namespace in the test XML: https://github.com/glenrobson/cantaloupe/pull/1/commits/962dbff42d2b2e7ebf7d2fd9a62c4a2e24da9ee2 and https://github.com/glenrobson/cantaloupe/pull/1/commits/cb8997cb0b94b01d43fdfa3f7cbe1e8db7dd2dd6

The first issue was a problem where the new Jena code returned a RiotException rather than the expected IllegalArgumentException https://github.com/glenrobson/cantaloupe/pull/1/commits/a03e63c9168d5f93ab42828f46bd550f44e57d5a

[glenrobson]

Updated jetty and waiting to see what breaks but this leaves the following vunrabilities:

com.google.guava:guava comes from selenium-api:jar:2.53.0 which is only used during the automated testing.

commons-compress comes from Jena and Jena is on the latest version.

plexus-archiver comes from maven-assembly-plugin which is on the latest version and only affects the build.

[glenrobson]

Thats raised a few more issues:

[ERROR]   HttpSourceHTTPS1Test>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyReturningHash:474 » Script
[ERROR]   HttpSourceHTTPS1Test>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyReturningString:437 » Script
[ERROR]   HttpSourceHTTPS1Test>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyWithContextReturningString:458 » Script
[ERROR]   HttpSourceHTTPS2Test>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyReturningHash:474 » Script
[ERROR]   HttpSourceHTTPS2Test>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyReturningString:437 » Script
[ERROR]   HttpSourceHTTPS2Test>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyWithContextReturningString:458 » Script
[ERROR]   HttpSourceHTTPTest>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyReturningHash:474 » Script
[ERROR]   HttpSourceHTTPTest>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyReturningString:437 » Script
[ERROR]   HttpSourceHTTPTest>HttpSourceTest.testGetRequestInfoUsingScriptLookupStrategyWithContextReturningString:458 » Script

Fixed in https://github.com/glenrobson/cantaloupe/pull/1/commits/3b6b762e6e693891a2d5adbcb31ddbab36d29094 and was due to Ruby 3 moving uri.escape to cgi.escape (see https://github.com/ruby/uri/issues/14). It looks like only Ruby 3 is still current according to: https://www.ruby-lang.org/en/downloads/branches/

[glenrobson]

Agh! It looks like Grok has removed the debian release we were using in the automated testing:

#11 [ 6/13] RUN wget -q https://github.com/GrokImageCompression/grok/releases/download/v7.6.5/libgrokj2k1_7.6.5-1_amd64.deb     && wget -q https://github.com/GrokImageCompression/grok/releases/download/v7.6.5/grokj2k-tools_7.6.5-1_amd64.deb     && dpkg -i 

The releases only contain version 11 now:

https://github.com/GrokImageCompression/grok/releases/

Grok is aviliable in Ubuntu versions (lunar, mantic and noble):

https://packages.ubuntu.com/lunar/grokj2k-tools

but not in jessie which is which we are currently using (ubuntu:latest from https://hub.docker.com/_/ubuntu)

So its either update the DockerFile to use lunar or install grok from source...

I've gone for updating ubuntu to use lunar in https://github.com/glenrobson/cantaloupe/pull/1/commits/9556c9ae188e5d892cf2d2404afb7c7d05d0159c

[glenrobson]

Build all green:

https://github.com/glenrobson/cantaloupe/pull/1

So I've contacted the folks at Illinois to see how we can get this released.

[janhoy]

So what's the status of this release? Were they too busy to do the release? Or was there a problem with your PR? Perhaps the best way is to assign you or someone with time and interest as a project maintainer to increase the projects' bandwidth?

[glenrobson]

Hi @Janhoy, I'm afraid its on me. I was given permssions to create a release before chritsmas but have been buried in other work. I will try and get the release done next week.

[glenrobson]

This will teach me to leave this and not finish the release sooner but there are some more security issues:

┌─────────────────────────────────────┬────────────────┬──────────┬────────┬───────────────────┬──────────────────────────┬──────────────────────────────────────────────────────────────┐
│               Library               │ Vulnerability  │ Severity │ Status │ Installed Version │      Fixed Version       │                            Title                             │
├─────────────────────────────────────┼────────────────┼──────────┼────────┼───────────────────┼──────────────────────────┼──────────────────────────────────────────────────────────────┤
│ ch.qos.logback:logback-classic      │ CVE-2023-6378  │ HIGH     │ fixed  │ 1.2.12            │ 1.3.12, 1.4.12, 1.2.13   │ logback: serialization vulnerability in logback receiver     │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-6378                    │
├─────────────────────────────────────┤                │          │        │                   │                          │                                                              │
│ ch.qos.logback:logback-core         │                │          │        │                   │                          │                                                              │
│                                     │                │          │        │                   │                          │                                                              │
│                                     ├────────────────┤          │        │                   ├──────────────────────────┼──────────────────────────────────────────────────────────────┤
│                                     │ CVE-2023-6481  │          │        │                   │ 1.4.14, 1.3.14, 1.2.13   │ logback: A serialization vulnerability in logback receiver   │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-6481                    │
├─────────────────────────────────────┼────────────────┼──────────┤        ├───────────────────┼──────────────────────────┼──────────────────────────────────────────────────────────────┤
│ com.google.guava:guava              │ CVE-2023-2976  │ MEDIUM   │        │ 24.1.1-jre        │ 32.0.0-android           │ guava: insecure temporary directory creation                 │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-2976                    │
│                                     ├────────────────┼──────────┤        │                   │                          ├──────────────────────────────────────────────────────────────┤
│                                     │ CVE-2020-8908  │ LOW      │        │                   │                          │ local information disclosure via temporary directory created │
│                                     │                │          │        │                   │                          │ with unsafe permissions                                      │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2020-8908                    │
├─────────────────────────────────────┼────────────────┼──────────┤        ├───────────────────┼──────────────────────────┼──────────────────────────────────────────────────────────────┤
│ com.squareup.okio:okio-jvm          │ CVE-2023-3635  │ MEDIUM   │        │ 3.2.0             │ 3.4.0                    │ okio: GzipSource class improper exception handling           │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-3635                    │
├─────────────────────────────────────┼────────────────┤          │        ├───────────────────┼──────────────────────────┼──────────────────────────────────────────────────────────────┤
│ org.apache.commons:commons-compress │ CVE-2023-42503 │          │        │ 1.23.0            │ 1.24.0                   │ Improper Input Validation, Uncontrolled Resource Consumption │
│                                     │                │          │        │                   │                          │ vulnerabi ...                                                │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-42503                   │
├─────────────────────────────────────┼────────────────┼──────────┤        ├───────────────────┼──────────────────────────┼──────────────────────────────────────────────────────────────┤
│ org.codehaus.plexus:plexus-archiver │ CVE-2023-37460 │ HIGH     │        │ 4.7.1             │ 4.8.0                    │ plexus-archiver: Arbitrary File Creation in                  │
│                                     │                │          │        │                   │                          │ AbstractUnArchiver                                           │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-37460                   │
├─────────────────────────────────────┼────────────────┤          │        ├───────────────────┼──────────────────────────┼──────────────────────────────────────────────────────────────┤
│ org.eclipse.jetty.http2:http2-hpack │ CVE-2023-36478 │          │        │ 9.4.52.v20230823  │ 10.0.16, 11.0.16, 9.4.53 │ jetty: hpack header values cause denial of service in http/2 │
│                                     │                │          │        │                   │                          │ https://avd.aquasec.com/nvd/cve-2023-36478                   │
└─────────────────────────────────────┴────────────────┴──────────┴────────┴───────────────────┴──────────────────────────┴──────────────────────────────────────────────────────────────┘

[glenrobson]

Fixes:

• logback - updated
• okio-jvm - updated okhttp3

• commons-compress updated Jena

  Can't fix:

  • plexus-archiver comes from maven-assembly-plugin which is on the latest version and only affects the build.
• com.google.guava:guava comes from selenium-api:jar:2.53.1 which is only used during the automated testing. Later versions will require re-writing the tests.
• plexus-archiver - from maven-assembly-plugin which is on the latest version

Down to:

┌─────────────────────────────────────┬────────────────┬──────────┬────────┬───────────────────┬────────────────┬──────────────────────────────────────────────────────────────┐
│               Library               │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version  │                            Title                             │
├─────────────────────────────────────┼────────────────┼──────────┼────────┼───────────────────┼────────────────┼──────────────────────────────────────────────────────────────┤
│ com.google.guava:guava              │ CVE-2023-2976  │ MEDIUM   │ fixed  │ 24.1.1-jre        │ 32.0.0-android │ guava: insecure temporary directory creation                 │
│                                     │                │          │        │                   │                │ https://avd.aquasec.com/nvd/cve-2023-2976                    │
│                                     ├────────────────┼──────────┤        │                   │                ├──────────────────────────────────────────────────────────────┤
│                                     │ CVE-2020-8908  │ LOW      │        │                   │                │ local information disclosure via temporary directory created │
│                                     │                │          │        │                   │                │ with unsafe permissions                                      │
│                                     │                │          │        │                   │                │ https://avd.aquasec.com/nvd/cve-2020-8908                    │
├─────────────────────────────────────┼────────────────┼──────────┤        ├───────────────────┼────────────────┼──────────────────────────────────────────────────────────────┤
│ org.codehaus.plexus:plexus-archiver │ CVE-2023-37460 │ HIGH     │        │ 4.7.1             │ 4.8.0          │ plexus-archiver: Arbitrary File Creation in                  │
│                                     │                │          │        │                   │                │ AbstractUnArchiver                                           │
│                                     │                │          │        │                   │                │ https://avd.aquasec.com/nvd/cve-2023-37460                   │
└─────────────────────────────────────┴────────────────┴──────────┴────────┴───────────────────┴────────────────┴──────────────────────────────────────────────────────────────┘

[glenrobson]

So I think this is done now with the release of 5.0.6:

https://github.com/cantaloupe-project/cantaloupe/releases