Closed dstenger closed 2 years ago
To solidify requirements for this removal script, I am using the find-repeated-jars.sh documented in https://github.com/opengeospatial/cite-private/issues/212 suggested by @dstenger. I got these results with my test... I would expect us to keep the newest version by number. Other expectations? See my questions to the right.
=== Current jars ===|=== jars to be Removed; jars to be kept marked with '<' ===
ets-gml311-3.1.jar <
ets-gml32-0.1.jar ets-gml32-0.1.jar ### Do we need to keep different versions of ets?
ets-gml32-1.21.jar ets-gml32-1.21.jar
ets-gml32-1.25.jar ets-gml32-1.25.jar
ets-gml32-3.2.jar <
fii-boo-zoom-fast-4.7.0.jar fii-boo-zoom-fast-4.7.0.jar
fii-boo-zoom-fast-4.7.2.jar fii-boo-zoom-fast-4.7.2.jar
fii-boo-zoom-fast-4.7.jar < ### the 4.7.2 version should be kept. Do we need to worry about this use case?
foo-bar-baz-2.6.2.jar foo-bar-baz-2.6.2.jar
foo-bar-baz-2.7.2.jar foo-bar-baz-2.7.2.jar
foo-bar-baz-2.7.9.jar <
geotk-utility-3.21.jar <
geotk-utility-pending-3.21.jar geotk-utility-pending-3.21.jar ### Should both pending and "non pending" versions be kept?
json-20080701.jar json-20080701.jar
json-20090211.jar <
liba-2.1.jar liba-2.1.jar
liba-2.2.jar <
lib-client-1.17.1.jar lib-client-1.17.1.jar
lib-client-1.18.1.jar <
lib-server-3.16.jar lib-server-3.16.jar
lib-server-3.19.jar <
teamengine-spi-5.3.jar teamengine-spi-5.3.jar
teamengine-spi-ctl-5.3.jar < ### BUG This appears to be a different jar should and should keep both.
wii-gee-2019-11-11.jar < ### BUG Does not work -- will not fix?
wii-gee-2020-09-00.jar < ### BUG Does not work -- will not fix?
zii-bee-20201110.jar zii-bee-20201110.jar
zii-bee-20210990.jar <
A real example
=== Current jars ===|=== Jars to be Removed; Jars to be kept marked with '<' ===
activation-1.1.jar <
cite1-utils-1.1.1.jar <
collection-0.6.jar <
commons-cli-1.3.jar <
commons-codec-1.10.jar <
commons-codec-1.9.jar commons-codec-1.9.jar
commons-fileupload-1.3.1.jar <
commons-io-2.4.jar commons-io-2.4.jar
commons-io-2.5.jar <
commons-logging-1.2.jar <
derby-10.10.2.0.jar <
do <
ets-cat30-1.0.jar <
ets-gml32-1.21.jar ets-gml32-1.21.jar ### REMOVE or KEEP?
ets-gml32-1.25.jar <
ets-gpkg10-1.0.jar <
ets-gpkg12-1.0.jar <
ets-kml22-1.12.jar <
ets-sta10-1.0.jar <
ets-wfs11-1.29.jar <
ets-wfs20-1.26.jar <
ets-wms-client13-1.2.jar <
geoapi-pending-3.1-M04.jar <
geomatics-geotk-1.10.jar geomatics-geotk-1.10.jar
geomatics-geotk-1.14.jar <
geomatics-geotk-1.9.jar geomatics-geotk-1.9.jar
geotk-epsg-3.21.jar <
geotk-geometry-3.21.jar <
geotk-jtswrapper-3.21.jar <
geotk-metadata-3.21.jar <
geotk-metadata-sql-3.21.jar <
geotk-referencing-3.21.jar <
geotk-temporal-3.21.jar <
geotk-utility-3.21.jar < ### REMOVE OR KEEP non pending
geotk-utility-pending-3.21.jar geotk-utility-pending-3.21.jar ### REMOVE OR KEEP pending
geotk-xml-base-3.21.jar <
geotk-xml-gml-3.21.jar <
gson-2.2.2.jar <
hamcrest-core-1.3.jar <
httpclient-4.5.jar <
httpcore-4.4.1.jar <
isorelax-20030108.jar <
jai-imageio-core-1.3.1.jar <
javax.annotation-api-1.2-b03.jar <
javax.servlet.jsp.jstl-1.2.3.jar <
jaxen-1.0-FCS.jar <
jcip-annotations-1.0.jar <
jcommander-1.48.jar <
jena-base-3.1.0.jar <
jena-core-3.1.0.jar <
jena-iri-1.1.2.jar jena-iri-1.1.2.jar
jena-iri-3.1.0.jar <
jena-shaded-guava-3.1.0.jar <
jersey-client-1.19.jar <
jersey-multipart-1.19.jar <
jing-20091111.jar <
joda-time-2.8.jar joda-time-2.8.jar ### Should have same number of decimals
joda-time-2.9.4.jar <
json-20080701.jar json-20080701.jar ### BUG need to fix
json-20090211.jar json-20090211.jar ### BUG need to fix
json-simple-1.1.1.jar <
jsr-275-0.9.3.jar <
jstl-api-1.2.jar <
jts-1.13.jar <
junit-4.12.jar <
mail-1.4.7.jar <
mimepull-1.9.3.jar <
saxon9-9.0.0.8.jar <
saxpath-1.0-FCS.jar <
schema-utils-1.7.jar schema-utils-1.7.jar
schema-utils-1.8.jar <
slf4j-api-1.7.20.jar <
slf4j-jdk14-1.7.21.jar <
sqlite-jdbc-3.8.11.2.jar <
teamengine-core-5.3.jar <
teamengine-spi-5.3.jar teamengine-spi-5.3.jar ### BUG should be kept
teamengine-spi-ctl-5.3.jar <
testng-6.9.10.jar <
vecmath-1.5.2.jar <
webservices-api-2.3.jar <
webservices-rt-2.3.jar <
xercesImpl-2.11.0.jar <
xml-apis-1.4.01.jar <
xml-resolver-1.2.jar <
from teamengine docker /usr/local/tomcat/webapps/teamengine/WEB-INF/lib
@dstenger @ghobona
Do these rules work for removing jars?
As seen above...
Assumptions:
@kstegemoller I think we only need to keep the latest version of each ets jar.
@dstenger Please confirm.
ets-gml32-0.1.jar ets-gml32-0.1.jar ### Do we need to keep different versions of ets?
No, just one version. However, be careful that some names are longer: e.g. ets-wms13-dgiwg-1.0.jar. The current script is not able to handle those.
fii-boo-zoom-fast-4.7.jar < ### the 4.7.2 version should be kept. Do we need to worry about this use case?
As this is not a clean workflow anyway, it probably depends on the dependency. So, I would not worry about this use case for now.
geotk-utility-pending-3.21.jar geotk-utility-pending-3.21.jar ### Should both pending and "non pending" versions be kept?
Same as above. Please check how I did the manual deleting.
teamengine-spi-ctl-5.3.jar < ### BUG This appears to be a different jar should and should keep both.
Correct.
ets-gml32-1.21.jar ets-gml32-1.21.jar ### REMOVE or KEEP?
Old versions can be removed.
As far as I remember, I kept both versions if one was marked with "pending".
As this is just a workaround (which should be fixed by adjusting the versions in the test suites itself), in some cases there is no right or wrong (there might even be the case that one test suite needs one version and the other test suite another version; so both test suites are not compatible with each other).
So, I propose to keep it rather simple for now and do extensive testing of the new Beta environment instead.
E.g. create script to remove duplicated dependencies automatically.
Description of the problem: Currently, in many cases, there are the same dependencies in different versions in the classpath (lib folder of TEAM Engine webapp). This problem is caused by different test suites having the same dependency but not in the same version. Due to that fact, a random version of those multiple dependencies is chosen leading to system related test failures in different test suites (e.g. ets-wfs20;
java.lang.NoSuchMethodError: org.opengis.cite.validation.SchematronValidator.validate(Ljavax/xml/transform/Source;Z)Ljavax/xml/transform/Result;
). Because of that, we are deleting older versions of multiple dependencies manually so that just the newest version stays in the classpath (of course, this is not a good workflow and more a workaround; a general solution is needed here). When a Docker Container is started this task of deleting multiple dependencies should be executed automatically (e.g. via script).