sevntu-checkstyle / sevntu.checkstyle

Additional Checkstyle checks, that could be added as extension to EclipseCS plugin and maven-checkstyle-plugin, Sonar checkstyle plugin, extension for CheckStyle IDEA plugin.
http://sevntu-checkstyle.github.io/sevntu.checkstyle/
190 stars 147 forks source link

infra: added quarterly cache clean up for travis #931

Open rnveach opened 1 year ago

rnveach commented 1 year ago

Similar to https://github.com/checkstyle/checkstyle/pull/12365#issue-1435499136

rnveach commented 1 year ago

Will change to quarterly after run and stats.

coveralls commented 1 year ago

Coverage Status

Coverage remained the same at 98.798% when pulling a2c8961d495cbd77045c9b43ccc171c7f4e3fbef on rnveach:monthly-cache-cleanup into c74de7fb5c6e1893e3bd9c528e71dada1741cb65 on sevntu-checkstyle:master.

rnveach commented 1 year ago

Is interesting here. Master is 2600 megs but this PR only reduced it to 1,000 megs (38% reduction).

It is a similar savings, but I am surprised we have 1 gig of dependencies.

rnveach commented 1 year ago

90 might be better

Done.

rnveach commented 1 year ago

https://app.travis-ci.com/github/sevntu-checkstyle/sevntu.checkstyle/jobs/587535354

[WARNING] Error initializing: org.eclipse.tycho.p2.resolver.P2DependencyResolver@10db6131 java.lang.RuntimeException: java.io.IOException: File /home/travis/.m2/repository/org/eclipse/tycho/tycho-bundles-external/1.6.0/eclipse/configuration/config.ini does not exist

Rerunning to see if it is stable.

rnveach commented 1 year ago

From my local:

/.m2/repository/org/eclipse/tycho/tycho-bundles-external/1.6.0$ find -follow
.
./_remote.repositories
./tycho-bundles-external-1.6.0.zip
./tycho-bundles-external-1.6.0.zip.sha1
./eclipse
./eclipse/configuration
./eclipse/configuration/config.ini
./eclipse/.eclipseproduct
./eclipse/plugins
./eclipse/plugins/org.sat4j.core_2.3.5.v201308161310.jar
./eclipse/plugins/org.eclipse.equinox.p2.jarprocessor_1.1.500.v20191014-1732.jar
./eclipse/plugins/org.eclipse.equinox.p2.updatesite_1.1.300.v20191016-0515.jar
./eclipse/plugins/org.tukaani.xz_1.8.0.v20180207-1613.jar
./eclipse/plugins/org.eclipse.equinox.p2.artifact.repository_1.3.300.v20191015-1504.jar
./eclipse/plugins/org.eclipse.equinox.p2.garbagecollector_1.1.300.v20191015-1207.jar
./eclipse/plugins/org.eclipse.core.contenttype_3.7.500.v20190916-2125.jar
./eclipse/plugins/org.eclipse.equinox.p2.metadata_2.4.600.v20191002-0702.jar
./eclipse/plugins/org.eclipse.core.jobs_3.10.600.v20191122-2104.jar
./eclipse/plugins/org.eclipse.equinox.concurrent_1.1.400.v20190621-0852.jar
./eclipse/plugins/org.eclipse.tycho.noopsecurity_1.6.0.jar
./eclipse/plugins/org.apache.commons.codec_1.10.0.v20180409-1845.jar
./eclipse/plugins/org.apache.httpcomponents.httpclient_4.5.6.v20190503-0009.jar
./eclipse/plugins/org.eclipse.equinox.simpleconfigurator.manipulator_2.1.400.v20191015-1836.jar
./eclipse/plugins/org.eclipse.ecf.provider.filetransfer.httpclient45_1.0.100.v20191012-1656.jar
./eclipse/plugins/org.eclipse.equinox.security_1.3.400.v20191008-0645.jar
./eclipse/plugins/org.eclipse.equinox.registry_3.8.600.v20191017-2055.jar
./eclipse/plugins/org.eclipse.equinox.p2.engine_2.6.500.v20191014-1645.jar
./eclipse/plugins/org.eclipse.core.runtime_3.17.0.v20191122-2104.jar
./eclipse/plugins/org.eclipse.osgi.services_3.8.0.v20190206-2147.jar
./eclipse/plugins/org.sat4j.pb_2.3.5.v201404071733.jar
./eclipse/plugins/org.eclipse.core.net_1.3.700.v20191122-2107.jar
./eclipse/plugins/org.eclipse.equinox.p2.publisher.eclipse_1.3.400.v20191018-2256.jar
./eclipse/plugins/org.eclipse.equinox.p2.transport.ecf_1.2.300.v20191001-0955.jar
./eclipse/plugins/org.eclipse.osgi_3.15.100.v20191114-1701.jar
./eclipse/plugins/org.eclipse.equinox.p2.repository.tools_2.2.400.v20191016-0510.jar
./eclipse/plugins/org.eclipse.ecf.ssl_1.2.300.v20191012-1926.jar
./eclipse/plugins/org.eclipse.equinox.ds_1.6.100.v20191001-1333.jar
./eclipse/plugins/org.eclipse.equinox.app_1.4.300.v20190815-1535.jar
./eclipse/plugins/org.eclipse.ecf.filetransfer_5.0.300.v20191020-1846.jar
./eclipse/plugins/org.eclipse.ecf_3.9.4.v20191020-1846.jar
./eclipse/plugins/org.eclipse.equinox.common_3.10.600.v20191004-1420.jar
./eclipse/plugins/org.apache.commons.logging_1.2.0.v20180409-1502.jar
./eclipse/plugins/org.eclipse.equinox.simpleconfigurator_1.3.400.v20191015-1836.jar
./eclipse/plugins/org.eclipse.equinox.frameworkadmin_2.1.400.v20191002-0702.jar
./eclipse/plugins/org.eclipse.equinox.p2.publisher_1.5.300.v20191016-0507.jar
./eclipse/plugins/org.eclipse.equinox.p2.core_2.6.200.v20191014-1220.jar
./eclipse/plugins/org.eclipse.equinox.frameworkadmin.equinox_1.1.200.v20191015-1207.jar
./eclipse/plugins/org.eclipse.equinox.p2.director.app_1.1.400.v20191015-1945.jar
./eclipse/plugins/org.eclipse.equinox.preferences_3.7.600.v20191017-2055.jar
./eclipse/plugins/org.apache.felix.scr_2.1.14.v20190123-1619.jar
./eclipse/plugins/org.eclipse.equinox.launcher_1.5.600.v20191014-2022.jar
./eclipse/plugins/org.eclipse.osgi.util_3.5.300.v20190708-1141.jar
./eclipse/plugins/org.eclipse.equinox.p2.director_2.4.500.v20191014-1220.jar
./eclipse/plugins/org.eclipse.equinox.p2.touchpoint.natives_1.3.400.v20191017-1917.jar
./eclipse/plugins/org.eclipse.equinox.p2.metadata.repository_1.3.300.v20191002-0702.jar
./eclipse/plugins/org.eclipse.ecf.provider.filetransfer.ssl_1.0.101.v20191012-1926.jar
./eclipse/plugins/org.eclipse.equinox.p2.touchpoint.eclipse_2.2.500.v20191114-1206.jar
./eclipse/plugins/org.eclipse.osgi.compatibility.state_1.1.600.v20190814-1451.jar
./eclipse/plugins/org.eclipse.equinox.p2.repository_2.4.600.v20191016-0510.jar
./eclipse/plugins/org.eclipse.ecf.identity_3.9.300.v20191020-1846.jar
./eclipse/plugins/org.apache.httpcomponents.httpcore_4.4.10.v20190123-2214.jar
./eclipse/plugins/org.eclipse.ecf.provider.filetransfer_3.2.500.v20191017-1905.jar

Haven't checked travis, but it seems like tycho is using this folder for its own purposes and doesn't realize when things are partially removed.

romani commented 1 year ago

Maybe it s better to just remove whole cache ones in several months( Travis cron execution) or skip eclipse from removal.

rnveach commented 1 year ago

Confirmed on my local that tycho will reset when the whole directory is deleted and not fail.

It must unzip the zip file it stores there as the dates on the newly created files are 2020, so that explains why they were slated for deletion. The zip file has todays date and that is why it wasn't deleted.

rnveach commented 1 year ago

Adding -type d will ensure we delete the whole directory but we would still require to ignore this directory and sub-folders.

https://stackoverflow.com/questions/13868821/shell-script-to-delete-directories-older-than-n-days

rnveach commented 1 year ago

CI failure is only from same eclipse-cs test. I believe this to be the same cache issue from the first deletion attempt and it will not be fixed until the cache is manually cleared just this one time. It is not an issue from the new changes, IMO.

rnveach commented 1 year ago

Just blocking again until issues settle down in main repo.