search

fink / scripts

Various scripts used to maintain Fink, the web site and the binary distribution. They are not intended for use by the general public.
155 stars 363 forks source link

broken/wrong tarballs in finkmirrors #62

Open nieder opened 5 months ago

nieder commented 5 months ago

We don't have a separate repo for populating finkmirrors.net, but I think this is where the scripts for it are.

From a message on fink-users: 

When inspecting files from an rsync mirror of rsync://
[master.us.finkmirrors.net/distfiles/,](http://master.us.finkmirrors.net/distfiles/,) I noticed some of them are actually
HTML files, that seem to result from an error in an automatic download.

To check without rsync, the URLs are like
https://finkmirrors.net/distfiles/md5/gcc-3.3.6.tar.bz2.

For instance, among files updated today, there is
distfiles/md5/gcc-3.3.6.tar.bz2, that is an HTML page titled
"Download-By.Net - freeware, shareware and software downloads".
It's the first that catched my eye, as there is no obvious reason to have a
recent push of such an old version.
As a side note, it's rather troubling as this file should come from
https://ftp.gnu.org/gnu/gcc/gcc-3.3.6/gcc-3.3.6.tar.bz2, and not from a
shareware site. It's not the only one.

Other titles include "Account Suspended", "[DropCatch.com](http://dropcatch.com/)", "Go-Parts » Your
Go-To Place for Auto Parts!", etc.

These aren't just old forgotten stuff: file dates range from 2021-09-16 to
today. There are also 7 files from 2003.

On the other hand, it's not clear to me how they are referenced within fink:
* distfiles/md5/gcc-3.3.6.tar.bz2 if bad, but
distfiles/md5/6936616a967da5a0b46f1e7424a06414/gcc-3.3.6.tar.bz2 looks good
(SHA512 hash compared with the GNU version)
* same with distfiles/md5/Statistics-Regression-0.53.tar.gz and
distfiles/md5/761f4a266b2d376ebd15b2bc6038a757/Statistics-Regression-0.53.tar.gz
(SHA512 from CPAN version)
And the 4 bad files that were pushed today do not appear in
https://finkproject.org/package-updates.php.

Here is a list of 272 suspicious files (I checked all files < 2MB):

distfiles/md5/3bdbd9de06082d5de3c4d960b6d2d0ba/siag-3.5.2.tar.gz
distfiles/md5/4c65a9237fc496c5b60c2894d02d8130/libnids-1.17rc1.tar.gz
distfiles/md5/51054d260a8e2b503067deb306ce2247/unrarsrc-3.2.1.tar.gz
distfiles/md5/AnyEvent-7.17.tar.gz
distfiles/md5/AppConfig-1.71.tar.gz
distfiles/md5/BerkeleyDB-0.63.tar.gz
distfiles/md5/BibTool-2.55.tar.gz
distfiles/md5/BioPerl-1.7.7.tar.gz
distfiles/md5/Business-ISBN-3.008.tar.gz
distfiles/md5/Business-ISBN-Data-20230528.001.tar.gz
distfiles/md5/CPAN-Meta-2.150010.tar.gz
distfiles/md5/CPAN-Meta-Check-0.017.tar.gz
distfiles/md5/CPAN-Meta-Requirements-2.143.tar.gz
distfiles/md5/Capture-Tiny-0.48.tar.gz
distfiles/md5/Chemistry-3DBuilder-0.10.tar.gz
distfiles/md5/Chemistry-Bond-Find-0.23.tar.gz
distfiles/md5/Chemistry-File-MDLMol-0.21.tar.gz
distfiles/md5/Chemistry-File-QChemOut-0.10.tar.gz
distfiles/md5/Chemistry-File-VRML-0.10.tar.gz
distfiles/md5/Chemistry-MidasPattern-0.11.tar.gz
distfiles/md5/Class-Accessor-0.51.tar.gz
distfiles/md5/Class-MethodMaker-2.24.tar.gz
distfiles/md5/Clone-0.46.tar.gz
distfiles/md5/Cocoa-Growl-0.05.tar.gz
distfiles/md5/Compress-LZF-3.8.tar.gz
distfiles/md5/Compress-LZO-1.08.tar.gz
distfiles/md5/Compress-Raw-Lzma-2.101.tar.gz
distfiles/md5/Config-File-1.54.tar.gz
distfiles/md5/Config-IniFiles-2.94.tar.gz
distfiles/md5/Config-Std-0.903.tar.gz
distfiles/md5/Convert-Bencode_XS-0.06.tar.gz
distfiles/md5/Coro-6.55.tar.gz
distfiles/md5/Crypt-OpenSSL-Bignum-0.09.tar.gz
distfiles/md5/Crypt-OpenSSL-RSA-0.33.tar.gz
distfiles/md5/Crypt-OpenSSL-Random-0.15.tar.gz
distfiles/md5/Crypt-SSLeay-0.73_06.tar.gz
distfiles/md5/DBD-Pg-3.16.0.tar.gz
distfiles/md5/DBD-mysql-4.025.tar.gz
distfiles/md5/Data-MessagePack-1.02.tar.gz
distfiles/md5/DateTime-1.59.tar.gz
distfiles/md5/DateTime-1.65.tar.gz
distfiles/md5/DateTime-Format-MySQL-0.0701.tar.gz
distfiles/md5/DateTime-Format-Pg-0.16014.tar.gz
distfiles/md5/DateTime-Format-Strptime-1.79.tar.gz
distfiles/md5/DateTime-Format-W3CDTF-0.08.tar.gz
distfiles/md5/DateTime-Locale-1.39.tar.gz
distfiles/md5/DateTime-Locale-1.40.tar.gz
distfiles/md5/DateTime-TimeZone-2.60.tar.gz
distfiles/md5/DateTime-TimeZone-2.61.tar.gz
distfiles/md5/Devel-Profile-1.05.tar.gz
distfiles/md5/Email-Address-1.913.tar.gz
distfiles/md5/Event-1.26.tar.gz
distfiles/md5/ExtUtils-ParseXS-3.51.tar.gz
distfiles/md5/File-Find-Rule-0.34.tar.gz
distfiles/md5/Filesys-Virtual-Plain-0.10.tar.gz
distfiles/md5/FrameMaker-MifTree-0.075.tar.gz
distfiles/md5/Geo-Coder-Google-0.20.tar.gz
distfiles/md5/Getopt-Long-Descriptive-0.111.tar.gz
distfiles/md5/GraphicsMagick-1.3.36.tar.xz
distfiles/md5/Guard-1.023.tar.gz
distfiles/md5/HTML-Form-6.11.tar.gz
distfiles/md5/HTML-Formatter-2.16.tar.gz
distfiles/md5/HTML-HTML5-Parser-0.992.tar.gz
distfiles/md5/HTML-Parser-3.81.tar.gz
distfiles/md5/HTML-Template-2.95.tar.gz
distfiles/md5/HTTP-Message-6.44.tar.gz
distfiles/md5/HTTP-Request-AsCGI-1.2.tar.gz
distfiles/md5/Hermes-1.3.3.tar.bz2
distfiles/md5/Hook-LexWrap-0.26.tar.gz
distfiles/md5/IO-Compress-Brotli-0.004001.tar.gz
distfiles/md5/IO-Interactive-1.025.tar.gz
distfiles/md5/IO-Socket-SSL-2.083.tar.gz
distfiles/md5/IO-Tokenized-0.04.tar.gz
distfiles/md5/IPC-ShareLite-0.17.tar.gz
distfiles/md5/Imaging-1.1.7.tar.gz
distfiles/md5/LWP-Online-1.08.tar.gz
distfiles/md5/Locale-Maketext-Lexicon-1.00.tar.gz
distfiles/md5/Mac-PropertyList-SAX-0.91.tar.gz
distfiles/md5/Mac-SystemDirectory-0.14.tar.gz
distfiles/md5/Math-Combinatorics-0.08.tar.gz
distfiles/md5/Mixin-Linewise-0.111.tar.gz
distfiles/md5/Module-Refresh-0.17.tar.gz
distfiles/md5/Module-Release-2.128.tar.gz
distfiles/md5/MooseX-Role-Parameterized-1.11.tar.gz
distfiles/md5/Mouse-v2.5.10.tar.gz
distfiles/md5/Net-DAV-Server-1.305.tar.gz
distfiles/md5/Net-Delicious-1.14.tar.gz
distfiles/md5/Net-HTTP-6.23.tar.gz
distfiles/md5/Net-Ping-External-0.15.tar.gz
distfiles/md5/Net-SSH2-0.73.tar.gz
distfiles/md5/Net-Telnet-3.03.tar.gz
distfiles/md5/OLE-Storage_Lite-0.20.tar.gz
distfiles/md5/PCL-0.11.0.tar.gz
distfiles/md5/POE-1.370.tar.gz
distfiles/md5/POE-Test-Loops-1.360.tar.gz
distfiles/md5/PPI-XS-0.910.tar.gz
distfiles/md5/Package-Stash-0.40.tar.gz
distfiles/md5/Package-Stash-XS-0.30.tar.gz
distfiles/md5/Path-Tiny-0.114.tar.gz
distfiles/md5/PerlIO-gzip-0.20.tar.gz
distfiles/md5/PerlIO-utf8_strict-0.010.tar.gz
distfiles/md5/PodToHTML-0.09.tar.gz
distfiles/md5/Proc-ProcessTable-0.59.tar.gz
distfiles/md5/Rose-HTML-Objects-0.624.tar.gz
distfiles/md5/Rose-URI-1.00.tar.gz
distfiles/md5/SOAP-Lite-0.712.tar.gz
distfiles/md5/Scope-Upper-0.30.tar.gz
distfiles/md5/Set-Object-1.42.tar.gz
distfiles/md5/Statistics-Regression-0.53.tar.gz
distfiles/md5/String-Approx-3.28.tar.gz
distfiles/md5/Sys-Info-Driver-OSX-0.7961.tar.gz
distfiles/md5/Tcl-1.27.tar.gz
distfiles/md5/Template-Toolkit-2.27.tar.gz
distfiles/md5/Term-Caca-1.2.0.tar.gz
distfiles/md5/Term-Terminfo-0.08.tar.gz
distfiles/md5/Test-Compile-v3.2.2.tar.gz
distfiles/md5/Test-Differences-0.67.tar.gz
distfiles/md5/Test-Kwalitee-1.28.tar.gz
distfiles/md5/Test-Requires-0.11.tar.gz
distfiles/md5/Test-WWW-Mechanize-1.44.tar.gz
distfiles/md5/Text-Aspell-0.09.tar.gz
distfiles/md5/Text-CharWidth-0.04.tar.gz
distfiles/md5/Text-Haml-0.990118.tar.gz
distfiles/md5/Text-Kakasi-2.04.tar.gz
distfiles/md5/Tie-Array-Packed-0.13.tar.gz
distfiles/md5/Tk-Pod-0.9943.tar.gz
distfiles/md5/Tk-TableMatrix-1.26.tar.gz
distfiles/md5/URI-5.19.tar.gz
distfiles/md5/URI-Find-Rule-0.8.tar.gz
distfiles/md5/Variable-Magic-0.63.tar.gz
distfiles/md5/Version-Requirements-0.101023.tar.gz
distfiles/md5/WWW-Mechanize-1.73.tar.gz
distfiles/md5/WWW-RobotRules-6.02.tar.gz
distfiles/md5/XML-Catalog-1.03.tar.gz
distfiles/md5/XML-LibXML-2.0134.tar.gz
distfiles/md5/XML-LibXSLT-2.002001.tar.gz
distfiles/md5/XML-SAX-Machines-0.46.tar.gz
distfiles/md5/XML-XPath-1.13.tar.gz
distfiles/md5/Xerces-J-src.2.11.0.tar.gz
distfiles/md5/Xerces-J-tools.2.11.0.tar.gz
distfiles/md5/XorCurses-0.1.2.tar.bz2
distfiles/md5/a2ps-4.15.5.tar.gz
distfiles/md5/apache-tomcat-6.0.45.tar.gz
distfiles/md5/apache-tomcat-7.0.72.tar.gz
distfiles/md5/apache-tomcat-8.0.37.tar.gz
distfiles/md5/apr-util-1.6.1.tar.bz2
distfiles/md5/aspell6-es-1.11-2.tar.bz2
distfiles/md5/attica-0.2.9.tar.bz2
distfiles/md5/attica-0.4.2.tar.bz2
distfiles/md5/auctex-13.3.tar.gz
distfiles/md5/audiofile-0.2.6.tar.gz
distfiles/md5/autoconf-archive-2023.02.20.tar.xz
distfiles/md5/autogen-5.18.16.tar.xz
distfiles/md5/b86c537c5df4950af6da32e9311edb49/naim-0.11.6.4.tar.bz2
distfiles/md5/baloo-4.14.3.tar.xz
distfiles/md5/bcel-5.2-src.tar.gz
distfiles/md5/binutils-2.36.1.tar.xz
distfiles/md5/binutils-2.41.tar.xz
distfiles/md5/bison-2.7.1.tar.gz
distfiles/md5/boolean-0.46.tar.gz
distfiles/md5/c515539ba85741e33e1ab62a049b4dfe/patch-kdelibs3.darwin
distfiles/md5/c8ba9ec4674ca4a885ec83e4d5a05559/libmcrypt-2.5.7.tar.gz
distfiles/md5/commons-beanutils-1.8.3-src.tar.gz
distfiles/md5/commons-codec-1.4-src.tar.gz
distfiles/md5/commons-httpclient-3.0.1-src.tar.gz
distfiles/md5/corkscrew-2.0.tar.gz
distfiles/md5/cpan-outdated-0.32.tar.gz
distfiles/md5/cpio-2.15.tar.gz
distfiles/md5/cppi-1.18.tar.xz
distfiles/md5/ddd-3.3.12.tar.gz
distfiles/md5/diffutils-3.10.tar.xz
distfiles/md5/dxpc-3.9.2.tgz
distfiles/md5/eMovie.py
distfiles/md5/eMovie_rigimol.inp
distfiles/md5/edb-1.31.tar.gz
distfiles/md5/elisp-manual-21-2.8.tar.gz
distfiles/md5/emacs-23.4.tar.gz
distfiles/md5/emacs-24.5.tar.xz
distfiles/md5/emacs-25.3.tar.xz
distfiles/md5/emacs-26.3.tar.xz
distfiles/md5/fd7b8431ee66547b7737f7fc249ea5c8/koffice-i18n-nb-1.2.90.tar.bz2
distfiles/md5/fop-0.20.5-src.tar.gz
distfiles/md5/freeipmi-0.7.15.tar.gz
distfiles/md5/gcal-4.1.tar.xz
distfiles/md5/gcc-10.5.0.tar.xz
distfiles/md5/gcc-11.4.0.tar.xz
distfiles/md5/gcc-3.3.6.tar.bz2
distfiles/md5/gdb-10.1.tar.xz
distfiles/md5/gdb-13.2.tar.xz
distfiles/md5/gdbm-1.23.tar.gz
distfiles/md5/gengetopt-2.23.tar.xz
distfiles/md5/gimp-help-2.6.1-html-en.tar.bz2
distfiles/md5/gkII-0.4.7.tar.bz2
distfiles/md5/grep-3.11.tar.xz
distfiles/md5/groff-1.18.1.1.tar.gz
distfiles/md5/gsasl-1.10.0.tar.gz
distfiles/md5/gsasl-2.2.0.tar.gz
distfiles/md5/gsasl-2.2.1.tar.gz
distfiles/md5/gtkimageview-1.6.4.tar.gz
distfiles/md5/guile-2.0.14.tar.gz
distfiles/md5/guile-2.2.7.tar.xz
distfiles/md5/help2man-1.49.2.tar.xz
distfiles/md5/icu4c-4_8_1_1-src.tgz
distfiles/md5/icu4c-55_1-src.tgz
distfiles/md5/iwidgets4.0.1.tar.gz
distfiles/md5/jconv-0.8.1.tar.bz2
distfiles/md5/jwhois-4.0.tar.gz
distfiles/md5/kbackup-0.7.1.tar.bz2
distfiles/md5/kde-baseapps-14.12.3.tar.xz
distfiles/md5/kdeartwork-14.12.3.tar.xz
distfiles/md5/kiconedit-4.4.0.tar.bz2
distfiles/md5/lesspipe-1.83.tar.gz
distfiles/md5/libidn2-2.3.4.tar.gz
distfiles/md5/libstroke-0.5.1.tar.gz
distfiles/md5/libwww-perl-6.71.tar.gz
distfiles/md5/lincity-ng-2.0.tar.bz2
distfiles/md5/m17n-db-1.6.2.tar.gz
distfiles/md5/m17n-lib-1.6.2.tar.gz
distfiles/md5/match-simple-0.012.tar.gz
distfiles/md5/match-simple-XS-0.002.tar.gz
distfiles/md5/mathomatic-16.0.5.tar.bz2
distfiles/md5/mbedtls-1.3.22-gpl.tgz
distfiles/md5/miktex-2.9.6530.tar.xz
distfiles/md5/mpc-0.9.tar.gz
distfiles/md5/msp430mcu-20120406.tar.bz2
distfiles/md5/nettle-3.5.1.tar.gz
distfiles/md5/nettle-3.9.1.tar.gz
distfiles/md5/octave-3.6.4.tar.gz
distfiles/md5/okular-14.12.3.tar.xz
distfiles/md5/onig-5.9.2.tar.gz
distfiles/md5/oroborus_2.0.12-1.tar.gz
distfiles/md5/oxygen-icons-14.12.3.tar.xz
distfiles/md5/perl-5.16.2.tar.bz2
distfiles/md5/perl-ldap-0.68.tar.gz
distfiles/md5/physfs-1.0.0.tar.gz
distfiles/md5/pos-1.2.tgz
distfiles/md5/pspp-1.4.1.tar.gz
distfiles/md5/pvpgn-support-1.2.tar.gz
distfiles/md5/python-suds-0.4.1.tar.gz
distfiles/md5/ragel-6.7.tar.gz
distfiles/md5/readline-8.2.tar.gz
distfiles/md5/readline82-001
distfiles/md5/readline82-002
distfiles/md5/readline82-003
distfiles/md5/readline82-004
distfiles/md5/readline82-005
distfiles/md5/readline82-006
distfiles/md5/readline82-007
distfiles/md5/readline82-008
distfiles/md5/readline82-009
distfiles/md5/readline82-010
distfiles/md5/rrdtool-1.4.4.tar.gz
distfiles/md5/run-help
distfiles/md5/sed-4.9.tar.xz
distfiles/md5/smokegen-4.14.3.tar.xz
distfiles/md5/smokeqt-4.14.3.tar.xz
distfiles/md5/source-highlight-3.1.9.tar.gz
distfiles/md5/stow-2.2.2.tar.gz
distfiles/md5/subversion-1.14.0.tar.bz2
distfiles/md5/tbb43_20150424oss_src.tgz
distfiles/md5/texlive-20170524-texmf-delpdf.tar.xz
distfiles/md5/threads-2.21.tar.gz
distfiles/md5/throttled-0.5.1.tgz
distfiles/md5/tla-1.3.5.tar.gz
distfiles/md5/transfig.3.2.5d.tar.gz
distfiles/md5/vilearn-1.0.tar.gz
distfiles/md5/vobcopy-1.2.0.tar.bz2
distfiles/md5/w3c-libwww-5.4.0.tgz
distfiles/md5/w3c-libwww-5.4.2.tgz
distfiles/md5/wget-1.21.4.tar.gz
distfiles/md5/xalan-j_2_7_1-src.tar.gz
distfiles/md5/zita-convolver-1.0.0.tar.bz2
HTH,

Jean-Claude Arbaut
nieder commented 5 months ago

@thesin Not knowing how the rsync or mirror scripts work, do we delete the files and then kick the script to try to redownload again? Some files in that list are in the main 'md5' folder, while others are in a hashed subfolder, which seems weird.

@dak180

TheSin- commented 5 months ago

All the scripts are in the fink repo, I just run them ;)

see

https://github.com/fink/scripts/blob/master/distfiles/finkdist

My last commit to it was 7 years ago (and honestly don't remember doing it) but it was only to convert it from cvs for git so it could be used. I didn't actually write any of the logic.

dak180 commented 5 months ago

I could try rewriting the script in bash but if someone can work on it perl to reuse fink's logic that might be better.