search

termux / termux-packages

A package build system for Termux.
https://termux.dev
Other
13.02k stars 2.99k forks source link

[Packaging]: Need to fix CRLF line endings files #11744

Open truboxl opened 2 years ago

truboxl commented 2 years ago

Problem description

There are many files that are improperly using CRLF line endings. Unless its Windows specific (eg: scripts/*.ps1), we should be using LF instead. We really need to have this kind of linting.

./packages/amber/Cargo.lock.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/amber/Cargo.toml.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/delve/dump.go.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/delve/gdbserver_conn.go.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/exiv2/exiv2json.cpp.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/frogcomposband/util.c.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/k2pdfopt/0006-k2pdfopt.h-dst_ocr_visibility_flags-workaround.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/k2pdfopt/0007-k2pdfoptlib-k2ocr.c-conditionally-enable-tesseract-r.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/k2pdfopt/0011-disable-tesseract.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/k2pdfopt/no-android.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/libmediainfo/01_remove_libpthread.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/libunqlite/unqlite.c.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/libzen/01_remove_libpthread.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/nsis/Source-ResourceEditor.cpp.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/plantuml/build.xml.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/portmidi/porttime-ptlinux.c.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/pypy/added.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy/fix-hardcoded-pathes.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy/fix-loaded-libs.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy/fix-not-available-functions.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy/termux-build.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy3/added.patch: unified diff output, Unicode text, UTF-8 text, with CRLF line terminators
./packages/pypy3/fix-hardcoded-pathes.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy3/fix-loaded-libs.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy3/fix-not-available-functions.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/pypy3/termux-build.patch: unified diff output, ASCII text, with CRLF line terminators
./packages/rgbds/src-extern-getopt.c.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/rinetd/src-rinetd.h.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/simh/slirp-slirp.c.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/softether-vpn/src-Cedar-Cedar.c.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/softether-vpn/src-Mayaqua-FileIO.c.orig.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/softether-vpn/src-Mayaqua-Mayaqua.c.orig.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/softether-vpn/src-Mayaqua-Network.c.orig.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./packages/softether-vpn/src-Mayaqua-Unix.c.orig.patch: unified diff output, ASCII text, with CRLF, LF line terminators
./scripts/run-docker.ps1: ASCII text, with CRLF line terminators
./scripts/update-docker.ps1: ASCII text, with CRLF line terminators
./x11-packages/lenmus/use-pulseaudio.patch: unified diff output, ASCII text, with CRLF, LF line terminators

What steps will reproduce the bug?

$ cd termux-packages
$ find . -type f -exec file "{}" \; | grep CRLF

What is the expected behavior?

No response

System information

NA

sylirre commented 2 years ago

There is a reason why certain patches have CRLF. For example the sources of k2pdfopt have CRLF line endings. If you convert patches to LF, they will be broken. Same issue with few more packages, but not all.

Consider this before converting patches into "unix-friendly" format.

Screenshot_20220829-000120

I guess these are all patches marked as with CRLF, LF line terminators.

xtkoba commented 2 years ago

Thus no problem so far.

Maybe just a matter of documentation. I have no objection to adding a sentence to Coding guideline stating that text files should avoid CRLF line endings as far as possible.

stale[bot] commented 1 year ago

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

xtkoba commented 1 year ago

Of those files listed in the description, the patches for pypy and pypy3 seem to be actually affected. I noticed that when I tried to bump them (we must check if the patches cleanly apply to the new source tree when bumping.)

For pypy:

$ for f in ~/termux-packages/packages/pypy/*.patch*; do echo; echo "[$f]"; cat $f | patch -b -p1; done

[XXX/termux-packages/packages/pypy/added.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_resource_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_sysconfigdata.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rsocket.py

[XXX/termux-packages/packages/pypy/fix-hardcoded-pathes.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/aifc.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/mailcap.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/mimetypes.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/subprocess.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/tempfile.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/uuid.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/posixpath.py

[XXX/termux-packages/packages/pypy/fix-loaded-libs.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_audioop_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_syslog_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/dbm.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file pypy/module/_multiprocessing/interp_semaphore.py
Hunk #1 succeeded at 41 (offset -2 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rposix.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rtime.py
Hunk #3 succeeded at 195 (offset 1 line).

[XXX/termux-packages/packages/pypy/fix-not-available-functions.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_pwdgrp_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/2.7/smtpd.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file pypy/module/pwd/interp_pwd.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file pypy/module/pwd/moduledef.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rvmprof/cintf.py

[XXX/termux-packages/packages/pypy/package-zip.patch]
patching file pypy/tool/release/package.py
Hunk #1 succeeded at 35 with fuzz 2.

[XXX/termux-packages/packages/pypy/termux-build.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/config/translationoption.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/translator/platform/__init__.py
Hunk #1 succeeded at 350 (offset 18 lines).
Hunk #2 succeeded at 360 (offset 18 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/translator/platform/termux.py

For pypy3:

$ for f in ~/termux-packages/packages/pypy3/*.patch*; do echo; echo "[$f]"; cat $f | patch -b -p1; done

[XXX/termux-packages/packages/pypy3/added.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_resource_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_sysconfigdata.py
Hunk #1 succeeded at 86 with fuzz 2 (offset 19 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rsocket.py
Hunk #1 succeeded at 1636 (offset 9 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_posixshmem_build.py

[XXX/termux-packages/packages/pypy3/fix-hardcoded-pathes.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/aifc.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/mailcap.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/mimetypes.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/subprocess.py
Hunk #1 succeeded at 1597 (offset 145 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/tempfile.py
Hunk #1 succeeded at 164 (offset -9 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/uuid.py
Hunk #1 succeeded at 353 (offset -9 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/posixpath.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/multiprocessing/heap.py

[XXX/termux-packages/packages/pypy3/fix-loaded-libs.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_audioop_build.py
Hunk #1 succeeded at 667 (offset 38 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_syslog_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_dbm.py
Hunk #1 succeeded at 144 (offset 21 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file pypy/module/_multiprocessing/interp_semaphore.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rposix.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rtime.py
Hunk #3 succeeded at 195 (offset 1 line).

[XXX/termux-packages/packages/pypy3/fix-not-available-functions.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/http/server.py
Hunk #1 succeeded at 1177 (offset 16 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib_pypy/_pwdgrp_build.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file lib-python/3/smtpd.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file pypy/module/pwd/interp_pwd.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file pypy/module/pwd/moduledef.py
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/rlib/rvmprof/cintf.py

[XXX/termux-packages/packages/pypy3/package-zip.patch]
patching file pypy/tool/release/package.py
Hunk #1 succeeded at 35 with fuzz 1.

[XXX/termux-packages/packages/pypy3/termux-build.patch]
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/config/translationoption.py
Hunk #2 succeeded at 291 (offset 2 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/translator/platform/__init__.py
Hunk #1 succeeded at 347 (offset 15 lines).
Hunk #2 succeeded at 357 (offset 15 lines).
(Stripping trailing CRs from patch; use --binary to disable.)
patching file rpython/translator/platform/termux.py

Note that IIRC this is the first case I encountered in my career as a package maintainer, and I believe the majority of the patches listed are false positive.

xtkoba commented 1 year ago

BTW, I didn't know that GNU patch can cleanly apply a patch with CRLF to an original file with LF. The reverse (a patch with LF to an original file with CRLF) is not possible, as mentioned above, or as we saw in commit 15c75910d8bbef6ea1b1c3beb2fc6b3a18121f5a.