search

rrthomas / mmv

Other
50 stars 7 forks source link

lowercasing folder issues #13

Closed EdwinKM closed 1 year ago

EdwinKM commented 1 year ago

What i actually tried to accomplish is simply to lowercase all (only) files. And without recursion if possible. But i notice some strange behavior for folders? Suse seems to crash the second run. Alpine seems to fail (exit code 2) but it actually does not have to rename anything the second run.

Suse

test@linux-5q3o:~/TEMP/mmv> mmv --version mmv 2.3

test@linux-5q3o:/TEMP/mmv> mkdir FOLDER test@linux-5q3o:/TEMP/mmv> mmv -vt "*" "#l1" FOLDER -> folder : done

test@linux-5q3o:~/TEMP/mmv> mmv -vt "*" "#l1" buffer overflow detected : terminated Aborted (core dumped)

Alpine

/tmp $ mmv --version mmv 2.3

/tmp $ mkdir FOLDER /tmp $ mmv -vt "*" "#l1" FOLDER -> folder : done

/tmp $ mmv -vt "*" "#l1" folder -> folder/folder has failed. The following left undone: folder -> folder/folder

rrthomas commented 1 year ago

Thanks, I'll look at this ASAP. My initial impression is that there are two problems: 1) a buffer overrun needs fixing, and 2) mmv shouldn't try to rename a folder into itself. (This is the first bit of evidence about why that functionality wasn't implemented in the past!)

EdwinKM commented 1 year ago

No worries, things can get complicated very quickly. People have different use cases. Mine are modest as stated.

Not sure about your remarks about folder renaming. Is this recently introduced? In that case we should consider a new flag because of it changes what a person can expect of the tool.

EdwinKM commented 1 year ago

@rrthomas is this project dead? Unfortunately i do not have c-knowledge. But it the tool i unsupported i will script something myself for my specific usecase.

rrthomas commented 1 year ago

I'm sorry, I have not found time to fix your bug. In fact, I did something rather more ambitious: I rewrote the entire project in Python (see the python branch). However, I have not found time to finish this either. I will try to look into a fix for this bug soon (on the original C branch).

rrthomas commented 1 year ago

I cannot reproduce the SuSE buffer overflow when compiling current master HEAD with ASAN.

rrthomas commented 1 year ago

By the way, since you're just renaming files in one directory, you can use -r, and it will work already.

rrthomas commented 1 year ago

I've pushed a simple fix that certainly works in this case, and I think should fix the general problem. I would be delighted if you could test current git master; if not, I shall in any case make a new release shortly.

EdwinKM commented 1 year ago

Seems updated by the package manager. Do you still want to investigate Suse? In that case can you guide me how to enable the debug logging.

test@linux-5q3o:~/TEMP/mmv> mmv --version mmv 2.4

Copyright (c) 2023 Reuben Thomas rrt@sc3d.org. Copyright (c) 1990 Vladimir Lanin. Licence GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. test@linux-5q3o:~/TEMP/mmv> mkdir FOLDER test@linux-5q3o:~/TEMP/mmv> mmv -vt "" "#l1" FOLDER -> folder : done test@linux-5q3o:~/TEMP/mmv> mmv -vt "" "#l1" buffer overflow detected : terminated Aborted (core dumped)

EdwinKM commented 1 year ago

re-validated the problem with mmv 2.4 on Alpine

2023-07-18 19:21:06.729639+00:00mmv 2.4 2023-07-18 19:21:06.729719+00:002023-07-18T19:21:06.729719816Z 2023-07-18 19:21:06.729742+00:00Copyright (c) 2023 Reuben Thomas rrt@sc3d.org. 2023-07-18 19:21:06.729760+00:00Copyright (c) 1990 Vladimir Lanin. 2023-07-18 19:21:06.729813+00:00Licence GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html. 2023-07-18 19:21:06.729848+00:00This is free software: you are free to change and redistribute it. 2023-07-18 19:21:06.729866+00:00There is NO WARRANTY, to the extent permitted by law. 2023-07-18 19:21:06.731896+00:00rename 1st 2023-07-18 19:21:06.733932+00:00FOLDER -> folder : done 2023-07-18 19:21:06.734238+00:00rename 2nd 2023-07-18 19:21:06.736250+00:00The following left undone: 2023-07-18 19:21:06.736303+00:00folder -> folder/folder 2023-07-18 19:21:06.736280+00:00folder -> folder/folder has failed. 2023-07-18 19:21:06.736538+00:00Sleeping forever ...

Dockerfile:

FROM alpine:3.18.2
ENV UID=1001
ENV GID=1001
ENV UID_NAME=uid1001
ENV GID_NAME=gid1001
ENV TZ=Europe/Amsterdam
ENV MMV_VERSION=2.4

RUN \
  echo "**** install build packages ****" && \
  apk add -U --update --no-cache --virtual=build-dependencies \
    build-base \
    pkgconfig \
    patch \
    screen \
    g++ \
    gc-dev \
    make && \
  echo "**** install packages ****" && \
  apk add -U --update --no-cache \
    bash \
    gc \
    git \
    wget \
    curl \
    tzdata && \
  addgroup --gid ${GID} ${GID_NAME} && \
  adduser --ingroup ${GID_NAME} --uid ${UID} --disabled-password ${UID_NAME} && \
  mkdir '/empty' && chown ${UID}:${GID} '/empty' && \
  cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime && \
  echo "**** install mmv from source ****" && \
  mkdir /tmp/mmv && \
  curl -o \
    /tmp/mmv.tar.gz -L \
    "https://github.com/rrthomas/mmv/releases/download/v${MMV_VERSION}/mmv-${MMV_VERSION}.tar.gz" && \
  tar xf \
    /tmp/mmv.tar.gz -C \
    /tmp/mmv --strip-components=1 && \
  cd /tmp/mmv && \
  ./configure && \
  make && \
  make install

COPY --chown=${UID}:${GID} scripts_bin /scripts_bin
USER ${UID_NAME}:${GID_NAME}
CMD /scripts_bin/infinite.sh

# docker build -f Dockerfile -t mmv-compile . && docker save mmv-compile:latest > mmv-compile.tar && scp mmv-compile.tar root@san: ; ssh root@san "docker load < /root/mmv-compile.tar"

Script (infinite.sh)

#!/usr/bin/env bash

cd
mmv --version
mkdir FOLDER
echo "rename 1st"
mmv -vt "*" "#l1"

echo "rename 2nd"
mmv -vt "*" "#l1"

echo "Sleeping forever ..."
sleep infinity
echo "Line after infinity (should not happen): $(date)"
EdwinKM commented 1 year ago

Fix seems to work

2023-07-18 21:25:20.537380+00:00mmv 2.5
2023-07-18 21:25:20.537423+00:002023-07-18T21:25:20.537423713Z
2023-07-18 21:25:20.537433+00:00Copyright (c) 2023 Reuben Thomas <rrt@sc3d.org>.
2023-07-18 21:25:20.537441+00:00Copyright (c) 1990 Vladimir Lanin.
2023-07-18 21:25:20.537450+00:00Licence GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
2023-07-18 21:25:20.537464+00:00This is free software: you are free to change and redistribute it.
2023-07-18 21:25:20.537473+00:00There is NO WARRANTY, to the extent permitted by law.
2023-07-18 21:25:20.539036+00:00rename 1st
2023-07-18 21:25:20.539825+00:00FOLDER -> folder : done
2023-07-18 21:25:20.540137+00:00rename 2nd
2023-07-18 21:25:20.540941+00:00folder =^ folder : done
2023-07-18 21:25:20.541083+00:00Sleeping forever ...

Dockerfile

FROM alpine:3.18.2
ENV UID=1001
ENV GID=1001
ENV UID_NAME=uid1001
ENV GID_NAME=gid1001
ENV TZ=Europe/Amsterdam

RUN \
  echo "**** install build packages ****" && \
  apk add -U --update --no-cache --virtual=build-dependencies \
    build-base \
    pkgconfig \
    pkgconf \
    patch \
    screen \
    g++ \
    gc-dev \
    m4 \
    gpg \
    gengetopt \
    help2man \
    perl \
    autoconf \
    libtool \
    automake \
    coreutils \
    make && \
  echo "**** install packages ****" && \
  apk add -U --update --no-cache \
    bash \
    gc \
    git \
    wget \
    curl \
    tzdata && \
  addgroup --gid ${GID} ${GID_NAME} && \
  adduser --ingroup ${GID_NAME} --uid ${UID} --disabled-password ${UID_NAME} && \
  mkdir '/empty' && chown ${UID}:${GID} '/empty' && \
  cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime && \
  echo "**** install mmv from source ****" && \
  cd /tmp/ && \
  git clone https://github.com/rrthomas/mmv.git && \
  cd /tmp/mmv && \
  echo "bootstrap..." && \
  ./bootstrap && \
  echo "configure..." && \
  ./configure && \
  make && \
  make install

COPY --chown=${UID}:${GID} scripts_bin /scripts_bin
USER ${UID_NAME}:${GID_NAME}
#USER root
CMD /scripts_bin/infinite.sh

# docker build -f Dockerfile -t mmv-git . && docker save mmv-git:latest > mmv-git.tar && scp mmv-git.tar root@san: ; ssh root@san "docker load < /root/mmv-git.tar"
rrthomas commented 1 year ago

Thanks very much for the updates. Please can you let me know whether 2.5 also fixes the buffer overflow?

EdwinKM commented 1 year ago

bufferoverflow seems fixed also :)

test@linux-5q3o:~/TEMP/mmv_TEST> rmdir folder; mkdir FOLDER
test@linux-5q3o:~/TEMP/mmv_TEST> mmv -vt "*" "#l1"
FOLDER -> folder : done
test@linux-5q3o:~/TEMP/mmv_TEST> mmv -vt "*" "#l1"
*** buffer overflow detected ***: terminated
Aborted (core dumped)
test@linux-5q3o:~/TEMP/mmv_TEST> rmdir folder; mkdir FOLDER
test@linux-5q3o:~/TEMP/mmv_TEST> /home/test/TEMP/mmv/mmv/mmv -vt "*" "#l1"
FOLDER -> folder : done
test@linux-5q3o:~/TEMP/mmv_TEST> /home/test/TEMP/mmv/mmv/mmv -vt "*" "#l1"
folder =^ folder : done

building

sudo zypper install gengetopt help2man pkgconf-pkg-config gc-devel
git clone https://github.com/rrthomas/mmv.git
cd mmv
./bootstrap
./configure
make
/home/test/TEMP/mmv/mmv/mmv --version
rrthomas commented 1 year ago

Great, thanks for testing!