[ecryptfs] `make` called via `ble-update` always rebuilds files (Linux Mint 21.2 with home directory encryption)

[juanejot]

I post this knowing it's not a ble.sh issue, but one with how it works with the latest update to a popular distro. Therefore, it is not up to akinomyoga to fix at all, but I wonder whether anyone might have some suggestions.

I made a new (wiped drive) installation of Linux Mint 21.2 the other day, and discovered once I had installed ble.sh 0.4 that the ble-update command was persistently re-installing ble.sh. I suspect the culprit may be potential changes to the beginning of .bashrc, which now reads (minus comments, plus my added ble.sh path line):

[[ $- == *i* ]] && source /home/[username]/.local/share/blesh/ble.sh --noattach

case $- in *i*) ;; *) return;; esac

While the .bashrc-ending ble.sh version/attach line appropriately reads:

[[ ${BLE_VERSION-} ]] && ble-attach

There is of course a lot more in between, as every .bashrc, even default, has. But I fear it's the default case through esac stuff that might* have changed from Linux Mint 21.1 to 21.2 ~(can't check without spinning up a VM of the older system), and may be affecting the behavior of ble-update.~ Other Debian- and Arch-based systems on which I've installed the same version of ble.sh do not exhibit this behavior, and of course their .bashrcs are all different.

I do not currently have a .blerc file, but can touch a blank one if it would help in diagnosis/alleviation. Thanks for any and all suggestions/questions!

— *UPDATE 7/21/2023 18:21 UTC: Looking at the default .bashrc from the Linux Mint 21.1 install ISO, the case through esac lines are the same as 21.2, so that’s not it. Let me know if you need to see any other lines of my otherwise default 21.2 .bashrc, and/or think it’s something unrelated.

[akinomyoga]

I've upgraded my copy of Linux Mint to 21, but the problem doesn't reproduce. Which version of ble.sh do you use? Could you paste the result of echo "$BLE_VERSION"?

[juanejot]

Thank you for your response. The output of your command is 0.4.0-devel4+9e713e96.

It did not seem to happen on a previous (since wiped) installation of Linux Mint 21.1, or 21. Only 21.2.

Also anecdotally (so I don't recommend it & it's fully to be taken with a grain of salt), it seems to happen with an LMDE "6" installation I kludged together in a VM on a completely separate machine, by changing official-package-repositories.list to "faye" rather than "elsie" (which uses what LM21.2-specific files the LM team has so far pulled over to that pre-alpha LMDE repository) and "bookworm" rather than "bullseye" (with the addition of "non-free-firmware" on those lines). I honestly didn't stay long enough in non-kludged LMDE 5 to determine whether it happened there.

[akinomyoga]

OK, I didn't know upgrading 21 doesn't automatically pull the latest minor version 21.2. I again performed the upgrade from 21 to 21.2. However, still, the problem doesn't seem to reproduce. My copy of Linux Mint is the Cinnamon edition.

[[ $- == *i* ]] && source /home/[username]/.local/share/blesh/ble.sh --noattach

case $- in *i*) ;; *) return;; esac

While the .bashrc-ending ble.sh version/attach line appropriately reads:

[[ ${BLE_VERSION-} ]] && ble-attach

There is of course a lot more in between, as every .bashrc, even default, has.

Does the problem happen when you only have two lines of source /path/to/ble.sh & ble-attach in your ~/.bashrc?

[juanejot]

Thank you so much for continuing to look into an issue which clearly is not with your software, but with how my software is configured! I wasn't sure what your last question referred to, so I (backed up my old .bashrc and) made a .bashrc with only the two source /path/to/ble.sh and ble-attach lines. The rest of bash obviously acted very boringly without conficuration, but ble.sh continued to work as expected. ble-update continued to work anomalously, ie. it continued to re-install even though there was not a new update. Because of this, I assume it's probably something else (not .bashrc) in the LM 21.2/LMDE "faye" configuration; it does not seem to happen on any other Ubuntu or Debian configuration I run, from Debian server through MX Linux through Peppermint OS through Raspberry Pi OS. Of course, it doesn't affect Arch, because the ble.sh update path for Arch is not ble-update, but through the AUR. I'm reasonably sure that my Linux Mint configuration of any other packages that interact with ble.sh is fairly vanilla: gawk is installed by default in Mint, and I installed git without any specific options. I'd be happy to check the versions & configurations of any other packages you can think of. But I'm perfectly happy if you'd rather not continue down this diversionary road from your real work; ble.sh otherwise continues to function very well, and I can put up with a re-install of it on every update check on one or two of my systems. Even better, I can post on Linux Mint forums, where someone responsible for the system's overall behavior with user-installed components might weigh in. Let me know if you'd like me to close this thread, and thank you again!

[akinomyoga]

I wasn't sure what your last question referred to, so I (backed up my old .bashrc and) made a .bashrc with only the two source /path/to/ble.sh and ble-attach lines. The rest of bash obviously acted very boringly without conficuration, but ble.sh continued to work as expected.

I'm not sure what is unclear to you, but in case you might misunderstand, I'm not requesting you to use Bash with the two-line ~/.bashrc for daily use. This is merely a test to pin down the cause of THE problem, so I expected you to revert your ~/.bashrc to the original one after you see the test result.

ble-update continued to work anomalously, ie. it continued to re-install even though there was not a new update.

OK.

Because of this, I assume it's probably something else (not .bashrc) in the LM 21.2/LMDE "faye" configuration;

Right. My previous question is merely the first question. There are still other Bash configurations that we need to check, and I planned to ask them depending on the result of the first question.

• Q2: Does THE problem reproduce with a child Bash session started within another Bash session?

$ bash        # <-- start a Bash inside a Bash session
$ ble-update  # <-- check if the problem reproduces
$ exit        # <-- end the child session

• Q3: Does the problem reproduce when you started the child Bash session in the following way?

$ INPUTRC=/dev/null bash --noprofile --norc # <-- start a child Bash without any configuration
$ source /path/to/ble.sh                    # <-- source ble.sh
$ ble-update                                # <-- check if the problem reproduces
$ exit                                      # <-- end the child session

• Q4: What are the results of the following commands in a ble session?

$ echo "$_ble_base"
$ ls -ld "$_ble_base"
$ echo "$_ble_base_repository"
$ ls -ld "$_ble_base_repository"
$ df -h "$_ble_base"
$ df -h "$_ble_base_repository"

[juanejot]

Thanks for your reply. I appear to have guessed correctly on your first point; hopefully that was a diagnostic help. Answers to your other questions (with privacy edits):

Q2: The problem reproduces. Q3: The problem reproduces (using, as my second command in the child shell, source /home/[username]/.local/share/blesh/ble.sh).

Q4:

$ echo "$_ble_base" /home/[username]/.local/share/blesh

$ ls -ld "$_ble_base_repository" drwxrwxr-x 12 [username] [username] 4096 Jul 23 23:25 /home/[username]/ble.sh

$ df -h "$_ble_base" Filesystem Size Used Avail Use% Mounted on /home/[username]/.Private 455G 34G 398G 8% /home/[username]

$ df -h "$_ble_base_repository" Filesystem Size Used Avail Use% Mounted on /home/[username]/.Private 455G 34G 398G 8% /home/[username]

Anecdotally, I went back and spun up a completely vanilla (but updated with no changes to official_packages_repositories.list, rather than fresh from the ISO) LMDE5 VM, and it too had the problem. I'd be happy to wipe it, install vanilla LMDE5 from the ISO, not update before I install ble.sh, and see if that has the problem; it may be a recent, backported, LM-versions-wide configuration change, though it is weird that it's not affecting you while it is me on a couple of otherwise vanilla systems. Something about locale, maybe?

[akinomyoga]

Thank you for checking the results.

Q4:

$ echo "$_ble_base" /home/[username]/.local/share/blesh

$ ls -ld "$_ble_base_repository" drwxrwxr-x 12 [username] [username] 4096 Jul 23 23:25 /home/[username]/ble.sh

Two commands seem to be skipped. What is the output of ls -ld "$_ble_base"? I need to check the permission.

Anecdotally, I went back and spun up a completely vanilla (but updated with no changes to official_packages_repositories.list, rather than fresh from the ISO) LMDE5 VM, and it too had the problem. I'd be happy to wipe it, install vanilla LMDE5 from the ISO, not update before I install ble.sh, and see if that has the problem;

Thank you for checking that it reproduces with the plain LMDE5. Then, I'll probably have to set up LMDE5 in VM and see if it reproduces.

Something about locale, maybe?

• Q5: What is the output of the following command in a ble session?

$ ble/widget/display-shell-version

[juanejot]

Q4 continued: Oops, sorry I hadn't copied over the results of that command. Luckily I could scroll back up to it; here it is:

$ ls -ld "$_ble_base" drwxr-xr-x 6 [username] [username] 4096 Jul 23 23:25 /home/[username]/.local/share/blesh

Q5:

$ ble/widget/display-shell-version GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu) [Linux Mint 21.2] ble.sh, version 0.4.0-devel4+9e713e96 (noarch) [git 2.34.1, GNU Make 4.3, GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0, GNU MP 6.2.1)] bash-completion, version 2.11 (hash:[Do you need this? Can repost if so.], 77071 bytes) (noarch) locale: LANG=en_US.UTF-8 terminal: TERM=xterm-256color wcwidth=14.0-west/15.0-2+ri, vte:6800 (65;6800;1)

On the LMDE5 VM straight-from-the-ISO front, it did need at least an apt update before it would allow me to install git, but I did not do any apt [full-/dist-]upgradeing. The problem persisted.

[akinomyoga]

I now tried LMDE5 Live ISO Image, but it still doesn't reproduce. What are the exact commands that you used to check the behavior? The following are the commands that I tested in the Live LMDE5:

$ sudo apt update
$ sudo apt install git
$ git config --global pull.ff only # to suppress warnings on ble-update
$ cd
$ git clone https://github.com/akinomyoga/ble.sh.git
$ cd ble.sh
$ make -j
$ make install
$ bash --norc
$ source /home/mint/.local/share/blesh/ble.sh --norc
$ ble-update
$ ble-update
$ exit

I'll proceed to install it on the actual VM disk.

[akinomyoga]

I have a question about the installation configuration of LMDE5. Do you encrypt your home folder?

Are there any other special instructions for the installation to reproduce your environment as much as possible?

Edit: Do you use LVM? Do you encrypt the system? I'm asking this because I'm now suspecting that the problem is related to the filesystem.

[juanejot]

After having make installed, editing my .bashrc & running ble-update, I tried all three git config options, and while they made git's intro configuration text go away, it did not stop the cp -p and ln -sf persistent behavior of ble-update after it had printed Already up to date. I also have not used the -j option for make, and since I had already updated my .bashrc, I did not on the VM source ble.sh, --norc or not, as we tested on my main install, above.

In answer to your most recent question, I do encrypt my home folder, and it occurs to me that that selection is recent, likely with my latest installs of LM21.2 (daily driver) & LMDE (VMs, both 5 & "6") (while I've always used LVM and its option to encrypt the entire drive). Moreover, I don't believe that most other systems I've got ble.sh installed on have an encrypted home folder (the machine with the problem is a laptop that might leave the house; hence the interest in its unattended security).

Re-reading (sorry), no; I cannot think of any other special instructions to reproduce my environment except some that are different between laptop & VMs (specifically, a WiFi driver the VMs don't use). Ble.sh is often the first thing I install on the system, and other than Clement Tsang's "bottom" system monitor utility or Duplicati for offsite backup, it's one of the few that doesn't come from an LM/Ubuntu/Debian repository (I stopped using PPAs on that machine a while ago).

If you agree, I think we have our culprit: home folder encryption.

[akinomyoga]

I'm now installing LMDE5 in a VM.

it did not stop the cp -p and ln -sf persistent behavior of ble-update after it had printed Already up to date.

Does it mean that the update proceeds even after the message "Already up to date" is printed? Hmm, could you paste the full output of ble-update when it shouldn't update yet does update?

[juanejot]

Sure! Though forgive me if I don't go through the effort of cosmetically "``"ing every line. 🤣

$ ble-update
cd into /home/[username]/ble.sh...
Already up to date.
cp -p lib/keymap.emacs.sh out/lib/keymap.emacs.sh
cp -p lib/keymap.vi.sh out/lib/keymap.vi.sh
cp -p lib/keymap.vi_digraph.sh out/lib/keymap.vi_digraph.sh
cp -p lib/keymap.vi_digraph.txt out/lib/keymap.vi_digraph.txt
cp -p lib/keymap.vi_test.sh out/lib/keymap.vi_test.sh
cp -p lib/init-term.sh out/lib/init-term.sh
cp -p lib/init-bind.sh out/lib/init-bind.sh
cp -p lib/init-cmap.sh out/lib/init-cmap.sh
cp -p lib/core-complete.sh out/lib/core-complete.sh
cp -p lib/core-test.sh out/lib/core-test.sh
cp -p lib/core-cmdspec.sh out/lib/core-cmdspec.sh
cp -p lib/core-debug.sh out/lib/core-debug.sh
cp -p lib/core-edit.ignoreeof-messages.txt out/lib/core-edit.ignoreeof-messages.txt
cp -p lib/core-decode.emacs-rlfunc.txt out/lib/core-decode.emacs-rlfunc.txt
cp -p lib/core-decode.vi_imap-rlfunc.txt out/lib/core-decode.vi_imap-rlfunc.txt
cp -p lib/core-decode.vi_nmap-rlfunc.txt out/lib/core-decode.vi_nmap-rlfunc.txt
cp -p lib/vim-surround.sh out/lib/vim-surround.sh
cp -p lib/vim-arpeggio.sh out/lib/vim-arpeggio.sh
cp -p lib/vim-airline.sh out/lib/vim-airline.sh
cp -p lib/test-bash.sh out/lib/test-bash.sh
cp -p lib/test-main.sh out/lib/test-main.sh
cp -p lib/test-util.sh out/lib/test-util.sh
cp -p lib/test-decode.sh out/lib/test-decode.sh
cp -p lib/test-edit.sh out/lib/test-edit.sh
cp -p lib/test-syntax.sh out/lib/test-syntax.sh
cp -p lib/test-complete.sh out/lib/test-complete.sh
cp -p lib/util.bgproc.sh out/lib/util.bgproc.sh
cp -p contrib/colorglass.bash out/contrib/colorglass.bash
cp -p contrib/histdb.bash out/contrib/histdb.bash
cp -p contrib/prompt-defer.bash out/contrib/prompt-defer.bash
cp -p contrib/prompt-elapsed.bash out/contrib/prompt-elapsed.bash
cp -p contrib/prompt-git.bash out/contrib/prompt-git.bash
cp -p contrib/prompt-vim-mode.bash out/contrib/prompt-vim-mode.bash
cp -p contrib/airline/atomic.bash out/contrib/airline/atomic.bash
cp -p contrib/airline/dark.bash out/contrib/airline/dark.bash
cp -p contrib/airline/dark_minimal.bash out/contrib/airline/dark_minimal.bash
cp -p contrib/airline/google_dark.bash out/contrib/airline/google_dark.bash
cp -p contrib/airline/google_light.bash out/contrib/airline/google_light.bash
cp -p contrib/airline/landscape.bash out/contrib/airline/landscape.bash
cp -p contrib/airline/light.bash out/contrib/airline/light.bash
cp -p contrib/airline/minimalist.bash out/contrib/airline/minimalist.bash
cp -p contrib/airline/monochrome.bash out/contrib/airline/monochrome.bash
cp -p contrib/airline/solarized.bash out/contrib/airline/solarized.bash
cp -p contrib/airline/solarized_flood.bash out/contrib/airline/solarized_flood.bash
cp -p contrib/airline/term.bash out/contrib/airline/term.bash
cp -p contrib/airline/term_light.bash out/contrib/airline/term_light.bash
cp -p contrib/airline/transparent.bash out/contrib/airline/transparent.bash
cp -p contrib/airline/xtermlight.bash out/contrib/airline/xtermlight.bash
cp -p contrib/config/execmark.bash out/contrib/config/execmark.bash
cp -p contrib/config/github265-prompt-path-level-colors.bash out/contrib/config/github265-prompt-path-level-colors.bash
cp -p contrib/config/github288-filter-sabbrev-completion.bash out/contrib/config/github288-filter-sabbrev-completion.bash
cp -p contrib/config/github296-named-execmark.bash out/contrib/config/github296-named-execmark.bash
cp -p contrib/config/github302-perlre-server.bash out/contrib/config/github302-perlre-server.bash
cp -p contrib/integration/bash-preexec.bash out/contrib/integration/bash-preexec.bash
cp -p contrib/integration/fzf-completion.bash out/contrib/integration/fzf-completion.bash
cp -p contrib/integration/fzf-git.bash out/contrib/integration/fzf-git.bash
cp -p contrib/integration/fzf-initialize.bash out/contrib/integration/fzf-initialize.bash
cp -p contrib/integration/fzf-key-bindings.bash out/contrib/integration/fzf-key-bindings.bash
cp -p contrib/integration/nix-completion.bash out/contrib/integration/nix-completion.bash
cp -p contrib/integration/zoxide.bash out/contrib/integration/zoxide.bash
ln -sf integration/bash-preexec.bash out/contrib/bash-preexec.bash
ln -sf integration/fzf-completion.bash out/contrib/fzf-completion.bash
ln -sf integration/fzf-git.bash out/contrib/fzf-git.bash
ln -sf integration/fzf-initialize.bash out/contrib/fzf-initialize.bash
ln -sf integration/fzf-key-bindings.bash out/contrib/fzf-key-bindings.bash
cp -p README.md out/doc/README.md
cp -p README-ja_JP.md out/doc/README-ja_JP.md
cp -p LICENSE.md out/doc/LICENSE.md
cp -p docs/CONTRIBUTING.md out/doc/CONTRIBUTING.md
cp -p docs/ChangeLog.md out/doc/ChangeLog.md
cp -p docs/Release.md out/doc/Release.md
cp -p contrib/LICENSE out/doc/contrib/LICENSE
cp -p contrib/README-ja.md out/doc/contrib/README-ja.md
cp -p contrib/README.md out/doc/contrib/README.md
[ble: reload]

(Thanks; did not realize 3 in a line would work for a whole block of text; was afraid that would take some major sed even to set it up!)

[akinomyoga]

Sure! Though forgive me if I don't go through the effort of cosmetically "``"ing every line. 🤣

You can actually use ``` ... ``` to quote a code block. You can also specify the language for syntax highlighting.

```bash
echo 'This is a code block.'
echo 'Multiple lines can be put.'
```

As far as I see the provided log, there are two strange behaviors. One is that ble.sh is built even when there are no updates from the upstream GitHub. The other is that the build is running but the installation doesn't seem to be performed.

[juanejot]

Cool, thanks!

Huh. So to ensure safe updating of ble.sh, is it best not to encrypt one's home folder, maybe? I've not had trouble with LVM full-disk encryption messing with ble.sh in the past, just since checking "Encrypt my home folder" on the new LM 21.2 install, and on the LMDE VM installs.

[akinomyoga]

OK, it reproduces in a VM. It seems cp doesn't preserve subsecond timestamps with this filesystem, which confuses make.

murase@lmde5:~/ble.sh$ stat lib/keymap.emacs.sh out/lib/keymap.emacs.sh 
  File: lib/keymap.emacs.sh
  Size: 10020       Blocks: 40         IO Block: 4096   regular file
Device: 23h/35d Inode: 924687      Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  murase)   Gid: ( 1000/  murase)
Access: 2023-07-26 09:53:12.553717259 +0900
Modify: 2023-07-26 09:52:47.285268409 +0900
Change: 2023-07-26 09:52:47.285268409 +0900
 Birth: -
  File: out/lib/keymap.emacs.sh
  Size: 10020       Blocks: 40         IO Block: 4096   regular file
Device: 23h/35d Inode: 1051495     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  murase)   Gid: ( 1000/  murase)
Access: 2023-07-26 10:12:55.529726827 +0900
Modify: 2023-07-26 09:52:47.000000000 +0900 # ■ <-- See this line ■
Change: 2023-07-26 10:01:10.360393590 +0900
 Birth: -

Because of this, the build of ble.sh runs every time one runs make. Actually, the installation is not performed, which is expected.

Huh. So to ensure safe updating of ble.sh, is it best not to encrypt one's home folder, maybe?

If there is a reason, you should encrypt it. Rather, I'll think about the adjustments of the build instructions at the ble.sh side.

[juanejot]

Thank you!

And I can confirm: spinning up a new VM with LM 21.2 and LVM full disk encryption (but not home folder encryption) allows ble-update to exhibit normal behavior. Further, making a new test user without home folder encryption on the LMDE "6" VM (sorry, I've since deleted the vanilla LMDE 5 ones) also returns ble-update to normal behavior.

As you suggest for security, I'll look forward to how your possible adjustments to the build instructions may work in an encrypted home folder's filesystem. Thanks again for all your help!

[akinomyoga]

I've been searching for the related issues caused by the filesystem (ecryptfs). Several similar problems caused by the ecryptfs filesystem seem to have already been reported [1,2]. All are related to the file syncing based on the timestamps.

• [1] Rclone sync does not properly preserve modification times when syncing to ecryptfs mounts - Suspected Bug - rclone forum
• [2] ecryptfs causes Cargo to rebuild when it shouldn't · Issue #7775 · rust-lang/cargo

Then, it seems to be reported to the upstream in 2020-08 and confirmed to be a bug of ecryptfs in 2021-08 [3]. However, there seem to be no updates since then.

• [3] Bug #1890486 “utimensat truncates timestamp to whole second gran...” : Bugs : eCryptfs

I've also searched the mailing list of ecryptfs for any related discussions. There seem to be some movements related to the implementation of the granularities of the timestamps [4-6], but these seem to be unrelated to the present problem.

• [4] [PATCH v6 0/7] fs: implement multigrain timestamps — Ecryptfs
• [5] [PATCH v5 0/8] fs: implement multigrain timestamps — Ecryptfs
• [6] Ecryptfs over sshfs and timestamps — Ecryptfs

There was an old bug report arising in a similar situation [7], but it seems to be caused by another different bug, which was already fixed.

• [7] 33372 – ecrypts does not preserve correctly timestamps for "cp -p"

[akinomyoga]

I've left comments in the upstream bug tracker of ecryptfs (Ref. [3] of the above reply) and voted for bug #1890486.

[akinomyoga]

I pushed a fix 969a763e to master. I gave up preserving the timestamps in copying the files.

[juanejot]

Thank you for all your hard work on this! Unfortunately, with version 0.4.0-devel4+969a763e, the issue persists. I rebooted just to make sure, and it's still doing it.

A workaround could be to make sure to make install outside the encrypted home folder (a couple of such options being on your ble.sh's main Github page). I may go test that.

[akinomyoga]

Ah, thank you for testing. I forgot to update the Makefile of contrib repository. I should have tested it with LMDE5 in which I reproduced the problem. I pushed another fix to master and also tested it with LMDE5 with encrypted home directory.

[juanejot]

This latest one functioned as expected (outputting nothing after "Already up to date.") while installed within the encrypted home folder on LM 21.2, thank you! Anecdotally, it also worked when installed in /usr/share, but with the warning every time ble-update was run, that another user (root) owned the blesh folder. This is cleaner, and provided that not using timestamps is not a concern going forward, I'm satisfied. Thanks!

[akinomyoga]

Thank you for the confirmation of the fix.

but with the warning every time ble-update was run, that another user (root) owned the blesh folder.

This is the intended behavior.