“You need to start and close Firefox again” will always be shown once this script has been run once

[mirabilos]

Description

Once the script was run once on any given machine, it will always output…

Firefox install path found in '/usr/lib/firefox-esr'.
Error: You need to start and close Firefox again to apply the changes before running this script.

… and terminate abnormally.

• [x] I made sure that the issue still occurs in the latest commit of the script.
• [x] I made sure to apply the changes made by Firefox or the script properly.
• Firefox version and build ID: 78.6.0esr (64-bit)
• Operating system: Debian GNU/Linux amd64
• Zip and UnZip distribution and version: unzip 6.0-25, zip 3.0-11+b1

Steps to reproduce the issue

I had run this once manually, some time ago.

Firefox was updated, so I thought to run it again: I cloned it and ran sudo bash F* and always got that message.

This is the terminal output

Firefox install path found in '/usr/lib/firefox-esr'.
Error: You need to start and close Firefox again to apply the changes before running this script.

I expected this to happen instead:

That it works

Analysis

The script, once run, creates the file /usr/lib/firefox-esr/browser/.purgecaches for some, probably valid, reason. It does, however, seem to expect the file to vanish before the script is run ever again, perhaps by Firefox itself. That is, Firefox has no write permissions on that file or the enclosing directory (as it should be) and therefore cannot remove that file.

Manual intervention is required:

$ sudo rm /usr/lib/firefox-esr/browser/.purgecaches

After having run this, the script works again… exactly once.

This is a conceptional mistake, and I don’t know how to best fix it. (Incidentally, if multiple users use Firefox on that machine, the file must be kept around until all users have run it once…) Perhaps there is no good fix for it.

[SebastianSimon]

“for some, probably valid, reason” — Yes, the cache needs to be cleared for Firefox to use the newly changed omni.ja.

“Firefox was updated, so I thought to run it again” — Normally, if the .purgecaches file is still there, it means that Firefox was never started since, because Firefox is supposed to clear this file as soon as it is started.

Creating the .purgecaches file is still the ideal option. If Firefox cannot remove it, this is a problem.¹ I can’t use a MOZ_PURGE_CACHES environment variable, because Firefox isn’t guaranteed to be started within the same execution context as the script. I can’t use a --purgecaches option, because that would require starting Firefox from terminal, or editing the .desktop file, which is difficult to find automatically.

“Firefox install path found in '/usr/lib/firefox-esr'.” — The script is supposed to find firefox and firefox-esr and let you choose which one to fix. Are you sure you’re updating and applying the script to the correct Firefox install? You have both Firefox ESR and regular Firefox, right?

“Firefox has no write permissions on that file” — Which ownership does the .purgecaches file have in the end? Does it belong to user root, group root? Even if I manually create the file via sudo touch .purgecaches, it works fine for me.

Which ownership does the containing directory have? Is Firefox able to clear the file after changing the .purgecaches ownership with something like chown someotheruser:someothergroup .purgecaches?

I wonder if Firefox doesn’t have write permissions because you update it via some package manager. For me, the script works fine, probably because Firefox Nightly has its own updater, and is not updated through a package manager.

“if multiple users use Firefox on that machine, the file must be kept around until all users have run it once” — Why? The script fixes a specific Firefox installation, not a user profile or anything related to users.

---

¹: I’m also not sure if Firefox actually clears its cache if it cannot remove the file…

[SebastianSimon]

I could, of course, let the script bypass this check, maybe via a --force option, or some condition, or something. But what will the condition be …?

[mirabilos]

Sebastian Simon dixit:

“for some, probably valid, reason” — Yes, the cache needs to be _cleared for Firefox to use the newly changed omni.ja.

Yeah, I guess so.

“Firefox was updated, so I thought to run it again” — Normally, if _the .purgecaches file is still there, it means that Firefox was _never started since, because Firefox is supposed to clear this file as _soon as it is started.

It doesn’t have write permissions there, though.

Creating the .purgecaches file is still the ideal option. If Firefox cannot remove it, this is a problem. I can’t use a MOZ_PURGE_CACHES environment variable, because Firefox isn’t guaranteed to be started within the same execution context as the script. I can’t use a --purgecaches option, because that would require starting Firefox from terminal, or editing the .desktop file, which is difficult to find automatically.

The .desktop file is not used here either, I run it from terminal always.

Yeah, tricky.

Perhaps interactively ask the user to run it once with MOZ_PURGE_CACHES in the environment (with the correct value which should be shown) if the script is run as root (because that’s when this normally happens)…

“Firefox install path found in '/usr/lib/firefox-esr'.” — The _script is supposed to find firefox and firefox-esr and let you _choose which one to fix. Are you sure you’re updating and applying the _script to the correct Firefox install? You have both Firefox ESR and _regular Firefox, right?

No, I only have ESR.

“Firefox has no write permissions on that file” — Which ownership does the .purgecaches file have in the end? Does it belong to user root, group root? Even if I manually create the file via sudo _touch .purgecaches, it works fine for me.

The ownership of the file does not matter much considering the directory is not writable by the regular user that runs Firefox, so it cannot unlink the dirent.

I wonder if Firefox doesn’t have write permissions because you update it via some package manager.

Eh of course this is the case. Firefox is installed from a Debian package, as this is the only safe way to install it at all.

For me, the script works fine, probably because Firefox Nightly has its own updater, and is not updated through a package manager.

Uaaaah shudder that’s untrusted and not reliable system operation. (It’s probably also against the IT security policies at work.)

_“if multiple users use Firefox on that machine, the file must be kept around until all users have run it once” — Why? The script fixes a _specific Firefox installation, not a user profile or anything related _to users.

If so, why can’t the script purge the caches of that specific Firefox installation by itself then?

I assume the user has to run Firefox once to purge the caches because they are part of the per-user setup? (Eh maybe even once per profile?)

---

Cheapest solution: leave the .purgecaches file there and just don’t complain; this way it will purge caches on every startup but as we have seen it will work anyway.

[mirabilos]

Sebastian Simon dixit:

I could, of course, let the script bypass this check, maybe via a --force option, or some condition, or something. But what will the condition be …?

Being run as root…

test x"$(id -u)" = x"0"

… and the Firefox directory in question is under /usr or so?

[SebastianSimon]

Alright, this has been really helpful. The docs say “Gecko (layout engine) has a JavaScript cache, which is not reset on startup, this clears it.”. I got the suspicion that it’s in ~/.mozilla/firefox/⟨profileid⟩.⟨profiletype⟩/startupCache, and it turns out, it is!

I’m also not sure if Firefox actually clears its cache if it cannot remove the file…

this way it will purge caches on every startup but as we have seen it will work anyway.

And now I’ve confirmed: if the browser directory belongs to root, and .purgecaches is still there, Firefox actually does clear the cache on startup. Nice! Thanks for pointing me in that direction!

But if the .purgecaches doesn’t persist, deleting ~/.mozilla/firefox/⟨profileid⟩.⟨profiletype⟩/startupCache works, too, but which ⟨profileid⟩.⟨profiletype⟩ do I pick? All of them? The most recently updated one? Ask the user? Provide a command line option? Possibly parse the ~/.mozilla/firefox/profiles.ini and ~/.mozilla/firefox/installs.ini to check which profiles are used by the newly fixed Firefox install by default?

I bet iterating through all users’ Firefox profiles would not be appropriate.

---

I could base the check for .purgecaches on the condition [[ "$(stat --format='%u' /usr/share/firefox/browser/)" == '0' ]], which tests if the directory belongs to root. But I think, an rm -r on a startupCache directory of the current user is more appropriate. Who cares about these cache files anyway?

A cache purge, be it manually or via script, is definitely necessary. No matter how many times Firefox is restarted, the changes of the script are simply not applied without clearing the script cache.

---

Perhaps interactively ask […]

I’m kinda trying to steer away from asking the user too many things.

Uaaaah shudder that’s untrusted and not reliable system operation.

:man_shrugging: For Nightly, it works better than any “package” providing a Nightly version…

[SebastianSimon]

All of them?

Who cares about these cache files anyway?

It’s really tempting to just do something like this:

function clear_firefox_caches(){
  local -r cache_dir="$(eval echo "~${SUDO_USER:-${USER}}/.cache/mozilla/firefox")"

  if [[ -d "${cache_dir}" ]]; then
    shopt -s nullglob

    for startupCache in "${cache_dir}"/*/startupCache; do
      rm --recursive --force -- "${startupCache}" 2>/dev/null
    done

    shopt -u nullglob
  fi

  touch -- "${firefox_dir}/browser/.purgecaches"
}

# …

clear_firefox_caches

[mirabilos]

Sebastian Simon dixit:

but which ⟨profileid⟩.⟨profiletype⟩ do I pick? All of them? The

All of them for all users.

I bet iterating through all users’ Firefox profiles would not be appropriate.

No, not really.

But I think, an rm -r on a startupCache directory of the current user is more appropriate. Who cares about these cache files anyway?

Yes, except: which user’s do you pick?

Some profiles might also belong to different browser versions…

A cache purge, be it manually or via script, is definitely necessary. No matter how many times Firefox is restarted, the changes of the script are simply not applied without clearing the script cache.

How about just leaving the file there if the browser is owned by root and letting the user confirm that once? The more I think about the various ways, the more I think that that’s (probably…) the least fragile solution for the problems at hand…

Perhaps interactively ask […]

I’m kinda trying to steer away from asking the user too many things.

Right. Given that this applies only to users who fixup a system-wide browser installation in the first place, I’d think they could do with a one-time confirmation…

[mirabilos]

Sebastian Simon dixit:

It’s really tempting to just do something like this:

local -r cache_dir="$(eval echo "~${SUDO_USER:-${USER}}/.cache/mozilla/firefox")"

Yeah, but you can’t rely on either, and these things can be fragile…

(It probably would also cause untold fun if these files were removed while a browser instance is still running…)

[SebastianSimon]

This issue seems to be about two problems now:

1. In some cases .purgecaches persists, i.e. is not deleted by Firefox startup, but the script complains. This is practically already fixed, by removing the check.

2. In some cases .purgecaches does not persist. Then the cache is not necessarily cleared for all profiles, if a user uses multiple profiles; only the cache of the profile that happens to be used first when Firefox starts gets cleared. But clearing all profiles’ caches cannot be achieved with a single Firefox startup. That’s why the caches of several profiles should be cleared directly. The function I proposed even does both:

   1. clear the caches directly, and
   2. let Firefox clear the first cache used on startup.

   This is done in case any one of the approaches fails.

This also relates to your other comment:

Perhaps interactively ask the user to run it once with MOZ_PURGE_CACHES in the environment (with the correct value which should be shown) if the script is run as root (because that’s when this normally happens)…

That alone would be less effective, for the reason above. I could export it, but that’s equivalent to simply creating the .purgecaches file and letting it sit there.

Yeah, but you can’t rely on either, and these things can be fragile…

The first approach is more effective than any built-in purgecaches approach, if it works. If it doesn’t, the second, already working approach backs it up. This hybrid approach doesn’t seem that fragile to me. If actually none of the two approaches work, feel free to submit another issue, once this fix gets released.

---

All of them for all users.

No, not really.

So… not for all users then. I was referring to clearing the cache in all profiles of all /home directories, belonging to different users. That would certainly not be fine, even if it ultimately wouldn’t matter in most cases.

For other users, this will be a matter of reading the documentation and making sure to manually clear the cache, as it’s impossible for the script or the current user to do anything about it without violating other users’ privacy, or at least it’s very hard.

Some profiles might also belong to different browser versions…

That doesn’t really matter; the cache is populated when Firefox starts, by whichever version runs and uses the profile. If the startup cache is cleared in a profile which is used by a different browser version, the cache will simply be repopulated. The script doesn’t affect this in any way.

It probably would also cause untold fun if these files were removed while a browser instance is still running…

It’s just the startup cache. I’ve just tried it: nothing bad happens. 😯

I’d think they could do with a one-time confirmation

How about just leaving the file there if the browser is owned by root and letting the user confirm that once?

It’s not a one-time confirmation, though. For me, Nightly updates every single day. I usually use this script a few times a week.

[SebastianSimon]

The original purpose of checking if .purgecaches is still there was to ensure that no sed call ever replaces anything twice. I’m going to change the script to simply create empty .⟨filename⟩.fixed files which will be checked before using sed.

[mirabilos]

Sebastian Simon dixit:

It’s not a one-time confirmation, though. For me, Nightly updates every single day. I usually use this script a few times a week.

I was thinking of saving the state (confirmation response) here.

The original purpose of checking if .purgecaches is still there was to ensure that no sed call ever replaces anything twice. I’m going to change the script to simply create empty .⟨filename⟩.fixed files which will be checked before using sed.

Ah!

Just making things reentrant would work and be the best fix, sure.

Thanks!

[SebastianSimon]

c1331dc fixes this.

The script attempts to clear all startup caches of the current user (or sudo user) directly and create a .purgecaches file, but that file is no longer checked before running.