Option to disable / delay auto update

[unbracketed]

I find the behavior of running an upgrade process prior to executing the command I've specified troubling.

The last thing I want when running a system command (especially if it is a time-critical / production command) is an arbitrary update process blocking it. What if the upgrade process fails? Is my CLI crippled at a critical moment?

I understand that keeping users up-to-date on the toolbelt benefits everyone in terms of security, but I depend on the CLI tool executing in a predictable manner.

[fabn]

+1 for this

[fgrehm]

Same here, we run some commands as part of our CD pipeline and it sucks that it breaks every now and then because of an update :cry:

[ervinb]

+1 this causes critical issues in a CI environment.

[jdx]

This will happen, but ensuring it is done right is easier said than done. I apologize I don't have a timeframe for it, but it's a near-term goal of the project.

[AnwarShah]

+1. This is annoying

[dpetersen]

I've built some docker images for useful tasks, like pulling the latest database backup onto our staging server. Autoupdate completely screws that up, and I keep having to rebuild my image from scratch so there's nothing to update. Any automated processes that use the CLI run into this issue.

[jdx]

I'm actively working on this now, but it's a big project and I'm not sure when it will land.

[toots]

Guys, seriously, how hard can it be to add an option to disable this? Our heroku cli keeps breaking in our automated docker image system and it is really, really annoying.

[AnwarShah]

I just hardcoded the heroku-cli not to check for upgrade.

[brissenden]

:+1: from me too.

[jdx]

I have some updates, you may not like them, but I want to be open about this. Also, this turned into a novel, sorry about that.

The main reason you all want this is updates sometimes fail or introduce new bugs. We're in the process of rewriting the CLI. This has caused some bugs, but it's also fixed a lot of long standing issues we've had with the CLI for years (especially on windows). This project is wrapping up though and the churn in the project is expected to decrease dramatically soon.

Other reasons for not wanting the autoupdater might be the download size. We have some code in place to use bsdiff to release patches the the CLI, turning updates from ~13mb to ~130kb. This isn't out yet, but should be relatively soon.

One issue with updating is that the updater used npm and npm update doesn't work on some platforms (like in docker which is this issue https://github.com/npm/npm/issues/9863). npm just generally seems to fail sometimes as well.

Disabling the autoupdater isn't the solution, it's making sure updates and the CLI always work. To that end, this week we shipped a big change to the CLI this week (v5.0.0) that does not require npm at all unless extra plugins are used. It's already proven to be much more stable than using npm and these update issues should be a thing of the past.

That's not necessarily true with added plugins which still use npm, but we have a solution planned for those as well. They will still work and the rest of the updates will be reliable, npm just fails sometimes so you'll have to retry and if it really screws up, reinstall them. That is not ideal, I realize, but it's not an issue with our CLI, we've tried to fix these unreliable issues for over a year and have finally come to the realization we just can't use npm.

There may be some bugs with any kind of big change like this, right now the only known and unresolved one is with nixos, but as other unusual setups come in we may find another bug or two with specific setups and we will patch them. Our turnaround time for bugs like that is very fast and it's our top priority to fix them.

Anyways, I get your frustration, but the reason you want this is because we have bugs, not because we should include ways to disable the updater.

Having said that, on platforms that have a good package manager that users update we will ship non-autoupdateable versions. Right now that's just debian, but I plan to add rpm and nixos as well. Homebrew we've done research in and have found users only rarely update their clients and usually only when having problems, so we will not be adding that functionality on darwin even as an option (because homebrew maintainers would just set it and we don't have control over that). Our turnaround time for getting formula updates into homebrew sometimes takes weeks after I submit the PR.

Having the autoupdater is very important. It ensures that I can fix bugs for you without you ever seeing them, that you're on current versions of the API, and that new features are readily available. I still get support tickets from users on ancient, several years old, clients that predate the autoupdater and are not working. Nobody wants to have to submit a support ticket to find out just that their CLI was out of date, and I certainly don't want to answer them. It's a waste of everyone's time.

Also, package managers are typically far out of date. The nixos one for example I just looked at and is 6 months out of date. We want to be able to ship features immediately. We would have problems if launching products depended on us getting updates into package managers. This is why I can't just make it an environment variable or something, because package managers would disable it.

It pains me to write this, because I know you all aren't going to like it, and that's definitely not an experience I want you to have, but adding this as an option would cause more problems overall, not less. Forcing you to use the autoupdater gives us a strong responsibility to ensure its stability, and one we take very seriously.

However if you really want this, you could compile the CLI yourself. A compiled dev version of the CLI will not autoupdate. In fact, I'll add some developing tips on how to do that to the README. (likely just a make install)

[toots]

I have to say that I am disappointed by this. This is, of course, your call to decide what you want to do with this software but if you believe that the solution is to never ship with bugs anymore then I believe that this is at the very least wishful thinking. If you look at other similar tools such as the gcloud SDK for instance, when an update is available, the software prompts for the suggested command line call to do it but let the user decides when's a good time to do so. Breaking in CI context is already annoying but it could be worst like auto-update breaking the CLI tools while in the middle of a time-critical deploy.

[dpetersen]

Certainly understand why you want autoupdates for the majority of users. Thanks for the explanation. I will probably switch to using the API for what I'm currently using the CLI for.

[jdx]

We always recommend using the API directly for automated things. It's much less likely to change, the CLI is a human interface and will have some change (though we try to keep it very minimal)

[jdx]

Also please do try the dev install instructions I just posted. If it doesn't work for you, let me know and I can fix whatever or add more docs around it.

[mikehale]

However if you really want this, you could compile the CLI yourself. A compiled dev version of the CLI will not autoupdate

What is to stop nix, homebrew, and other package managers from just using dev builds? A config option vs a dev build to disable auto updates seems a bit arbitrary.

[jdx]

It's not trivial to build the CLI, and it's targeted to specific architectures. It'd be easier for them just to use the precompiled tarballs. Technically speaking though, they could, but I doubt they would spend the time to.

[pconrad]

So, our problem with the auto-update is that I'm in an academic computing environment, and the students not only have disk quotas, but they have "number of files" quotas as well.

The auto-update to the CLI exceeds their "number of files quota" (which is 4000) before they can even complete their first heroku assignment.

We NEED a way to install a version of the CLI in one place, and then have all of them use it, and only have the INSTRUCTOR or TA install the updates, centrally, from one place.

[luckymike]

This thread is extremely disappointing. A flag to disable auto-update seems fairly trivial and would give folks a path forward where auto-updates are a problem. Assuming use-cases and explaining away legitimate user requests seems like a very short-sighted approach, and is frankly rude.

[pconrad]

So compiling from source wasn't an option for me in the tightly academic computing environment in which we found ourselves. I just did a jury-rig nasty hack, namely wrapping the heroku command line tool with a wrapper that, if the user was not the instructor, created a symbolic link from the users ~/.share/heroku to the instructor account ~/. share/heroku. It was not, and is not, a good long term solution. But it got us through the course.

I do rather wish though, that auto-update were a configurable option without having to build the entire tool from source. And that it wasn't the case that in a shared Linux environment, every user has to have their own copy of the entire toolbelt, consisting of over 20,000 files, under their own home directory. I want to use Heroku in my course this Fall. But with 85 students, my sys admin is going to be mighty unhappy backing up 85 copies of the same 20000 files.

If there were a way to get to logs through the web interface, honestly I don't think I'd need the toolbelt at all. I'd just have my students do everything through the web interface. But if there is one, I haven't found it.

[edmorley]

If there were a way to get to logs through the web interface

https://dashboard.heroku.com/apps/<APP_NAME> -> "More" menu (top right) -> "View logs"

Links to a page of form: https://dashboard.heroku.com/apps/<APP_NAME>/logs

[pconrad]

@edmorley ... Thanks! That is super helpful. If i can figure out a way to have my students do everything they need to do without the heroku command line tool, so much the better.

Make no mistake... I WOULD PREFER to use the command line tool. But, the auto-updating to each of 85 students home directories in an academic computing environment, and the fact that there is no "nice" way to stop that from happening, short of compiling from source, or hacks---this makes it ugly, ugly, ugly. I have to put a "wrapper" around the Heroku tutorials, because they all assume you have root on the system where you are working---and no user in an academic computing environment EVER has root. And the frustrating thing is---you really DON'T NEED root. But I have to reverse engineer the install, and explain all of that to my students. It's frustrating.

[jdx]

The new CLI install won't require root. It's needed now on standalone because some systems don't have user access to /usr/local. The new CLI won't use a script piped to sh. It'll just be a few commands to execute.

With the automatic updating. I'm currently working on a new iteration of the CLI that will be pure-node instead of half go and half node like now. I'm still working on how the updater will work for that, but it is likely the CLI will be an npm package.

While installing with npm will never be the official install method, it might provide a good way to distribute a non-automatic updating CLI in addition to the automatic updating CLI.

This is doable with npm because users on the non updating version can still update manually with npm.

[altV]

Just runned from multiple processes, and Heroku CLI just died on me with process exit code 1 and message

▸    /app/.cache/heroku/update.lock is locked with readers active: 166 3265

heroku -v says heroku-cli/6.14.41-47054d2 (linux-x64) node-v9.2.0

could it be due to autoupdate?

(Command was heroku pg:info ..)

[boltronics]

Just tried to install on a Debian box, but got stuck on the curl -L https://cli-assets.heroku.com/apt/release.key | sudo apt-key add - step. No thanks - I'd like to verify the fingerprint before trusting this key with root access to boxes I administrate. Unfortunately I couldn't find the fingerprint published anywhere, and the fetched key is from 2011. Not a great start.

So then I figured I'd just use the stand-alone instructions, and run the command in a somewhat more secure environment using firejail. I used a standard restrictive template as a first step and ran the heroku help command, however the process locked up. Through some troubleshooting efforts I discovered it was stuck trying to update itself, but I don't intend to give the process the ability to write executable code locally. What was frustrating was that the command just stalled in this situation. Instead of simply showing an error message, I had to kill the command manually.

To update, I'll just throw this in a cron job and have it execute weekly. Really sucks that I'll have to waste 23+ Mb each time I check, but reading from https://cli-assets.heroku.com/heroku-cli/channels/stable/ looks to be blocked.

#!/bin/bash
# Updates a Heroku Standalone installation for when autoupdate is not
# possible.
# https://devcenter.heroku.com/articles/heroku-cli#standalone-installation

declare -r OS='linux'
declare -r ARCH='x64'
declare -r HEROKU_URL="https://cli-assets.heroku.com/heroku-cli/channels/stable/heroku-cli-${OS}-${ARCH}.tar.gz"
declare -r BASE_DIR="${HOME}/firejail/heroku/opt"
declare -r HEROKU_SYMLINK="${BASE_DIR}/heroku"
declare ARCHIVE_DIR

wget -q "${HEROKU_URL}" -O "${BASE_DIR}/heroku.tar.gz"
ARCHIVE_DIR="$(tar ztf "${BASE_DIR}/heroku.tar.gz" | grep '.*/$' | sort -V | \
                   head -n 1)"
if [ -d "${BASE_DIR}/${ARCHIVE_DIR}" ]
then
    echo "No update available."
    rm -f "${BASE_DIR}/heroku.tar.gz"
else
    echo "Updating..."
    tar zxf "${BASE_DIR}/heroku.tar.gz" -C "${BASE_DIR}"
    rm -f "${HEROKU_SYMLINK}" "${BASE_DIR}/heroku.tar.gz"
    ln -s "${ARCHIVE_DIR}" "${HEROKU_SYMLINK}"
fi
if [ "$(ls -1 "${BASE_DIR}" | wc -l)" -gt 2 ]
then
    echo "Old versions under ${BASE_DIR}/ can be removed."
fi

Fortunately heroku doesn't attempt to auto-update every time I run the command, but I can see these crashes getting annoying. It's really sad to see Heroku be in such a sorry state.

[4s7r0]

Still think this is stupid as hell in 2018.

[futuretap]

So how can I build a heroku cli that doesn't auto-update? I use heroku in my crontab to schedule some stuff and it spawns node processes for the auto-update that aren't terminated. So after some time there are hundreds of node processes bringing down my machine completely.

[jdx]

@futuretap there are protections in place to prevent that from happening. What happens if you run heroku update? You might just be on an old version

[futuretap]

Here's the output from heroku update:

heroku: Updating CLI... already on latest version: 7.19.4
Updating completions... done

So it look's like I'm on the newest version already. Nevertheless, each time cron launches heroku, a stale node process is left over (despite the heroku command doing its job fine):

~/.local/share/heroku/client/7.19.4/bin/node ~/.local/share/heroku/client/7.19.4/bin/run update --autoupdate

[jdx]

yeah it'll create one process for the autoupdate, but not hundreds

[futuretap]

One process for each cron job invocation. And the process doesn't terminate. So after some time, there are indeed hundreds.

[4s7r0]

I know the Heroku-cli maintainers aren't trying to reach the moon like NASA engineers, but it's disappointing to see them foolishly rebuke basic critical system design principles like "don't set a timer on a bomb you can't defuse."

The operative word in the original post was "critical" and it was largely ignored with a spurious argument that what people were really mad at was inevitable errors happening, which is probably the worst possible response to someone clearly stating "critical" in their description and implying a low risk tolerance for failure. The maintainer's response completely ignores the more important communication that various posters have tried to impart that they too have a very low risk tolerance which could be solved entirely by making the autoupdate easily disabled or default to false.

Is there a logistical reason this feature cannot be implemented or is this a personal/personnel issue?

It's not trivial to build the CLI, and it's targeted to specific architectures. It'd be easier for them just to use the precompiled tarballs. Technically speaking though, they could, but I doubt they would spend the time to.

I think you mean it would be easier for YOU if they used the precompiled tarballs. Unfortunately, that is not a condition of their environment that a lot of people can control, as clearly stated by various posters. What you might consider an edge case is someone else's mission critical infrastructure, like it or not.

[futuretap]

Yeah, I can't understand either why there isn't simply an option to disable the autoupdate. Even it defaults to true, I could at least disable it when used in scripts. Not sure if that's easily detectable but it would be even better if autoupdate is disabled by default when run non-interactively.

[mikehale]

I think that to date the story has been if you need something to be scripted you should be using the api, but the problem is that there isn't a public api for all operations, notably heroku postgres. This leaves admins in the situation where they have to automate a tool that doesn't really support automation.

[pconrad]

And the use case of shared installations for academic computing environments.

Here you don't want the Heroku tool belt to automatically update because you want it to update once for the entire system, in a directory to which end users don't have write permission. In academic computing environments with hundreds, thousands, or 10s of thousands of users say at UC Santa Barbara or UC San Diego, the cost of maintaining 2GB of space per user for each user to install their own set of binaries and npm modules for the Heroku tool belt is not negligable.

I think our sysadmin finally found something she could install using a package manager, and it's been working for us. So I sort of assumed this issue had been quietly addressed. From the traffic here, apparently not.

On Fri, Feb 1, 2019, 7:39 AM Michael Hale <notifications@github.com wrote:

I think that to date the story has been if you need something to be scripted you should be using the api, but the problem is that there isn't a public api for all operations, notably heroku postgres. This leaves admins in the situation where they have to automate a tool that doesn't really support automation.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/heroku/cli/issues/129#issuecomment-459764014, or mute the thread https://github.com/notifications/unsubscribe-auth/ABETKdyip3v4epUbFIFsqy9RR9IT0OcZks5vJF_KgaJpZM4GhPrh .

[futuretap]

@pconrad Would be glad to know how your sysadmin solved the issue!

[pconrad]

This is what she did (we are running fedora)

dnf install snapd -y ln -s /var/lib/snapd/snap /snap snap install --classic heroku

Following these instructions:

https://docs.snapcraft.io/installing-snap-on-fedora/6755 https://devcenter.heroku.com/articles/heroku-cli

And it seems to be working for us.

--

Phill Conrad, Senior Lecturer (SOE)*, Dept. of Computer Science University of California, Santa Barbara Joint Appointment: College of Creative Studies (www.ccs.ucsb.edu) pconrad@cs.ucsb.edu, www.cs.ucsb.edu/~pconrad

*Senior Lecturer (SOE): a UC teaching faculty appointment, corresponding in rank and job security to a tenured full professor

On Fri, Feb 1, 2019 at 9:48 AM Ortwin Gentz, FutureTap < notifications@github.com> wrote:

@pconrad https://github.com/pconrad Would be glad to know how your sysadmin solved the issue!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/cli/issues/129#issuecomment-459807307, or mute the thread https://github.com/notifications/unsubscribe-auth/ABETKaxLL7Pk9lOq32P-Kmlwe9_lDMMPks5vJH4GgaJpZM4GhPrh .

[jdx]

Is npm install -g heroku not sufficient for these use-cases?

[jdx]

@futuretap it shouldn't be creating multiple processes like that. I've not seen that behavior before and the logic has been the same for 1.5 years. It should only create a single process to do the autoupdate an hour after the CLI is no longer in use. If you've modified the permissions to block the CLI from accessing some of its files it might cause that to happen.

[futuretap]

Thanks, this seems to work.

The CLI was installed under a different user than the script / cron job runs under. I still consider it a bug that it creates multiple update processes if it can't update due to permissions.

[SampsonCrowley]

However if you really want this, you could compile the CLI yourself. A compiled dev version of the CLI will not autoupdate. In fact, I'll add some developing tips on how to do that to the README. (likely just a make install)

@jdxcode where are the instructions to do it yourself? forcing package managers to give users a package that refuses to use the version they chose is unacceptable