Atuin won't update history after initial import.

Flightkick commented 4 months ago

New user here, I'm trying to get atuin to work under OpenSUSE Tumbleweed, but my history isn't updating and atuin wasn't working out of the box. I have installed the package through the package repositories with zypper. I'm not sure whether this is an issue for this repo, the package maintainers at OpenSUSE or if I'm doing something wrong.

To reproduce

Install package zypper in atuin (will also automatically install 'atuin-bash-completion')
Start a new terminal window, keybinds don't seem to be registered.
Manually add eval "$(atuin init bash)" to .bashrc
Spawn a new terminal window, keybinds seem to be registered now
Manually import history by running atuin import auto
History is now imported
Type some commands
Notice that none end up in the history (neither Global, Host, Session, nor Directory)

$ atuin --version
atuin 18.0.1

$ bash --version
GNU bash, version 5.2.26(1)-release (x86_64-suse-linux)

My terminal is Konsole, with the command set to /bin/bash.

Operating System: openSUSE Tumbleweed 20240223
KDE Plasma Version: 5.27.10
KDE Frameworks Version: 5.115.0
Qt Version: 5.15.12
Kernel Version: 6.7.5-1-default (64-bit)
Graphics Platform: Wayland

Flightkick commented 4 months ago

I don't know why I missed this but it seems to be fixed with the latest update (18.0.2) as per https://github.com/atuinsh/atuin/issues/1750#issuecomment-1963973448

I am not using bash-preexec nor ble.sh by the way and there's no indication from the install process that either of these are necessary.

I'll wait for the patch to land in the OpenSUSE repo. Sorry for the noise.

ellie commented 4 months ago

Glad it's sorted!

there's no indication from the install process that either of these are necessary.

Our docs specify this: https://docs.atuin.sh/guide/installation/#shell-plugin. It's also listed in the README

If you use the install script, this is handled for you

Flightkick commented 4 months ago

Our docs specify this: https://docs.atuin.sh/guide/installation/#shell-plugin. It's also listed in the README

If you use the install script, this is handled for you

@ellie Thanks for linking that! I've read the guide, but it seems like I've missed this line:

Please note that it’s not enough to install the binary, there is also a shell plugin that requires settings up.

I did read this one, but the phrasing was a bit ambiguous to me:

Once the binary is installed, the shell plugin requires installing. If you use the install script, this should all be done for you! After installing, remember to restart your shell

Was misunderstood as: If you need to use the shell plugin (whatever that may be), you would need to install it separately.

Especially since it mentions ble.sh, I was under the impression that installing this was only necessary if you are using atuin in conjunction with something like ble.sh. Since I wasn't, I thought that it would not be necessary to install.

I defaulted to using the package manager because I noticed there was a package available for my distro. For multiple reasons, that's usually my preferred approach rather than running arbitrary install scripts.

I would have expected that installing atuin through my package manager would install the required components as well, just like the bash-completions are automatically installed.

By the way, I tried to check my installation validity with atuin status, it just fails with:

$ atuin status
Error: Could not fetch history, client error: session not found.

Location:
    /home/abuild/rpmbuild/BUILD/atuin-18.0.1/atuin-client/src/api_client.rs:162:9

(I now remember why I wanted to open this issue in the first place, it was because I couldn't find any other issues with this error. I just simply forgot to put this output in the issue description. My bad)

I think this raw error justifies an actual issue. I would have expected atuin to tell me that required components are missing.

Okay, I'm reopening this issue. While some of this is definitely best described as 'user error', I'd like to propose some suggestions that might help improve the project and the experiences of first time users.

My suggestions:

Docs: Add some more context to the docs on why the installer is the recommended method. Especially what it does and what may not work out of the box if using a package from the distro repo instead.
Docs: Add some basic context on the component architecture so it's easier to understand the requirements and how to make it work.
Docs: Make 'shell plugin' seem less optional. Possibly make it a subheading of 'Manual Install' in the docs as well so that it doesn't look like something separate. Suggested rephrase: 'Once the binary is installed, you need to install a shell plugin (for ... to work).'
Code: Gracefully handle the error thrown when executing atuin status or help guide the user to a possible mitigation.

I've played around with the docs for a bit, I may have changed a bit too much for your liking, but let me know if you'd be interested and I'll create a PR for it.

ellie commented 4 months ago

That actually looks great, I'd be happy to review the PR!

The code error you see is because you're not logged in, but it's been fixed in main anyway

bn-jbischko commented 4 months ago

Just wanted to say, same issue on NixOS, atuin simply doesn't update history for me.

All I did was install the NixOS package, and put the eval line in my bashrc.

With bash being the most common shell in the world, atuin should really just work out of the box without the need for installing other dependencies. Maybe you can vendor them?

Currently on NixOS one needs to add pkgs.blesh or pkgs.bash-preexec to environment.systemPackages in addtion to the atuin package. Then one needs to source the appropriate file, for ble.sh, that is source "$(blesh-share)/ble.sh". One of them should really be a dependency of the atuin Nix package and the atuin executable wrapped to include it in its path and shell hook, but that is a nix packaging matter, I suppose. Bottom line, if it's a hard dependency it should be vendored.

bn-jbischko commented 4 months ago

Okay, even with ble.sh installed and using 18.0.2 this is still a problem for me - just can't get it to work no matter what I do. Just won't add new entries to the DB after the initial history import.

akinomyoga commented 3 months ago

Just installing ble.sh doesn't make it work. You need to load ble.sh in the Bash session.

Also, blesh-share is a file provided by the NixOS package and not a part of ble.sh itself. blesh-share doesn't recognize the flag --version. If you want to get the version of ble.sh provided by the NixOS package, you should run something like

$ bash "$(blesh-share)/ble.sh" --version

bn-jbischko commented 3 months ago

Then one needs to source the appropriate file, for ble.sh, that is source "$(blesh-share)/ble.sh"

I'm aware that it needs to be sourced. The version thing was just to demonstrate, that it was installed. Didn't realize that it doesn't have version option. Still atuin doesn't receive incoming history, with ble.sh sourced. But thanks for you chiming in, I'm sure it's something weird about my setup.

akinomyoga commented 3 months ago

Do you change the setting of ble.sh so that the syntax highlighting is turned off? I'm asking because the image in your comment https://github.com/atuinsh/atuin/issues/1798#issuecomment-1988127644 doesn't look inside blesh's attached state. If ble.sh is correctly loaded in your session, the syntax highlighting (which is not seen in your image) should be turned on unless it is intentionally turned off.

What is the result of the following command?

$ declare -p BLE_VERSION BLE_ATTACHED

Also what is the result of the following command?

$ declare -p ATUIN_SESSION

bn-jbischko commented 3 months ago

Yes, I first tried bash-preexec precisely because ble.sh did too much for my tastes (nothing against it, it's a nice piece of software and I respect your work, Koichi).

declare -- BLE_VERSION="0.4.0-devel3+1a5c451c"
declare -- BLE_ATTACHED="1"

declare -x ATUIN_SESSION="018e3c49197c7cdaae237c1c99e6d7c5"

akinomyoga commented 3 months ago

Thanks. Hmm, I'm a bit confused about whether you are now using ble.sh or not using ble.sh.

bn-jbischko commented 3 months ago

Currently, I'm using ble.sh. I only care about atuin, so I tried bash-prexec on its own first. Bash-preexec didn't even show me the atuin menu, so I removed the source line from my bashrc and added the one for ble.sh. With ble.sh I get the menu, but the history is not updated. Hope that clears it up.

akinomyoga commented 3 months ago

If you are using ble.sh, you need to source ble.sh before eval "$(atuin init bash)" so that Atuin correctly detects ble.sh. What is the result of the following command?

$ ble-import -q

bn-jbischko commented 3 months ago

/nix/store/8sylzfdgpx0yvls6bv4za6wx3ixvjqa2-blesh-0.4.0-devel3/share/blesh/lib/_package.sh
/nix/store/8sylzfdgpx0yvls6bv4za6wx3ixvjqa2-blesh-0.4.0-devel3/share/blesh/keymap/emacs.sh
/nix/store/8sylzfdgpx0yvls6bv4za6wx3ixvjqa2-blesh-0.4.0-devel3/share/blesh/lib/core-cmdspec.sh
/nix/store/8sylzfdgpx0yvls6bv4za6wx3ixvjqa2-blesh-0.4.0-devel3/share/blesh/lib/core-syntax.sh
/nix/store/8sylzfdgpx0yvls6bv4za6wx3ixvjqa2-blesh-0.4.0-devel3/share/blesh/lib/core-complete.sh

akinomyoga commented 3 months ago

Thanks. The result tells that integration/bash-preexec is not loaded, so I suspect you source ble.sh after initializing Atuin.

~~Ah, wait..., but the version of your ble.sh is old. The module name might not be integration/bash-preexec in your version.~~ edit: Sorry, this is OK. I've checked it now. The NixOS package name says devel3, so it should be v0.4.0-devel3 (or newer). In that version, the module name is already integration/bash-preexec so the module name shouldn't be a problem.

bn-jbischko commented 3 months ago

I noticed another, possibly unrelated, thing: when I start bash from another shell session (here xonsh) and exit bash again, then things like Ctrl+d and Ctrl+c don't work anymore: The prompt is different from my other screenshots because I also tried to strip out as much of my other things in bashrc out as possible, to make sure it doesn't interact with anything in ble.sh/atuin. Still no history update.

But please don't invest too much time in my peculiarities here. The only point that is probably applicable to other people is, that atuin should come with bash-preexec and the atuin shell hook should include it, imho. The install guide mentions that now, but it's still easy to miss that that is a requirement for bash.

akinomyoga commented 3 months ago

then things like Ctrl+d and Ctrl+c don't work anymore:

What is your terminal? This is related to the feature called modifyOtherKeys introduced by XTerm. However, the implementations by different terminals are not consistent. The implementations by some terminals are half-broken and fail to reset the state by some of the control sequences for modifyOtherKeys. I include workarounds for such implementations, but the list of incomplete implementations is not complete. I think your terminal is not on the list in ble.sh.

akinomyoga commented 3 months ago

Still no history update.

Yes, that is because blesh's module integration/bash-preexec is not loaded, which is implied by the result of ble-import -q: https://github.com/atuinsh/atuin/issues/1798#issuecomment-1997308923. And that is caused when eval "$(atuin init bash)" is performed before source /path/to/ble.sh. eval "$(atuin init bash)" needs to come after source /path/to/ble.sh. Could you check the ordering of initialization?

akinomyoga commented 3 months ago

But please don't invest too much time in my peculiarities here. The only point that is probably applicable to other people is, that atuin should come with bash-preexec and the atuin shell hook should include it, imho.

Atuin's default installation (using https://setup.atuin.sh) already does that.

The install guide mentions that now, but it's still easy to miss that that is a requirement for bash.

This only applies to the users who install Atuin manually (without using https://setup.atuin.sh). When a user uses https://setup.atuin.sh, the user doesn't need to care about bash-preexec/blesh because it would automatically set up bash-preexec.

I guess it's NixOS's packaging issue.

bn-jbischko commented 3 months ago

The ordering is ble.sh then atuin and I'm using Kitty as my terminal emulator. But yeah, it has to be a little detail about my machine, maybe NixOS packaging, though I doubt it.

akinomyoga commented 3 months ago

Thanks. I tried ble-0.4.0-devel3, but there doesn't seem to be any problem in my environment (but it's not NixOS, so something may be different). What is clear is that integration/bash-preexec is not loaded in your environment for some reason.

Could you try the following setting in your bashrc and provide the result?

# bashrc

source "$(blesh-share)/ble.sh"
eval "atuin_init() { $(atuin init bash); }"
set -x
atuin_init
set +x

akinomyoga commented 3 months ago

I'm using Kitty as my terminal emulator.

Thanks. kitty is supposed to be supported by ble.sh. For this specific case, ble.sh uses kitty's keyboard protocol. Kitty's keyboard protocol uses a pair of push/pop sequences (\e[<u and \e[>1u), and ble.sh counts the number of those sequences so that it doesn't break the state. But maybe it is broken for some reason. I'll test kitty later when I have time. Maybe some other factors, such as terminal multiplexers, can also interfere with it here.

Could you provide the result of the following command?

$ ble/widget/display-shell-version

bn-jbischko commented 3 months ago

$ ble/widget/display-shell-version
GNU bash, version 5.2.26(1)-release (x86_64-pc-linux-gnu) [NixOS 24.05 (Uakari)]
ble.sh, version 0.4.0-devel3+1a5c451c (noarch) [git 2.39.0, GNU Make 4.3, GNU Awk 5.1.1, API: 3.1 (GNU MPFR 4.1.0-p13, GNU MP 6.2.1)]
bash-completion, version 2.11 (hash:2bf1bb7c3e7593cc63953675f56def7030310791, 76396 bytes) (noarch)
locale: LANG=en_US.UTF-8
terminal: TERM=xterm-kitty wcwidth=15.0-west/15.0-2+ri, kitty:0 (1;4000;32)

Could you try the following setting in your bashrc and provide the result?

+ atuin_init
++ atuin uuid
+ ATUIN_SESSION=018e3dba5e93712181aef359170e197b
+ export ATUIN_SESSION
+ [[ -n 0.4.0-devel3+1a5c451c ]]
+ blehook PRECMD-+=_atuin_precmd
+ local set shopt
+ ble/base/adjust-BASH_REMATCH
+ (( _ble_bash_BASH_REMATCH_level++==0 ))
+ ble/base/adjust-BASH_REMATCH/is-updated
+ local i n=0
+ (( n!=0 ))
+ (( i=0 ))
+ (( i<n ))
+ return 1
+ return 1
+ ble/base/.adjust-bash-options set shopt
+ ble/base/restore-BASH_REMATCH
+ (( _ble_bash_BASH_REMATCH_level>0&&
--_ble_bash_BASH_REMATCH_level==0 ))
+ [[ '' =~ none ]]
+ return 0
+ blehook PREEXEC-+=_atuin_preexec
+ local set shopt
+ ble/base/adjust-BASH_REMATCH
+ (( _ble_bash_BASH_REMATCH_level++==0 ))
+ ble/base/adjust-BASH_REMATCH/is-updated
+ local i n=0
+ (( n!=0 ))
+ (( i=0 ))
+ (( i<n ))
+ return 1
+ return 1
+ ble/base/.adjust-bash-options set shopt
+ ble/base/restore-BASH_REMATCH
+ (( _ble_bash_BASH_REMATCH_level>0&&
--_ble_bash_BASH_REMATCH_level==0 ))
+ [[ '' =~ none ]]
+ return 0
+ bind -x '"\C-r": __atuin_history'
+ ble/builtin/bind -x '"\C-r": __atuin_history'
+ local set shopt
+ ble/base/.adjust-bash-options set shopt
+ return 0
+ bind -x '"\e[A": __atuin_history --shell-up-key-binding'
+ ble/builtin/bind -x '"\e[A": __atuin_history --shell-up-key-binding'
+ local set shopt
+ ble/base/.adjust-bash-options set shopt
+ return 0
+ bind -x '"\eOA": __atuin_history --shell-up-key-binding'
+ ble/builtin/bind -x '"\eOA": __atuin_history --shell-up-key-binding'
+ local set shopt
+ ble/base/.adjust-bash-options set shopt
+ return 0
+ set +x

bn-jbischko commented 3 months ago

# flake.nix
# Defines a standalone NixOS virtual machine with ble.sh and atuin installed and configured.
# Requires only the nix executable installed, see https://nixos.org/download/#download-nix
# build and run with:
#  nix build "./#nixosConfigurations.myvm.config.virtualisation.vmVariant.system.build.vm" --experimental-features 'nix-command flakes'  && $TERMINAL result/bin/run-*-vm &
{
  outputs = {nixpkgs,...}: { nixosConfigurations = {
    myvm = nixpkgs.lib.nixosSystem {
      modules = [
        ({ pkgs, lib, config, ... }:
        {
          virtualisation.vmVariant.virtualisation = {
            graphics = false;
            diskImage = null;
            mountHostNixStore = true;
            writableStoreUseTmpfs = false;
          };
          users.mutableUsers = false;
          users.users = { me.isNormalUser = true; me.hashedPassword = ""; };
          users.users.me.extraGroups = [ "wheel" ];
          services.getty.autologinUser = "me";
          security.sudo.wheelNeedsPassword = false;
          nix.settings.trusted-users = [ "root" "@wheel" ];
          environment.etc.nixpkgs.source = pkgs.path;
          nix.nixPath = [ "nixpkgs=/etc/nixpkgs" ];
          environment.sessionVariables = {
            XDG_CACHE_HOME = "\${HOME}/.cache";
            XDG_CONFIG_HOME = "\${HOME}/.config";
            HISTFILE = "\${HOME}/.bash_history";
          };
          environment.shellAliases = { sht = "sudo shutdown -h now"; tt = "stty columns 200"; };
          environment.enableAllTerminfo = true;
          environment.extraSetup = ''rm --force $out/bin/nix-channel'';
          system.stateVersion = config.system.nixos.release;
          nixpkgs.hostPlatform = "x86_64-linux";
          console.useXkbConfig = true;
          programs.neovim = { enable = true; defaultEditor = true; viAlias = true; vimAlias = true; };

          environment.systemPackages = with pkgs; [ blesh atuin ];
          system.userActivationScripts.bash = ''
            mkdir -p .cache
            echo 'echo hallo' > .bash_history
            echo '
              export TERM=xterm-kitty
              source "$(blesh-share)/ble.sh"
              eval "$(atuin init bash)"
            ' > .bashrc
            echo '[[ $- == *i* && -f "$HOME/.bashrc" ]] && source "$HOME/.bashrc"' > .bash_profile
            ${lib.getExe pkgs.atuin} import auto
          '';
        })];
      };
    };
  };
}

Btw. corroborating that it's not a Nix packaging issue is, that it works just fine in a quick NixOS virtual machine. Above is the definition of said machine, if you want to try yourself.

akinomyoga commented 3 months ago

Thank you for the results.

+ atuin_init
++ atuin uuid
+ ATUIN_SESSION=018e3dba5e93712181aef359170e197b
+ export ATUIN_SESSION
+ [[ -n 0.4.0-devel3+1a5c451c ]]
+ blehook PRECMD-+=_atuin_precmd

This doesn't seem atuin-18.0.2. Possibly, do you have two different versions of Atuin in your system? What is the result if you run atuin --version inside ~/.bashrc?

# bashrc

type -a atuin >&2
atuin --version >&2
source "$(blesh-share)/ble.sh"
eval "$(atuin init bash)"

tvanbaak commented 6 days ago

I had a similar issue with history not being recorded. I'm on Pop!_OS 20.04 with atuin 18.2.0 and ble.sh 0.4.0-devel3 both installed via nix profile. My problem was that I only added eval "$(atuin init bash)" to my .bashrc, and it worked once I added source "$(blesh-share)/ble.sh" before that.

I think I missed this because I got linked to #installing-the-shell-plugin and missed the warning just above that, which warns the shell plugin needs to be set up.

atuinsh / atuin

Atuin won't update history after initial import. #1798