fish shell: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 251776: invalid continuation byte

[wizonesolutions]

I dumped my fish history to history.txt by running history > ~/Downloads/history.txt

Then I ran:

shell_history_analysis --shell=fish ~/Downloads/history.txt

The output I got was:

Assumed shell: fish
Traceback (most recent call last):
  File "/home/kevin/.local/bin/shell_history_analysis", line 8, in <module>
    sys.exit(entry_point())
  File "/home/kevin/.local/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/home/kevin/.local/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/home/kevin/.local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/kevin/.local/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/home/kevin/.local/lib/python3.9/site-packages/shell_history_analysis/cli.py", line 21, in entry_point
    main(filename, shell, grouping)
  File "/home/kevin/.local/lib/python3.9/site-packages/shell_history_analysis/analyze.py", line 24, in main
    df = read_history(history_filepath, shell)
  File "/home/kevin/.local/lib/python3.9/site-packages/shell_history_analysis/shell_io.py", line 13, in read_history
    content = f.read()
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 251776: invalid continuation byte

It seems to be something with the file encoding, but I'm not entirely sure what. My LANG is set to en_US.UTF-8.

I tried with a copy of ~/.local/share/fish/fish_history as well, but I got the same error with the position of the invalid continuation byte being different.

What am I doing wrong?

[wizonesolutions]

Oh, I think I see the issue: fish isn't supported by the tool 😅

[wizonesolutions]

It does seem like the fallback regex should be working, though. Here's what history.txt looks like:

env | grep -i utf
shell_history_analysis --shell=fish ~/Downloads/history.txt
cp ~/.local/share/fish/fish_history ~/Downloads/history.txt
head ~/Downloads/history.txt
vim ~/.local/share/fish/fish_history
vim ~/Downloads/history.txt
wc -l ~/Downloads/history.txt
man history
history
shell_history_analysis --shell=fish

Here's what ~/.local/share/fish/fish_history looks like:

- cmd: pacman -S brave-browser
  when: 1565611653
- cmd: sudo pacman -S brave-browser
  when: 1565611653
- cmd: sudo sysctl kernel.unprivileged_userns_clone=1
  when: 1565611653
- cmd: echo kernel.unprivileged_userns_clone = 1 | sudo tee /etc/sysctl.d/00-local-userns.conf
  when: 1565611653
- cmd: inix -G
  when: 1565611653