`nixops check` can't decode utf-8, unexpected end of data.

[roberth]

I'm getting the following error in nixops check:

UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4094-4095: unexpected end of data

full trace

``` Traceback (most recent call last): File "/nix/store/3r1d0s93v6i8bjvinw3hcgsaim6g1f9q-python3.9-nixops-2.0.0/bin/.nixops-wrapped", line 9, in sys.exit(main()) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/__main__.py", line 56, in main args.op(args) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/script_defs.py", line 578, in op_check results = run_tasks(nr_workers=len(machines), tasks=machines, worker_fun=worker) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/parallel.py", line 106, in run_tasks raise list(exceptions.values())[0] File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/parallel.py", line 70, in thread_fun work_result = (worker_fun(t), None, t.name) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/script_defs.py", line 516, in worker res = m.check() File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/backends/__init__.py", line 200, in check self._check(res) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops_aws/backends/ec2.py", line 2113, in _check super()._check(res) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/backends/__init__.py", line 222, in _check self.run_command( File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/backends/__init__.py", line 600, in run_command return self.ssh.run_command( File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/ssh_util.py", line 373, in run_command return nixops.util.logged_exec(cmd, self._logger, **kwargs) File "/nix/store/q489rpbgqgn9xqsjjdim1ddfq5ck0xgp-python3-3.9.6-env/lib/python3.9/site-packages/nixops/util.py", line 338, in logged_exec data = process_stdout.read() File "/nix/store/nki9ywqzbvz68vr75kn2r7g1q84f5agy-python3-3.9.6/lib/python3.9/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4094-4095: unexpected end of data ```

This looks suspiciously like a utf-8 symbol was cut off at the end of a buffer.

nixops check runs this

                self.run_command(
                    "systemctl --all --full --no-legend | cat", capture_stdout=True
                )

which contains the ● symbol in front of inactive/dead units.

Running it again reveals the same problem at different offsets

UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 12286-12287: unexpected end of data

UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 8190-8191: unexpected end of data

Running the command manually reveals that the proper end of the command output has some distance to the buffer boundary.

> 53160 `mod` 4096
4008

When I save the output to a file, indeed a UTF-8 character spans the buffer boundary

0000aff0  20 20 20 20 20 20 20 20  20 20 20 20 20 0a e2 97  |             ...|
0000b000  8f 20 62 6c 6f 63 6b 64  65 76 40 64 65 76 2d 64  |. blockdev@dev-d|

That's two bytes right before 0xb000 and one byte at 0xb000, which is therefore part of the next page. It's e2 97 8f aka U+25CF aka ●.

This appears to be a python bug, as this happens in the code belonging to the "file handle" (pardon my terminology) returned from subprocess.Popen(..., text=True).

[roberth]

As a workaround, NixOps could read bytes in ssh util and do the parsing to utf8 afterwards.