Add support for live data reading

[twiclo]

It's a little unclear from the readme but it seems like you don't support reading from a live match. When building peppi and running the slp command on a live file it will error out saying the buffer isn't full. Here's the error when trying to parse from a live game

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ParseError { pos: Some(55), error: Custom { kind: InvalidData, e rror: "expected: [85, 8, 109, 101, 116, 97, 100, 97, 116, 97, 123], got: [54, 3, 7, 0, 0, 50, 1, 134, 76, 195, 0]" } }', src/main.rs:26: 5

[hohav]

Peppi does support live matches, if you use event-based parsing and pass in a stream instead of a file. (Although the parse wrapper in lib.rs requires Seek, that's just for better error reporting and I plan to relax that. In the meanwhile you can use parse::parse directly.)

The best way to get a stream would be to talk to Slippi directly via a network connection or UNIX socket (instead of letting it write to a file and then reading that), but I don't know if there's an easy way to do that right now.

As a workaround, you can use tail on Linux or Mac (e.g. tail -c+1 -f game.slp), either via Command or by just piping the output of tail to your Rust program's STDIN.

On Windows, I believe you can use something like PowerShell's Get-Content with -Wait and -Tail, but I can't guarantee anything.

The error message you're getting means peppi thinks it's done reading the raw block, which most likely means you're on a commit prior to 2ddd958438ac53f421c8df80a933a5ed41a43113. If so, can you update and let me know if that fixes things?

[hohav]

Ah, I didn't notice you're using the slp tool. It doesn't make a lot of sense to run that on an in-progress replay.

You could still do it with tail if you want (e.g. slp <(tail -c+1 -f game.slp)), but that will just block until the file is complete (assuming you're on 2ddd958438ac53f421c8df80a933a5ed41a43113 or later). So there's not much point in doing that instead of just waiting until the file's complete before running slp.

[twiclo]

I tried passing a file via stdin into peppi::parse::parse which returned an UnexpectedEof error. I believe this is coming from your read_exact in parse.rs

I believe read_exact is the problem because looking at the slippi file of a live game reveals the last event doesn't seem to have all of its bytes yet

I double checked and I am on the latest commit

[hohav]

How are you running your program? If you're using tail -f then it shouldn't ever reach EOF.

[twiclo]

I forward the live game stream into cargo: cargo run < ~/Slippi/Game_20200826T233310.slp

And gave it to my program: peppi::parse::parse(io::stdin(), &mut handlers)

Running with tail: tail -f ~/Slippi/Game_20200827T125636.slp | cargo run >expected: [123, 85, 3, 114, 97, 119, 91, 36, 85, 35, 108], got: [186, 247, 0, 14, 194, 32, 0, 0, 65, 200, 0]

[hohav]

You need to use tail -f for live replays. Otherwise you'll run into exactly this problem— peppi hits EOF and has no way to know the file is still being written to. tail -f creates a stream that never closes, so peppi will keep blocking on reads until it has read all the game data.

~In bash, you can do this with process substitution: cargo run -- <(tail -c+1 -f game.slp).~

Nevermind, if your command reads from STDIN you can just use a regular pipe. But don't forget the -c+1 argument to tail so that it doesn't skip part of the file.

[twiclo]

tail -c+1 -f ~/Slippi/Game_20200827T131405.slp | cargo run did the trick. Thanks. Any suggestions for making a blocking reader so I don't have to deploy tail in my program?

[hohav]

You could look at https://github.com/uutils/coreutils/tree/master/src/uu/tail if you want to implement the same functionality natively. But if you just want to avoid having to call tail externally, a simpler approach might be to run tail via Command and read from its STDOUT.

[hohav]

A better approach, if possible, would be to read directly from Slippi instead of letting it append to a file over and over and then polling that file. I haven't tried to do that, but if you ask on the Slippi discord they might be able to give some advice.