Peek API?

[tmm1]

WDyt about adding Peek() to the Reader interface? Here's my use-case..

I'm using the new NextReaderFromNow() on top of a mpegts video stream. mpegts packets are 188 bytes long, where each starts with the byte G. Since a new reader does not necessarily start at a packet boundary, I'd like to be able to discard all bytes til the first G.

One way to do this without peek would be to read 188 bytes, find the G, and then calculate how many more bytes to read and discard. This would mean discarding one or more packets, whereas with Peek I could discard only the bytes required.

[djherbis]

I like the idea of supporting Peek, though unfortunately it would be a 'breaking' change to add it to the interface so I'd have to do a major version number bump. It was a bit of a bad habit of mine when I originally developed these libraries to prefer returning interfaces from methods, which unfortunately is much less extensible than returning structs.

One option might be to just add the method to the underlying struct, and allow something like:

type Peeker interface{
  Peek(n int) ([]byte, error)
}

r := buf.NextReaderFromNow()
if pr, ok := r.(Peeker); ok {
  pr.Peek(...)
}

That being said, I think you could accomplish your end goal (only dropping data before the first 'G') by just doing the following:

r := buf.NextReaderFromNow()
data := make([]byte, 188)
n, err := r.Read(data)
// handle err...
i := bytes.Index(data[:n], []byte("G"))
// handle if i is negative... (read some more?)
alignedReader := io.MultiReader(bytes.NewReader(data[i:]), r)
// alignedReader will start at the first G returned from the original reader w/o dropping an extra pkt

[mstaack]

any updates on this?

is there now a more recommended approach (including synced aligned reads)?

[djherbis]

@mstaack Can you clarifying what you mean by synced aligned reads?

The suggestion I made before could also be done using a bufio.Scanner:

s := bufio.NewScanner(buf.NextReaderFromNow())
s.Split(func(data []byte, atEOF bool) (advance int, token []byte, err error) {
  ...find the desired token in data...
})
for s.Scan() {
  ...scanner.Bytes()... // <- your tokens
}

Runnable sample code of that ^ https://play.golang.org/p/EevLkwDt_db

[mstaack]

I meant reading alway starts with that sync byte of 0x47 aka G

[djherbis]

I think one shouldn't usually expect the io.Reader interface to work like that, I think using a Scanner + Split function that handles alignment and then processing the Scanned tokens is probably the right away to get aligned byte slices out of any reader.

Any reason not to prefer that?

[mstaack]

@djherbis thanks for your response.

sounds reasonable, but i am just getting into golang development and wrapping my head around this topic and how to solve re-streaming mpegts binary streams...so that each new reader starts with a fresh 0x47 sync-byte :)

[tmm1]

You could read bytes off until the sync, and also use a io.MultiReader if you need to send extra buffered bytes down the client.

I found that alignment was unnecessary because most players will sync the stream to find the G themselves.