search

staticfloat / Ogg.jl

Julia package to interface with Ogg containers and streams
Other
3 stars 5 forks source link

[WIP] Streaming support #9

Open ssfrr opened 5 years ago

ssfrr commented 5 years ago

This is a WIP PR that I'm using in some of my research code. It implements basically the API in #5, and also supports sample-accurate seeking within a file. I've only really implemented the streaming decoding, not encoding yet (though that should be pretty easy to add).

It also makes use of libogg's zero-copy semantics so it should be very fast, or at least amenable to being made fast with some benchmarking.

It doesn't need review yet, and I don't have an ETA on cleaning it up for public usage, I just figured I'd stick it here so it doesn't get totally lost. I may update it from time to time (and who knows, maybe even finish it someday)

@staticfloat

ssfrr commented 5 years ago

Oh, one of the open issues with this implementation is that it doesn't support chained files. Originally the thought was just that you'd use separate OggDecoders for each link in the chain (and iterating through the pages/packets would end when you reached the end of a chain). But the problem with that is that the underlying IO reads in chunks before parsing the pages, so it will generally read some data that belongs to the next chain. It either needs

  1. a way to push that unused data back into the IO (requiring the user to supply an IO type that supports that), so it's available for the next OggDecoder
  2. a way for the user to retrieve the leftover data from one OggDecoder and pass it to the next one as initial data
  3. Have a single OggDecoder handle the whole chain.

I think maybe @samoconnor has maybe dealt with this sort of thing in HTTP.jl, where you have a streaming reader that might overshoot and needs a way to feed the leftover data to the next reader. Might be nice to have a standardized way to handle this.

samoconnor commented 5 years ago

Maybe this is a good general solution:
https://github.com/BioJulia/BufferedStreams.jl ?

staticfloat commented 4 years ago

Just FYI, I just fixed a terrible bug where, since OggStreamState objects were immutable, and (unbeknownst to me) libogg will take the address of the header field and assign it to OggPages to use as scratch space, you can end up in situations where stack-allocated Julia objects get moved around and all of a sudden your Ogg pages get clobbered by literally anything else. Super fun to track down, but it's fixed in my PR #10.