search

polydawn / repeatr

Repeatr: Reproducible, hermetic Computation. Provision containers from Content-Addressable snapshots; run using familiar containers (e.g. runc); store outputs in Content-Addressable form too! JSON API; connect your own pipelines! (Or, use github.com/polydawn/stellar for pipelines!)
https://repeatr.io
Apache License 2.0
68 stars 5 forks source link

Version Mux stream #40

Closed kofalt closed 7 years ago

kofalt commented 9 years ago

We should probably throw an int out front of the output stream before anything else.

This would facilitate the storage + playback of output streams across any protocol changes. And in general it makes us less awful people.

warpfork commented 9 years ago

There's a byte mentioned somewhere in the cbor spec that's appropriate. I forget what. (Equivalent of the typical assumption that { means json.)

Probably wouldn't hurt to put a {"v": 1} at the front of the stream, too.

warpfork commented 9 years ago

Ah, here it is: https://tools.ietf.org/html/rfc7049#section-2.4.5 Self-Describe CBOR with 0xd9d9f7.

kofalt commented 9 years ago

Cool. That's just a byte marker for "I am a CBOR stream", yes?

So we'd probably do both that & a {"v": 1}, maybe as the first element in the stream array

warpfork commented 7 years ago

Closing. It's issue tracker cleanup day.

This issue is probably also outdated: the serialized mux system has not proven to be as useful as we might've originally hoped; the replay capabilities are essentially unused; and the combined mux has ordering race conditions by the very nature of how filedescriptors work in linux that result in it being unreproducible in most nontrivial cases, thus not great to store... it's possible that the serialize mux feature should be dropped entirely from repeatr core.