search

itsjohncs / phial

A Python microframework for building static sites. Will not be completed.
BSD 2-Clause "Simplified" License
4 stars 0 forks source link

Figure out better solution to binary pipelines. #47

Open itsjohncs opened 9 years ago

itsjohncs commented 9 years ago

An idea I had is that Phial won't take care of trying to open the file, and will instead just package up all of the paths into special path objects (or just leave them as strings...) and then you can pipe the names into something that will open the files. return paths | phial.pipes.open_all. I'm not sure how this idea could be extended to customize how the output file(s) is/are opened.

itsjohncs commented 9 years ago

I could also just make a separate decorator for binary pipelines. It does seem to be a fairly different use case because most of the pipes I've made so far operate line-by-line, whereas binary pipes would want to deal with arbitrary chunks...

itsjohncs commented 9 years ago

The second seems better to me, going to try it.

itsjohncs commented 9 years ago

Hmm, I seem to have forgotten why the current implementation is bad... Maybe it will come to me with time.

itsjohncs commented 9 years ago

I think the current implementation is bad because the files are automatically opened for you. The binary pipelines are necessary in the first place because we do some trickiness to ensure that various ways to encode unicode are picked up properly, and we don't want that trickiness to apply if the files aren't textual.

I think an interesting alternative would be lazily opening the files. So initially your pipeline function is given a list of special file objects that haven't been opened yet (so are just filenames). You can explicitly open them yourself (and pass in whatever parameters you want to enable/disable the unicode decoding), or you can let them be opened by the pipes (or whatever those are called) which will open them if they're not already open.

This should give you sane behavior most of the time, but it does feel a little magical. I think explicitly saying what parameters specific pipes will use when opening the pipeline (aside: holy crap I need to sit down and define this vocab better) will solve the magical problem tidily.

itsjohncs commented 9 years ago

My homework: read and appreciate the Node.js Stream API.

itsjohncs commented 9 years ago

Learned thing number 1: stream is a good name for the thing that the pipeline function receives.

itsjohncs commented 9 years ago

Well I guess currently stream would be a bad name, because the thing is not a stream. It's more like a list that each pipe can mutate. It probably should be a stream though, because those are pretty understandable and will align well with Gulp.

I can also pretty much implement Vinyl. There's not much there to implement, and it seems to work well.