search

Jack000 / Expose

A simple static site generator for photoessays
MIT License
4.39k stars 257 forks source link

Feature suggestions and improvements #19

Open mplewis opened 9 years ago

mplewis commented 9 years ago

Hey @Jack000! Thanks for sharing Expose. It's fantastic.

I was using it to build a photo album of a recent trip of mine and I ran into some pain points, so I tried rewriting Expose in Python to see how it was done.

Here are some features I implemented in my attempt at building Expose, expose.py. I'd like to help if you are open to implementing any of them. I'm not a fantastic Bash programmer but I can probably get the job done.

Caching

Images that were generated don't need to be regenerated if the source material hasn't changed. I save a my_file.jpg.src.sha256 file alongside products generated from my_file.jpg and check that before running ImageMagick or ffmpeg.

Dry Run

It helps me debug or add a new feature when I can see the actions that will be taken without them actually affecting the filesystem. expose.py has a --dry-run argument that prints every render/mkdir action without executing them.

Progress Bars

I was frustrated when I tried converting videos. I didn't know how long a video conversion would take, the current setup doesn't provide progress output from ffmpeg, and it's hard to see progress in the filesystem unless you monitor file sizes. expose.py uses a progress bar library to provide pretty progress bars and ETA for mass conversions.

Parallelism

Expose only ever uses 100% of my CPU, when I can go up to 800% with parallelism. My implementation uses Python's multiprocessing library to call as many simultaneous instances of ImageMagick as you want. I did this with ffmpeg too because I'm not convinced that you can spin ffmpeg up to max threads and get as big a performance boost as running multiple instances. This one may be a little trickier in Bash though.

Let me know what you think. I'm happy to help with any of these.

Jack000 commented 9 years ago

hey mplewis

awesome work. Python is definitely a nicer language to work with than a Bash script, and it'll definitely be faster.

caching: as far as caching goes, I currently just check for the existence of the output file. A hash check is a great idea though, I might just implement that.

progress bars: I kind of wish ffmpeg had an option that gave you progress bars. Right now you kind of have to scrape the text output, which seems kind of hacky to me.

performance: As far as performance goes, Python should be an order of magnitude faster at text manipulation stuff (and you can use libraries for templating too). In this bash version I think the most obvious place for performance improvement is still in the templating system, rather than squeezing out some extra performance from ffmpeg.

overall some great ideas! Feel free to use the html bits from this project if you'd like.

jpcaruana commented 8 years ago

we could use the Parallel command: https://www.gnu.org/software/bash/manual/html_node/GNU-Parallel.html