How could this be installed on OpenWRT/LEDE?

[g-rock84]

I am very interested in this project! It sounds awesome! I have no idea why it isn't more popular.

Anyways, I am unfamiliar with Go, so how I could I install this on a OpenWRT/LEDE?

How resource intensive is it? My router is has a 1.3 GHz dual-core with 512 MB of RAM.

[gaul]

Unfortunately we do not provide binaries yet, although there is a Docker image. You can easily use go get and go build to create the compy binary.

compy takes minimal resources although it needs to buffer any images in-memory so it can transcode them.

As for popularity, I suspect that configuration dissuades users, especially SSL certificates. We need more documentation as #24 suggests. Always happy to accept pull requests to the README although we should build out the wiki as well. A well-written blog post, especially with benchmarks, would help if you can publish one.

[g-rock84]

hmm... well I was looking for a project! I'll get to work and as soon as I get running ill type up a report with benchmarks. Thanks!

[g-rock84]

when i attempt to "go get" compy, I receive the following error:

jerald$ go get github.com/barnacs/compy
# github.com/pixiv/go-libjpeg/jpeg
go/src/github.com/pixiv/go-libjpeg/jpeg/compress.go:6:10: fatal error: 'jpeglib.h' file not found
#include "jpeglib.h"
         ^
1 error generated.

This on my Mac btw.

Edit: This appears to be a go-libjpeg issue and not a compy issue. I contacted them.

[gaul]

I expanded the installation instructions in the README. You need to brew install jpeg to satisfy this dependency. I opened #27 to track a compy Homebrew package if you would like to contribute this.

[g-rock84]

Thanks! That helped a lot. I have been experimenting with it and I finally got my laptop to route traffic through it. I am only testing locally (I have it installed on my laptop) and with just HTTP currently. I'm averaging 80-90% transcoded. I'll have to devise a way to measure the data difference.

I am getting this error that I can't quite figure out. the command I am running: compy -gzip 9 -gif -jpeg 10 -minify -png

the error:

2017/10/02 07:19:52 serving request: /
2017/10/02 07:19:52 error forwarding request: dial tcp 127.0.0.1:19536: getsockopt: connection refused while serving request: http://127.0.0.1:19536/

oh and testing on http://www.reactiongifs.com/ it doesn't convert the gifs to static images. Did I get the syntax wrong, maybe?

Oh and I am documenting what all I am doing so hopefully I can write a guide/review and spread the word!

[gaul]

compy should log a summary when it exits:

2017/10/02 18:42:01 compy exiting, total transcoded: 4078252 -> 280996 (6.9%)

The connection refused error is mysterious -- it looks like your client tries to connect to localhost and not a remote one? It would be good if compy supported more vebose logging so I opened #28.

For reactiongifs, compy converts to a static WebP for me using Chromium:

2017/10/02 18:38:37 serving request: http://www.reactiongifs.com/r/sdey.gif
2017/10/02 18:38:45 transcoded: 3847277 -> 56912 (1.5%)

[g-rock84]

Oh yeah I saw that. I am talking about like X KB/MB saved.

The connection refused error probably has a little to do with me running this on my Mac and pointing my Mac 127.0.0.1:9999. Other than the error everything seems to be working.

I found out the gif issue as well. Chrome and Safari keep images cached so when I went to the site to get the 'before' measurements, Chrome cached the images so when I went back to get the 'after' measurements it just reused the image already stored. It is the same for images that were transcoded by compy. When I go back to those sites the images are still the lower quality ones. At least I think that's what happened. That has thrown a wrench in my benchmarking plans! Back to the drawing board.

I am getting some strange numbers for the "amount transcoded" report. Example: 2017/10/03 07:53:27 transcoded: 53 -> 77 (145.3%) 2017/10/03 07:53:27 transcoded: 61 -> 74 (121.3%) 2017/10/03 07:52:56 transcoded: 61 -> 74 (121.3%) 2017/10/03 07:52:56 transcoded: 69 -> 93 (134.8%) 2017/10/03 07:52:28 transcoded: 0 -> 23 (+Inf%)

Is that normal? What does it mean? Maybe stuff was added to the transcoding queue after the initial measurement?

A logging function would be amazing! What do you guys think about maybe a .conf file?

[gaul]

Very small HTML and images can increase in size due to overhead of compression metadata and possibly different formats have different overhead, e.g., GIF vs. WebP. It would be good to classify these in another issue, at least to debug the 0 -> 23 case. We stream gzip data so returning the original data is hard but we buffer images so we could just return the original in this case.

Note that benchmarks can show slowdowns if your internet connection is fast since compy introduces latency while transcoding.

Adding a separate configuration mechanism is probably overkill. Instead could you just create a script with your preferred flags? Ideally the browser should be able to configure compy instead of flags. We support this today via Accept and X-Compy-Quality headers #14.

[g-rock84]

Adding a separate configuration mechanism is probably overkill. Instead could you just create a script with your preferred flags?

Absolutely! That's a great point. Just wondering what your thoughts were about it.

Ideally the browser should be able to configure compy instead of flags. We support this today via Accept and X-Compy-Quality headers.

Would this also be true for say mobile browsers? My ultimate goal is use this in combination with a VPN to cut down on mobile data usage.

[gaul]

It depends on which browser you use, but Firefox Android supports Modify Header Value. Again, ideally the browser should configure compy, but this may not always be possible so we will have command-line flags.