Add a python script to build, run and setup and tear down ip tables rules

[tfoote]

This provides a single entry point and takes care of both setup and cleanup of the iptables rules.

It's super convenient in that it will build local tweaks, and setup the iptables rules and strip them off when shutting down.

It would be great to get feedback on this. I can polish it up and add documentation if you're interested in merging.

I've also got a variant which enables the disk caching as it's very useful when using docker heavily for similar repeated tasks which is why I'm using local checkouts instead of the prebuilt image. I'll try to find a way to parameterize that too for submission back.

[jpetazzo]

Oh, that's nice! However — I think I would prefer to use Fig instead ...

I think it's possible to have the same result with Fig; with the added benefit that it would be maintainable for people who are not familiar with Python (just in case...)

What do you think?

[tfoote]

I'm not familiar with Fig. I'll take a look at it. It looks powerful for bringing up groups of tools. From a quick skim of the docs one issue I see is that my script is still doing things host side which I think breaks their model of being isolated. I'll try it out and see if it can work.

[jpetazzo]

What are you doing host-side?

If it's just network operations like iptables rules, you can do them from a container started with --net host option (and --privileged if needed).

If it's something else... Don't hesitate to let me know. I really love how Fig works and would be happy to help.

[tfoote]

Thanks for the pointers. I've made some noteable improvements. It now can cache on disk, the size is parameterizable. I also increased the max object size and made it parameterizable.

With the disk cache I documented and tested running with a mounted volume to have persistent cache between instances.

And the iptables rules are setup from inside. It will gracefully fail, and warn the user if not run with --privileged.

It does still use the python script, but I don't think you can do the launch as cleanly without a script for the sequencing. For example it waits for the squid instance to start before enabling the transparent redirect so that no requests fall in the gap when starting. And it's catching signals such that it correctly cleans up when stopped, including stripping the iptable rules.

I ended up not using Fig for this as it's not appropriate, but my other project I'm picking it up as it does look quite useful for deploying groups of containers reliably.

[jpetazzo]

Very nice job. There's only one last thing: running Squid itself in a privileged container... I'm a bit worried about this.

Do you think we could change the start instructions to be:

• start squid using the container
• start the same container, in privileged mode, with a special start option, so that it takes care of the iptables rule? (It can exit just after that.)

Maybe there can be another option for people who want to run only one command and are fine with Squid running in privileged mode.

Do you see what I mean?

In any case, thanks a lot, this is very helpful!

[tfoote]

Thanks for the feedback. I think i've found a relatively elegant solution to full isolation. There are now two containers one running squid, and one with permissions to run iptables. You can choose to run either squid or tproxy which links to squid.

There's a fig.yml in the root. Defining both and also demonstrates the parameters.

To run both you can use:

fig run tproxy

it will use the linking to start squid automatically

Or just the non-privlieged squid instance:

fig run squid

and it's up to you to do the iptables manually. (Or just set you http_proxy setting for regular use)

I have one outstanding issue i'd like to resolve. I've ticketed for myself at https://github.com/tfoote/squid-in-a-can/issues/2 where it can get into a state that it cannot be just rerun by fig. I am going to debug that some more.

My one other question is how this might be deployed easily. if both images are posted on hub.docker.com a slightly modified fig.yml could be made available for wget which would use the published images instead of the build commands. Is there a standard way for sharing fig.yml files and embedding published images?

[jpetazzo]

Nice! Sorry that it took me so long to review your changes.

I'll merge this now.

To answer your quetsions:

it can get into a state that it cannot be just rerun by fig. I am going to debug that some more.

I believe that this is a Fig issue, and if you can reproduce it in a simple scenario, the Fig authors will love to hear about it to fix it.

Is there a standard way for sharing fig.yml files and embedding published images?

Not yet! But the Fig authors (them again! :-)) are working on embedding Fig-like features into Docker. Let me know if you want a preview of that!

Thanks again.