Problems creating a trivial function from a Docker image

[fadams]

The use case here is to create a function from a Docker image hosting a simple unix command - in this case it's just stuff supported by busybox.

The tutorial here https://fnproject.io/tutorials/ContainerAsFunction/ doesn't actually seem to answer the question as it's actually based on fnproject/node and I note a require('@fnproject/fdk') and that example seems more about extending an image to include the required packages such as imagemagick.

What I tried was a trivial Dockerfile

FROM alpine:latest

# wc - print newline, word, and byte counts for each file
# With Alpine we don't need to explicitly install anything as wc is
# provided by busybox.
CMD /usr/bin/wc

and func.yaml

schema_version: 20180708
name: wordcount
version: 0.0.1
runtime: docker
triggers:
- name: wordcount
  type: http
  source: /wordcount

I next created an app and ran deploy

fn create app wordcount
fn --verbose deploy --app wordcount

The container gets created fine and I can run wc directly in Docker as follows (where docker-mint.local:5000 is my local simple registry which I've used happily for other functions):

echo -n $'some\nlines\nof\ntext' | docker run --rm -i docker-mint.local:5000/wordcount:0.0.2
        3         4        18

When I try to invoke as a function though:

echo -n $'some\nlines\nof\ntext' | fn invoke wordcount wordcount

I get:

Error invoking function. status: 502 message: container failed to initialize, please ensure you are using the latest fdk / format and check the logs

TBH the documentation relating to the "default container contract" seem a little sketchy though there did seem to be an implication that containers reading from stdin and writing to stdout could be used.

Now as it happens I have also tried hotwrap and replacing the above with a Dockerfile that looks like:

FROM alpine:latest

# Install hotwrap binary
COPY --from=fnproject/hotwrap:latest /hotwrap /hotwrap 

# wc - print newline, word, and byte counts for each file
# With Alpine we don't need to explicitly install anything as wc is
# provided by busybox.
CMD /usr/bin/wc

# update entrypoint to use hotwrap, this will wrap the command 
ENTRYPOINT ["/hotwrap"]

and func.yaml

schema_version: 20180708
name: wordcount-hotwrap
version: 0.0.1
runtime: docker
triggers:
- name: wordcount-hotwrap
  type: http
  source: /wordcount-hotwrap

Then creating app and deloying

fn create app wordcount
fn --verbose deploy --app wordcount

To run directly from Docker I have to do:

echo -n $'some\nlines\nof\ntext' | docker run --rm -i --entrypoint=/usr/bin/wc docker-mint.local:5000/wordcount-hotwrap:0.0.2
        3         4        18

which isn't totally unexpected given the hotwrap entrypoint and if I just do:

echo -n $'some\nlines\nof\ntext' | docker run --rm -i docker-mint.local:5000/wordcount-hotwrap:0.0.2

I get

url scheme must be unix with a valid path, got: 

Is there a way to pass input directly to the hotwrap entrypoint to test directly from a Docker container?

But in any case invoking properly I get:

echo -n $'some\nlines\nof\ntext' | fn invoke wordcount wordcount-hotwrap
        3         4        18

Which is what I'd expect.

So basically I'd like to understand whether one must use hotwrap for the scenario of executing a command from an arbitrary container? If that's the case then fair enough but I do think that the title of the "Creating a Function from a Docker Image" tutorial at https://fnproject.io/tutorials/ContainerAsFunction/ is a tad misleading as that really seems to be extending an FDK image rather than creating a function from a Docker Image - yes I know that technically it is but I certainly interpreted the title as implying a fairly arbitrary image.

If I've misunderstood things I'd be grateful for any pointers as to what I might be doing wrong with my original Dockerfile/func.yaml to get them to deploy as a non-hot function.

MTIA

[rdallman]

@fadams I think this is resolved from #1496 with the exception of the tutorial being a bit misleading? I'm trying to think what to do about it, after reading the page I've come to the conclusion that we never explicitly state that functions still need to use an FDK (hence some of the confusion), and we also never explicitly mention hotwrap as an alternative to using an FDK anywhere there, either. I think the jury is out on whether we've wanted to officially support hotwrap, it does seem worth mentioning at least, though.

here's some instructions here for debugging a container locally, we're also undecided about surfacing this (in a lot of ways, it's easier to get users to run things against fn so that we can more easily debug with them, so we're not sure about recommending/supporting this really, but you may find it useful):

Make a directory to be the UNIX socket filesystem

mkdir iofs

Launch the function in your local docker instance

docker run -d --name function --rm -e FN_FORMAT=http-stream -e FN_LISTENER=unix:/iofs/lsnr.sock -v ${PWD}/iofs:/iofs <function-image>:<image-tag>

Launch a client container (this can be skipped if running docker locally on linux)

docker run -it --rm -v ${PWD}/iofs:/iofs oraclelinux:7-slim

Invoke the function using curl

curl -v --unix-socket –unix-socket ${PWD}/iofs/lsnr.sock -H "Fn-Call-Id: 0000000000000000" -H "Fn-Deadline: 9999-01-01T00:00:00.000Z" http://function/call

Cleanup Kill the function container

docker kill function

Remove the iofs directory

rm -rf iofs

[fadams]

Hi @rdallman, yes I think what you observe is the real issue. Whilst clearly for a great many use cases using a language+FDK is the sensible thing I'm struck by the increasing number of use cases I seem to be thinking of where running some unix commands seems to be more logical. I think that the https://fnproject.io/tutorials/ContainerAsFunction/ is kind of a case in point - why the heck would I want to run node.js just to call the ImageMagick CLI, or ffmpeg or whatever. Either using the CLI on its own or using a simple bash script in combination with jq and curl if I want to do a bit of JSON parsing is the sort of thing I'd tend to do locally so I'm not sure I'd wish to overcomplicate that when running a function.

So yeah I've seen in a few places implications of "bring your own container" but the implication of your response is that some sort of FDK is required. TBH I'm cool with that, but I do think that the documentation is then quite confusing and I also think that hotwrap is a much more natural way to expose a sort of BYOC pattern than an example using node.js. TBH I actually think that the use case the node example is illustrating is subtly different - to me that is mostly showing how one would go about extending a node.js FDK container to include package dependencies. That's absolutely fine and useful but I really think that the tutorial title is quite misleading in that case.

As you say in your first paragraph it would be good to explicitly mention the need to use an FDK and also perhaps to "promote" the hotwrap quickstart into the tutorials section (and I'm tempted to say change CMD /usr/bin/wc -l to CMD ["/usr/bin/wc -l"] as per my observations in https://github.com/fnproject/fn/issues/1496).

On "I think the jury is out on whether we've wanted to officially support hotwrap" well, as I say above I think that there are a number of use cases where doing what it supports seems massively more logical than using another language to simply wrap something that already has a perfectly decent CLI so I'd hope that not officially supporting hotwrap actually means officially supporting something with equivalent behaviour.

Thanks for the instructions on debugging the container locally, that's cool. I'd agree with your observations about a project preference to get users to run things against fn, but when debugging it's often useful to have a number of alternative levers to pull. For example in https://github.com/fnproject/fn/issues/1496 none of my issues turned out to be anything at all to do with Fn per se and was more about the quirks of container entrypoints, I suspect I'd have clocked that sooner had I has those instructions, so I think this stuff is really helpful.

Thanks again for all your help and responses, I do appreciate it!

[carimura]

Thanks for the details writeup of your thoughts @fadams. I'm creating an issue to track the potential promotion of hotwrap or some other documentation for BYOC here: https://github.com/fnproject/docs/issues/74

I totally agree with your sentiment that bringing a container for stuff like you point out above is a first class pattern and thus should have easy-to-understand docs.