Test webtask.io support

[andrewkrug]

Webtask has some unique things regarding their execution envionment. Let's double check that we don't need to do something crazy like mash it into a single file.

Webtask is free to try. I did their hello world and got back... Just by running a one liner. But could not get two files to run.

https://wt-decac903c5c2d6bfe4e6f2927fd436b3-0.run.webtask.io/hello

{"PATH":"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","HOSTNAME":"98ba9fa69a96","USER_ID":"30039","CLUSTER_SIZE":"1","PORT":"/data/io/port.sock","BACKCHANNEL_PORT":"/data/backchannel.sock","OS_LIMITS":"nproc=100:100,nofile=2048:2048","SANDBOX_STATS_HEADER":"1","EDGE_CS_CACHE":"1","EDGE_CS_TEMP_DIR":"/data/io","TRIPWIRE_TIMEOUT_MS":"2000","COOLDOWN_TIMEOUT":"1000","MAX_CODE_SIZE":"100","GET_CODE_TIMEOUT":"10000","MAX_CODE_CACHE_COUNT":"10","VERQUIRE_DIR":"/data/_verquire","NPM_CONFIG_LOGLEVEL":"info","NODE_VERSION":"4.4.5","HOME":"/root","NODE_ENV":"webtask"}

I got the error when trying to load both files. { "code": 400, "message": "Invalid webtask code", "error": "Supplied code must return or export a function." }

[danielhartnell]

Hey @andrewkrug!

I can help work through this specific issue. Sorry for my lack of participation over the last few weeks. I've definitely let time get away from me.

Auth0 provides another project, called webtask-bundle, which might help us to use this specific profiler for both platforms. I'll look into it and see if there are some simple changes that can be made to enable portability.

[danielhartnell]

After utilizing wt-bundle and creating a new launcher for our profiler, I was able to make some progress on this issue.

Within the node-lambda-inspector folder, I created webtask.js:

var profiler = require('./profiler.js')

module.exports = function (cb) {
  profiler.do_lookups((res) => { cb(null, res); });
};

This may be revised (still learning JS) but it's suitable for now. wt-bundle allows us to bundle our existing code for use on webtask.io:

$ wt-bundle --output ./wt-bundle-output/webtask.js ./webtask.js

And now we can run wt create:

$ wt create wt-bundle-output/webtask.js
Webtask created

You can access your webtask at the following url:

https://wt-0ad02e9d22fed48c956fa4d6b11a93d3-0.run.webtask.io/webtask

Clickable link: https://wt-0ad02e9d22fed48c956fa4d6b11a93d3-0.run.webtask.io/webtask

And here's an excerpt after having this profiler run in the Webtask environment:

{

    "runtime": "nodejs",
    "timestamp": 1490485530,
    "env": {
        "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "HOSTNAME": "b3514de32126",
        "USER_ID": "30054",
        "CLUSTER_SIZE": "1",
        "PORT": "/data/io/port.sock",
        "BACKCHANNEL_PORT": "/data/backchannel.sock",
        "OS_LIMITS": "nproc=100:100,nofile=2048:2048",
        "SANDBOX_STATS_HEADER": "1",
        "EDGE_CS_CACHE": "1",
        "EDGE_CS_TEMP_DIR": "/data/io",
...

Looking good!

I'd like to review this further and make some adjustments prior to submitting a PR but this should push us a little further forward. Are you guys comfortable with this implementation or would you like to approach this problem in a different way?

Cheers!

[andrewkrug]

@danielhartnell can you submit your PR with "review required" and we'll make the appropriate adjustments.

[andrewkrug]

@jeffbryner you may find this interesting @danielhartnell has got us our first look at the auth0 webtask sandbox. https://wt-0ad02e9d22fed48c956fa4d6b11a93d3-0.run.webtask.io/webtask

There's not as much here as I would like but from a security standpoint a few things jump out at me:

Particularly the two raw sockets that seem to be a shared volume to the container. Also... I seem to continue to get the same container for every execution..

PORT "/data/io/port.sock" BACKCHANNEL_PORT "/data/backchannel.sock"

[andrewkrug]

Update webtask seems to recycle containers if you hit them more frequently than ~ 2 minutes

[danielhartnell]

Thank you @andrewkrug. https://github.com/ThreatResponse/node-lambda-inspector/pull/13 was submitted last night for review.