Implement FastCGI routes in Oak for developer testing?

[shah]

@jeremiah-shaulov this is a fantastic library, nice work! We use a Deno web server with Oak for developer sandbox testing and we use Nginx and Traefik in production. Right now developers have to install Nginx to test their FastCGI routes/"apps" so I was wondering if you could suggest an approach to use deno_fcgi as Oak middleware that could allow testing of FastCGI routes within our existing Oak-based web server.

Thoughts? Thanks!

[jeremiah-shaulov]

Hey. Thank you for your positive feedback.

I'm not sure that i understood you correctly. Do you have an app that works like a FastCGI server (like PHP-FPM)? And do you forward requests to this app from Nginx? And wanting also to forward from Deno+Oak in testing environment?

Something like this?

import {Application} from "https://deno.land/x/oak@v9.0.1/mod.ts";
import {fcgi} from 'https://deno.land/x/fcgi@v1.0.3/mod.ts';

const app = new Application;

const DOCUMENT_ROOT = '/var/www/...';

app.use
(   async ctx =>
    {   const resp = await fcgi.fetch
        (   {   addr: '...',
                scriptFilename: DOCUMENT_ROOT+ctx.request.url.pathname
            },
            new Request
            (   ctx.request.url.href,
                {   method: ctx.request.method,
                    headers: ctx.request.headers,
                    body: ctx.request.hasBody ? ctx.request.body({type: 'stream'}).value : undefined,
                }
            )
        );
        ctx.response.status = resp.status;
        ctx.response.headers = resp.headers;
        ctx.response.body = resp.body;
    }
);

app.listen('localhost:8123');

[shah]

Yes, @jeremiah-shaulov you got the gist - the example above is perfect, I'll give it a shot and see how it goes. Thanks!

[shah]

@jeremiah-shaulov I tried using the following:

  import * as oak from "https://deno.land/x/oak@v9.0.1/mod.ts";
  import * as FCGI from "https://deno.land/x/fcgi@v1.0.3/mod.ts";

  const router = new oak.Router();
  const staticRoot = `${Deno.cwd()}/public`;

  // error handler
  app.use(async (_ctx, next) => {
    try {
      await next();
    } catch (err) {
      console.log(err);
    }
  });

  router.get("/fcgi/experiment.pl", async (ctx) => {
    const resp = await FCGI.fcgi.fetch(
      {
        addr: "...",
        scriptFilename: path.join(staticRoot, "fcgi/experiment.pl"),
      },
      new Request(ctx.request.url.href, {
        method: ctx.request.method,
        headers: ctx.request.headers,
        body: ctx.request.body({ type: "stream" }).value,
      }),
    );
    ctx.response.status = resp.status;
    ctx.response.headers = resp.headers;
    ctx.response.body = resp.body;
  });

I have a simple Perl script for testing in public/fcgi/experiment.pl which contains the following:

#!/usr/bin/env perl
use strict;
use warnings;
use FCGI;

my $r = FCGI::Request();
while($r->Accept() >= 0) {
    print "Content-Type: text/plain\n\n";
    print "Hello, FCGI world!\n";
}

I'm not sure what to pass in at addr: "..." in FCGI.fcgi.fetch({ addr: "...", }. Since I have Oak's ctx.app can I use something from there or does ... mean something special? :-)

[jeremiah-shaulov]

I didn't touch Perl for years. As i understand, you need fcgiwrap that will listen on some socket.

See here how they set up Nginx. What appears in "fastcgi_pass" directive in Nginx config - this should go to addr: '...', parameter in case of deno_fcgi.

But maybe look at different FastCGI server solutions. I don't believe that Perl can offer you asynchronous server.

The deno_fcgi library also allows you to create FastCGI server.

Example:

// File: backend.ts

import {fcgi} from 'https://deno.land/x/fcgi@v1.0.3/mod.ts';

fcgi.listen
(   'localhost:9988',
    '',
    async request =>
    {   request.responseHeaders.set('Content-Type', 'text/plain');
        await request.respond({body: 'Hello world!'});
    }
);

// File: frontend.ts

import {Application} from "https://deno.land/x/oak@v9.0.1/mod.ts";
import {fcgi} from 'https://deno.land/x/fcgi@v1.0.3/mod.ts';

const app = new Application;

app.use
(   async ctx =>
    {   const resp = await fcgi.fetch
        (   {   addr: 'localhost:9988',
            },
            new Request
            (   ctx.request.url.href,
                {   method: ctx.request.method,
                    headers: ctx.request.headers,
                    body: ctx.request.hasBody ? ctx.request.body({type: 'stream'}).value : undefined,
                }
            )
        );
        ctx.response.status = resp.status;
        ctx.response.headers = resp.headers;
        ctx.response.body = resp.body;
    }
);

app.listen('localhost:8123');

Run:

deno run --unstable --allow-all backend.ts &
deno run --unstable --allow-all frontend.ts

[shah]

Great, thanks for additional context and elaboration @jeremiah-shaulov - I'll give these a shot today!

[jeremiah-shaulov]

You're welcome, friend

[shah]

Your suggestions worked great, I was missing the fact that the deno_fcgi wasn't auto-starting the socket-based executable on its own (which is what Apache, NGINX, etc. do). Once I started the FCI service on a port that worked but for some reason I could not get it to work on a socket.

If you're so inclined you might want to accept as options a callback that could start the executable if it's not already started, something like this (see onNotStarted):

const resp = await fcgi.fetch
        (   {   addr: 'localhost:9988',
                                onNotStarted: () => { 
                                    Deno.run({ cmd: `${staticRoot}/fcgi/experiment.pl` }); return true; // suggest retry, return false to error out
                                },
            },
            new Request
            (   ctx.request.url.href,
                {   method: ctx.request.method,
                    headers: ctx.request.headers,
                    body: ctx.request.hasBody ? ctx.request.body({type: 'stream'}).value : undefined,
                }
            )
        );

If you like the idea above, please reopen otherwise we can close it since what I need is available! :-)

[shah]

@jeremiah-shaulov I just saw that V2.0 was released, nice work! I was curious if there was any feature like I needed above (to auto-start an FCGI executable if it's not found) in the new release. :-)

[jeremiah-shaulov]

Hi. Maybe it would be useful to add process manager to this library. However what do you think about such solution: to use spawn-fcgi. I added usage example that explains how it can be used: https://github.com/jeremiah-shaulov/deno_fcgi/tree/main/examples/Deno%20%E2%86%92%20spawn-fcgi%20+%20Perl

[shah]

Great idea @jeremiah-shaulov and spawn-fcgi is something we're already using in some projects. The reason I mentioned integration into deno_fcgi is so that spawning will only be done if someone hits that service and no resources would be consumed if it wasn't hit.

Something like this would be useful:

const resp = await fcgi.fetch({ 
    addr: 'localhost:9988',
    onNotStarted: () => { 
        Deno.run({ cmd: `sudo spawn-fcgi -a 127.0.0.1 -p 9990 -F 4 -u $USER -g $USER -- /usr/bin/perl -wT $PWD/service.pl` }); return true; // suggest retry, return false to error out
    },
}...

In our use cases we have small microservices that should only be launched when needed and they go away when not needed. :-)