Way to block High-Level Maturity Sites?

[ghost]

I don't want people to view mature content.

[nfriedly]

Yes, you can do that. I just added two examples, a blacklist one and a whitelist one.

For the blacklist, you give it a list of the domains that you don't want people to visit, and it will block those.

For the whitelist example, you give it a list of the domains you do want people to visit, and it will block everything else.

It should be pretty straightforward to copy the code from one or the other over to the nodeunblocker.com example; let me know if you get stuck.

I think the whitelist is probably going to be the easier option, because there are a lot of mature websites and it's basically impossible to block all of them.

[ghost]

Yup, sounds like whitelisting is a whole lot easier thank you!

[ghost]

I placed them in the home directory, however Heroku takes a long time to load just to give me Error In Application Use Heroku —tail to view logs

Any help would be appreciated

[ghost]

[nfriedly]

Hum.. are you able to run that command to get the logs? It would help narrow down the problem.

[ghost]

I don’t have Heroku CLI Available. What do I do?

[ghost]

I placed / overwrited all the files in the whitelist folder to the root directory

[emilhem]

I don’t have Heroku CLI Available. What do I do?

You have to get the logs or we can't help at all.

Click that "check your logs for details" link for more help.

Just get the Heroku CLI...

[emilhem]

What if shows personal info of my account.

That is your problem not ours. Just filter out any data that you don't want the public to see.

No logs, no help.

[emilhem]

Finding what's wrong with your setup/coding/config/environment is like finding a needle in an infinitely large pitch black room unless you share the logs.

[nfriedly]

I placed / overwrited all the files in the whitelist folder to the root directory

I think that overwrited part was the issue. Copying over whitelist.js is good, but you'll need to combine https://github.com/nfriedly/node-unblocker/blob/master/examples/whitelist/server.js and https://github.com/nfriedly/nodeunblocker.com/blob/master/app.js

Specifically, you'll need to add this to app.js, somewhere near the beginning of the file:

https://github.com/nfriedly/node-unblocker/blob/5fe35200534e2f35b7b76adf42444a794b05b2d9/examples/whitelist/server.js#L5

and add this: https://github.com/nfriedly/node-unblocker/blob/17d76bfef8316da2fe7072ffbf2371ebb9bca80f/examples/whitelist/server.js#L10-L15

into the configuration here:

https://github.com/nfriedly/nodeunblocker.com/blob/fd0da6a82c09effd56f8595fe9e7fb993f38d20f/app.js#L56-L57

Presumably with your own choice of domains to whitelist.

[ghost]

Thank you :)

[nfriedly]

Also, you'll want to undo anything you overwrote.

[ghost]

I’ll just delete the fork and do it again

[ghost]

Well not sure if it worked or not but I’ll try to install Heroku CLI, the reason I couldn’t is some requirement problems, sorry

[IceHacks]

@real-cereal-boy yeah surviv.io is fun but adult sites aren't

[ghost]

Well not sure if it worked or not but I’ll try to install Heroku CLI, the reason I couldn’t is some requirement problems, sorry

How on earth did you get surviv.io to work?

[yodaluca23]

@nfriedly I am trying to set up the blacklist, I put the files from the example into the root directory, except for the ones that already had a file named that, for those, I merged it... But, what is the format for the code to have the websites blocked? In the server.js? Would it be like this?

const unblocker = Unblocker({
  requestMiddleware: [
    blacklist({
      blockedDomains: ["examplesite1.com, examplesite2.com, examplesite3.com"],
      message: "The requested url is not permitted.",
    }),
  ],
});

Or,

const unblocker = Unblocker({
  requestMiddleware: [
    blacklist({
      blockedDomains: ["examplesite1.com"], ["examplesite2.com"], ["examplesite3.com"],
      message: "The requested url is not permitted.",
    }),
  ],
});

Or is it neither? Also, I tried it with just one site, and I got the following deployment error

2023-02-06T17:19:36.696Z: Setting environment variables:
  GA_ID="********************"
  NEW_RELIC_LICENSE_KEY="********************"
  VCAP_APP_PORT="********************"
2023-02-06T17:19:36.563Z: [CYCLIC] cached, updating...
2023-02-06T17:19:36.770Z: From https://github.com/yodaluca23/Unblocker_2.0
 * branch            01a711592e6f1d7bc3a6ebf9fe87eb61b4b049aa -> FETCH_HEAD
2023-02-06T17:19:36.774Z: Previous HEAD position was 3503430 Create package-lock.json
2023-02-06T17:19:36.774Z: HEAD is now at 01a7115 Create server.js
2023-02-06T17:19:36.783Z: [CYCLIC] Building...
2023-02-06T17:19:36.803Z: Build Configuration:
  Root Path: /
  Output Path: /
  Static Site: false
  Runtime: nodejs16.x

2023-02-06T17:19:37.082Z: [CYCLIC] verifying...
2023-02-06T17:19:37.087Z: parse error: Expected separator between values at line 23, column 3
2023-02-06T17:19:37.089Z: parse error: Expected separator between values at line 23, column 3
2023-02-06T17:19:37.150Z: [CYCLIC] using: node:v16.16.0 npm:10.1.0 runtime:nodejs16.x
[CYCLIC] building from: /
2023-02-06T17:19:37.150Z: [CYCLIC] installing dependencies from: package-lock.json
2023-02-06T17:19:37.496Z: npm
2023-02-06T17:19:37.496Z:  WARN config production Use `--omit=dev` instead.
2023-02-06T17:19:37.636Z: npm 
2023-02-06T17:19:37.637Z: ERR! code EJSONPARSE
npm ERR! path /tmp/yodaluca23-unblocker_2-0/package.json
2023-02-06T17:19:37.638Z: npm 
2023-02-06T17:19:37.638Z: ERR! JSON.parse
2023-02-06T17:19:37.638Z:  Unexpected token "{" (0x7B) in JSON at position 547 while parsing near "...ense\": \"AGPL-3.0\"\n  {\n  \"name\": \"simple-..."
npm ERR! JSON.parse Failed to parse JSON data.
npm ERR! JSON.parse
2023-02-06T17:19:37.638Z:  Note: package.json must be actual JSON, not just JavaScript.
2023-02-06T17:19:37.639Z: 
2023-02-06T17:19:37.640Z: npm ERR! A complete log of this run can be found in:
npm ERR!     /tmp/yodaluca23-unblocker_2-0/.npm/_logs/2023-02-06T17_19_37_471Z-debug-0.log
2023-02-06T17:19:37.648Z: [CYCLIC] Build Failed

Please help me, thanks!

[nfriedly]

None of that looks quite right. For the blocklist, I think this is the format you want:

const unblocker = Unblocker({
  requestMiddleware: [
    blacklist({
      blockedDomains: ["examplesite1.com", "examplesite2.com", "examplesite3.com"],
      message: "The requested url is not permitted.",
    }),
  ],
});

Each domain name should be in quotes ("), but there should be only one set of brackets ([ and ]).

The logs are reporting two different syntax errors, one in a package.json file, one somewhere else.

[yodaluca23]

The logs are reporting two different syntax errors, one in a package.json file, one somewhere else.

Ok, I think I'm doing something wrong, so I added package-lock.json, server.js, and blacklist.js to the root directory with no edits, except adding the websites we want in server.json... But then I am confused, how am I supposed to merge the package.json's?

Edit: I tried an online JSON merger, and used that, but I get this deployment error:

2023-02-06T19:13:09.399Z: [CYCLIC] cloning...
2023-02-06T19:13:09.519Z: Setting environment variables:
  GA_ID="********************"
  NEW_RELIC_LICENSE_KEY="********************"
  VCAP_APP_PORT="********************"
2023-02-06T19:13:09.690Z: From https://github.com/yodaluca23/Unblocker_2.0
 * branch            29be360c659b1048f9cd52c098e0250879a38153 -> FETCH_HEAD
2023-02-06T19:13:09.693Z: HEAD is now at 29be360 Update package.json
2023-02-06T19:13:09.705Z: [CYCLIC] Building...
2023-02-06T19:13:09.750Z: Build Configuration:
  Root Path: /
  Output Path: /
  Static Site: false
  Runtime: nodejs16.x

2023-02-06T19:13:10.022Z: [CYCLIC] verifying...
2023-02-06T19:13:10.358Z: [CYCLIC] using: node:v16.16.0 npm:10.1.0 runtime:nodejs16.x
[CYCLIC] building from: /
2023-02-06T19:13:10.361Z: [CYCLIC] installing dependencies from: package-lock.json
2023-02-06T19:13:10.853Z: npm
2023-02-06T19:13:10.855Z:  WARN config production Use `--omit=dev` instead.
2023-02-06T19:13:10.959Z: npm 
2023-02-06T19:13:10.961Z: ERR! code EUSAGE
2023-02-06T19:13:10.962Z: npm ERR! 
npm ERR! The `npm ci` command can only install with an existing package-lock.json or
npm ERR! npm-shrinkwrap.json with lockfileVersion >= 1. Run an install with npm@5 or
npm ERR! later to generate a package-lock.json file, then try again.
npm ERR! 
npm ERR! Clean install a project
npm ERR! 
npm ERR! Usage:
npm ERR! npm ci
npm ERR! 
npm ERR! Options:
npm ERR! [--no-audit] [--foreground-scripts] [--ignore-scripts]
npm ERR! [--script-shell <script-shell>]
npm ERR! 
npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
npm ERR! 
npm ERR! Run "npm help ci" for more info
2023-02-06T19:13:10.964Z: 
npm ERR! A complete log of this run can be found in:
npm ERR!     /tmp/yodaluca23-unblocker_2-0/.npm/_logs/2023-02-06T19_13_10_825Z-debug-0.log
2023-02-06T19:13:10.972Z: [CYCLIC] Build Failed

[nfriedly]

Looking at https://github.com/yodaluca23/Unblocker_2.0/, you already had a fine package.json - no need to merge in anything. And package-lock.json isn't really necessary as it can generate one automatically.

You also don't need to copy over the entire server.js, instead you need to copy

https://github.com/nfriedly/node-unblocker/blob/4b05977150f163aaef8607d519501b57bfba575b/examples/blacklist/server.js#L6

and

https://github.com/nfriedly/node-unblocker/blob/4b05977150f163aaef8607d519501b57bfba575b/examples/blacklist/server.js#L10-L17

into the existing app.js.

[yodaluca23]

Looking at https://github.com/yodaluca23/Unblocker_2.0/, you already had a fine package.json - no need to merge in anything. And package-lock.json isn't really necessary as it can generate one automatically.

You also don't need to copy over the entire server.js, instead you need to copy

https://github.com/nfriedly/node-unblocker/blob/4b05977150f163aaef8607d519501b57bfba575b/examples/blacklist/server.js#L6

and

https://github.com/nfriedly/node-unblocker/blob/4b05977150f163aaef8607d519501b57bfba575b/examples/blacklist/server.js#L10-L17

into the existing app.js.

I did as you describe, but I got this error in the logs, when I try to load the site (I'm sorry for using your time...):

2023-02-06 13:54:17.025: /var/task/app.js:68
var unblocker = new Unblocker({
    ^

SyntaxError: Identifier 'unblocker' has already been declared
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1033:15)
    at Module._compile (node:internal/modules/cjs/loader:1069:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/var/task/node_modules/gatling/gatling.js:95:15)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
2023-02-06 13:54:17.142: /var/task/app.js:68
var unblocker = new Unblocker({
    ^

SyntaxError: Identifier 'unblocker' has already been declared
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1033:15)
    at Module._compile (node:internal/modules/cjs/loader:1069:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/var/task/node_modules/gatling/gatling.js:95:15)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
/var/task/node_modules/gatling/gatling.js:29
    throw new Error("Too many instant deaths");
    ^

Error: Too many instant deaths
    at createWorker (/var/task/node_modules/gatling/gatling.js:29:11)
    at EventEmitter.<anonymous> (/var/task/node_modules/gatling/gatling.js:53:5)
    at EventEmitter.emit (node:events:527:28)
    at ChildProcess.<anonymous> (node:internal/cluster/primary:203:13)
    at Object.onceWrapper (node:events:642:26)
    at ChildProcess.emit (node:events:527:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)

ERROR: Failed to run "npm run start". Start script defined in package.json:

            "scripts": {
                "start":  "gatling app.js"
                 ...

Exited with code: 1
2023-02-06 13:54:17.326: /var/task/app.js:68
var unblocker = new Unblocker({
    ^

SyntaxError: Identifier 'unblocker' has already been declared
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1033:15)
    at Module._compile (node:internal/modules/cjs/loader:1069:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/var/task/node_modules/gatling/gatling.js:95:15)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)

[nfriedly]

Oh, sorry, I said copy, but I should have said merge. Specifically, you need to merge

https://github.com/yodaluca23/Unblocker_2.0/blob/fdcd1f22b913dbfdad96a5e9c7ba81a506cec7e9/app.js#L59-L66

and

https://github.com/yodaluca23/Unblocker_2.0/blob/fdcd1f22b913dbfdad96a5e9c7ba81a506cec7e9/app.js#L68-L76

into something like this:

 const unblocker = Unblocker({ 
    prefix: '/proxy/',
    requestMiddleware: [
        youtube.processRequest,
        blacklist({ 
           blockedDomains: ["example.com", "example2.com", "example3.com"], 
           message: "The requested url is not permitted.", 
        }), 
    ],
    responseMiddleware: [
        googleAnalyticsMiddleware
    ]
});

[yodaluca23]

Oh, sorry, I said copy, but I should have said merge. Specifically, you need to merge

https://github.com/yodaluca23/Unblocker_2.0/blob/fdcd1f22b913dbfdad96a5e9c7ba81a506cec7e9/app.js#L59-L66

and

https://github.com/yodaluca23/Unblocker_2.0/blob/fdcd1f22b913dbfdad96a5e9c7ba81a506cec7e9/app.js#L68-L76

into something like this:

 const unblocker = Unblocker({ 
    prefix: '/proxy/',
    requestMiddleware: [
        youtube.processRequest,
        blacklist({ 
           blockedDomains: ["example.com", "example2.com", "example3.com"], 
           message: "The requested url is not permitted.", 
        }), 
    ],
    responseMiddleware: [
        googleAnalyticsMiddleware
    ]
});

That works! Thank you so much!