Want to create URL parsing Addon

[DragonOsman]

• Node.js Version: v11.6.0
• OS: Windows 10 Single Language Build 17134.472 Version 1804
• Scope (install, code, runtime, meta, other?): About building an Addon
• Module (and version) (if relevant):

I read the tutorial on C++ Addons on the Nodejs website and saw that it says any new Addons can use the Hello World example or the other example shown there as a starting point. But I still need some help in understanding how to do this.

Using this Addon code as a template:

// hello.cc
#include <node.h>

namespace demo {

using v8::FunctionCallbackInfo;
using v8::Isolate;
using v8::Local;
using v8::NewStringType;
using v8::Object;
using v8::String;
using v8::Value;

void Method(const FunctionCallbackInfo<Value>& args) {
  Isolate* isolate = args.GetIsolate();
  args.GetReturnValue().Set(String::NewFromUtf8(
      isolate, "world", NewStringType::kNormal).ToLocalChecked());
}

void Initialize(Local<Object> exports) {
  NODE_SET_METHOD(exports, "hello", Method);
}

NODE_MODULE(NODE_GYP_MODULE_NAME, Initialize)

}  // namespace demo

I want to export the urlparser function in this code as part of an Addon:

#include <iostream>
#include <string>
#include <vector>
#include <regex>

std::vector<std::string> parseurl(const std::string &url);

int main()
{
    std::string url{ "https://google.com/?q=cats" };
    std::vector result = parseurl(url);
    std::cout << "protocol: " << result[0] << '\n'
        << "host: " << result[1] << '\n'
        << "path: " << result[2] << '\n'
        << "query: " << result[3] << '\n'
        << "fragment: " << result[4] << '\n';
}

std::vector<std::string> parseurl(const std::string &url)
{
    unsigned counter = 0;
    std::string protocol, host, path, query, fragment;

    std::regex url_regex(
        R"(^(([^:\/?#]+):)?(//([^\/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?)",
        std::regex::extended
    );
    std::smatch url_match_result;

    if (std::regex_match(url, url_match_result, url_regex)) 
    {
        for (const auto &res : url_match_result) 
                {
            switch (counter)
            {
            case 2:
                protocol = res;
                break;
            case 4:
                host = res;
                break;
            case 5:
                path = res;
                break;
            case 7:
                query = res;
                break;
            case 9:
                fragment = res;
                break;
            }
            ++counter;
        }
    }
    else 
    {
        std::cerr << "Malformed url." << std::endl;
    }

    std::vector<std::string> result{ protocol, host, path, query, fragment };
    return result;
}

I found some code on StackOver flow and changed it a bit. First answer here, with 9 votes.

What I want to ask is: how should I change Method to have it do what I want? If I should use a completely different function, then please show me an example (if the example Addons that come with the Nodejs source code have something that can help me, then just point me to the right example)

[Hakerh400]

You need to convert std::string to v8::String and std::vector to v8::Object

C++

#include <iostream>
#include <string>
#include <vector>
#include <regex>
#include <node.h>

using namespace std;
using namespace v8;

vector<string> parseurl(const string &url) {
  // .....
}

void setVal(Isolate *iso, Local<Object> &obj, const char *prop, const char *val) {
  Local<Context> ctx = iso->GetCurrentContext();
  Local<String> strProp = String::NewFromUtf8(iso, prop);
  Local<String> strVal = String::NewFromUtf8(iso, val);
  obj->Set(ctx, strProp, strVal);
}

void parseurlWrap(const FunctionCallbackInfo<Value> &info) {
  Isolate *iso = info.GetIsolate(); // Get the v8 isolate object
  Local<Context> ctx = iso->GetCurrentContext(); // Current context

  // Make sure the arguments are valid
  if (info.Length() != 1 || !info[0]->IsString()) {
    Local<String> msg = String::NewFromUtf8(iso, "Invalid arguments");
    iso->ThrowException(Exception::TypeError(msg));
    return;
  }

  // Call the original function
  String::Utf8Value arg(iso, info[0]);
  string url = *arg;
  vector<string> parsed = parseurl(url);

  // Construct an object that can be accessed from JS
  Local<Object> obj = Object::New(iso);
  setVal(iso, obj, "protocol", parsed[0].c_str());
  setVal(iso, obj, "host", parsed[1].c_str());
  setVal(iso, obj, "path", parsed[2].c_str());
  setVal(iso, obj, "query", parsed[3].c_str());
  setVal(iso, obj, "fragment", parsed[4].c_str());

  // Return the JS object
  info.GetReturnValue().Set(obj);
}

void init(Local<Object> exports, Local<Object> module) {
  NODE_SET_METHOD(module, "exports", parseurlWrap);
}

NODE_MODULE(NODE_GYP_MODULE_NAME, init)

JavaScript

const parseurl = require('path/to/parseurl.node');

const url = 'http://example.com/page?name=value#tag';
const parsed = parseurl(url);

console.log(parsed);

Output

{ protocol: 'http',
  host: 'example.com',
  path: '/page',
  query: 'name=value',
  fragment: 'tag' }

[DragonOsman]

Thanks.

If it's not too much trouble, would you please point me to a place or resource where I can learn more about Addons and how to write code for them? A free resource would be good. And I'd also like documentation and explanation for each of the functions used here, like Set and String::NewFromUtf8.

And I just remembered that this also requires a binding.gyp file. What should the one for this Addon look like?

[ORESoftware]

I think URL parsing is faster to do in JS than in C/C++, because of the language boundary cost basically...if you needed to parse 1000+ URLs, then maybe send that to C++ for more speed, but 1 URL? JS is fine.

[DragonOsman]

I wanted to do this as an exercise for creating an Addon, so this should be fine.

Anyway, I have this in an earlier post, but I'll repeat it here just in case anyone missed it:

If it's not too much trouble, would you please point me to a place or resource where I can learn more about Addons and how to write code for them? A free resource would be good. And I'd also like documentation and explanation for each of the functions used here, like Set and String::NewFromUtf8.

And I just remembered that this also requires a binding.gyp file. What should the one for this Addon look like?

And I want to ask if binding.gyp files have compiler flags you can pass, and if it's possible to get it to use the C++17 standard when compiling the code.

Edit: I wrote this binding.gyp file and built the Addon:

{
    "targets": [
        {
            "target_name": "urlparser",
            "sources": [ "urlparser.cpp" ]
        }
    ]
}

But still, some help on more tutorials and documentation on writing Addons would be appreciated.

[DragonOsman]

I get an error when running the Node.js code:

E:\programming\Node_Addons\urlparser_addon>node parseurl_use.js
internal/modules/cjs/loader.js:750
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: A dynamic link library (DLL) initialization routine failed.
\\?\E:\programming\Node_Addons\urlparser_addon\build\Release\urlparser.node
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:750:18)
    at Module.load (internal/modules/cjs/loader.js:620:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
    at Function.Module._load (internal/modules/cjs/loader.js:552:3)
    at Module.require (internal/modules/cjs/loader.js:657:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (E:\programming\Node_Addons\urlparser_addon\parseurl_use.js:1:80)
    at Module._compile (internal/modules/cjs/loader.js:721:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:732:10)
    at Module.load (internal/modules/cjs/loader.js:620:32)

How do I fix this?

Edit to add: I ran it by doing node parseurl_use.js.

[Hakerh400]

I'd also like documentation and explanation for each of the functions used here, like Set and String::NewFromUtf8

All docs can be found here.

I get an error when running the Node.js code:

Make sure that you have rebuilt the addon. If you can't get it to work, download all files from here and then run node-gyp rebuild and node main.js after that to test the result.

[DragonOsman]

I got the Addon to build without any errors. I get the error when I try to run the Node.js code given above. When and why are you saying I should rebuild the Addon?

Thanks for the link. I'm using Node version 11.6.0, though, so I'll need to find documentation on that version.

And also, if the original C++ code had a main function that took the URL from a command-line argument, how would I do that on the Node.js side? Would the Addon code remain the same?

[Hakerh400]

I get the error when I try to run the Node.js code given above. When and why are you saying I should rebuild the Addon?

You should rebuild the addon every time you change Node.js version or binding.gyp file. The error you got suggests that your Node.js version is incompatible. Make sure that Node.js version and node-gyp version are synced. If the error persists, uninstall Node.js and node-gyp and install them again. Then rebuild the addon.

I'm using Node version 11.6.0, though, so I'll need to find documentation on that version.

The docs are the same. The only difference is that some functions are deprecated (and they are not used in this addon anyway).

And also, if the original C++ code had a main function that took the URL from a command-line argument, how would I do that on the Node.js side? Would the Addon code remain the same?

There is no main function in Node.js addons. You can't call the addon directly, it doesn't make sense. Please download the code from the linked repo, don't add main or other redundant functions.

[DragonOsman]

I'm not saying I'll add main, I just mentioned that to talk about a case involving command-line arguments. I'm just asking what to do in that case. If the code is used in a server app, and it gets its URL input from the browser, can be expressed through a command-line argument (and do JavaScript or Node.js apps take command-line arguments?)?

I didn't change my Node.js version or the binding.gyp file after building the Addon. I built Node.js from source and then installed node-gyp via NPM, and the one I built was Node.js v11.6.0. The error's talking about a DLL, and when I built Node.js, I also built the node.dll library by specifying the dll option when calling vcbuild. Should I not have done that?

And if there's a version discrepancy, could it be node-gyp's? How can I verify that?

[Hakerh400]

I don't know how well node-gyp works with home built Node.js and I recommend you to download the official Node.js installer and official node-gyp. Using custom Node.js build to build addons is advanced topic and you should not try that until you understand v8 to a decent level.

Do Node.js apps take command-line arguments

That's another thing you could find by googling. Yes, Node.js takes arguments and you can access them by process.argv. Then you can pass argument to the parseurl function, or you can access the arguments directly from C++ by accessing process object as a property of global and argv as a property of process

[DragonOsman]

I downloaded the MSI installer from the Nodejs website and installed it that way (version 11.6.0 of course), and then I built the Addon and ran the Node.js program using it and it worked like a charm. But there's still one problem. I set --msvs_version to 2017, but it still seems to be using 2015. I don't know why. With the Node.js I'd built, I could see it call MSBuild version 15.0 when I ran the node-gyp configuration command. But I didn't see that this time. I don't get it.

[DragonOsman]

I'll also say this here: I can't tell how I can build Zlib without also creating zlib.dll. And when building Node.js I have to link against that DLL when building the project file node.vcxproj. But it won't do that when I don't also specify the dll argument when running vcbuild. That gave me both node.lib and node.dll when I built Node.js. And as you can see in the error that was generated when I tried to run my Node.js app that uses the addon I built, it's talking about a DLL initialization routine failing. I probably need to find a way to get it to not fail (if possible). The error was Error: A dynamic link library (DLL) initialization routine failed.. So yeah, how do I fix that error? How do I get the DLL initialization routine to not fail?

[Qix-]

if you needed to parse 1000+ URLs, then maybe send that to C++

Send them to C++ in a batch is the key thing here. The language barrier cost for V8 is quite high, sadly.

[DragonOsman]

@Hakerh400 I opened a new about the DLL initialization routine failure issue here as well. With my home-built Node.js, which requires a DLL (node.dll) to be in the same directory as node.exe when I run said EXE, when I tried to run my Node.js application I got that error that said a dynamically linked library (DLL) initialization routine had failed. I don't think the problem is with understanding V8 in this case, since it seems like the main problem has to do with getting the DLL to initialize properly.

[Hakerh400]

Sorry, I'm not familiar with building addons using custom Node.js build. Try to ping someone from Node.js team or node-gyp team.

I think the DLL initialization error that you got is not related to node.dll, but to the addon itself (.node files are .dll files on Windows I think) and it is probably the result of version discrepancy between your Node.js build and node-gyp's internal library. If you have built Node.js from master, it is probably several commits ahead of v11.6.0, despite the version being displayed as same.

[DragonOsman]

I downloaded the source code for Node version 11.6.0 from the Nodejs website and built that with an MSI installer, and then used the installer to install it. So it couldn't be a version discrepancy.

Who should I ping? Give me some username suggestions, please.

[Qix-]

@DragonOsman there are ample resources for writing node modules. Look at nan, it's pretty much a must when working with Node.js addons.

[DragonOsman]

@Qix- I'll look into that and other resources I can find. Thanks. But for now, I just need to fix that DLL initialization routine failure error I'm having when I try to run a Node.js application using an addon I've built. This only happens when I try to use a home-built Node.js v11.6.0. I was thinking it could be a problem with node.dll (I have to build that with shared zlib and zlib.dll, or it won't build correctly), but as Hackerh400 said, it could also be the .node addon file itself since addons are treated as DLLs on Windows.

[Qix-]

.node is required, and yes they are DLLs with a different extension. Also, keep in mind that add-ons don't generally work across node versions. They have to be recompiled. There's no other way around this.

[DragonOsman]

@Qix- Again: I didn't use a different version. I used v11.6.0 for both the precompiled binary and also for the home-built one (I mentioned this in the post right above your first one here). And I actually did rebuild the addon even then, so yeah. My problem is only the DLL initialization routine failure issue that happens when I try to run a Node.js application that uses an addon and I'm using my home-built Node.js. The version isn't the problem because, again, I did download the source code for version 11.6.0 from the Node.js website.

[DragonOsman]

Who should I ping that can help me with using addons with home-built Node.js v11.6.0? I read that on Windows you have to add the path to DLLs you want to link against to the PATH variable. I did that just now. I don't know if it'll help with a .node file, though.

Edit for update: It really didn't help me. For reference, here's the error message again:

internal/modules/cjs/loader.js:750
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: A dynamic link library (DLL) initialization routine failed.
\\?\E:\programming\Node_Addons\urlparser_addon\build\Release\urlparser.node
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:750:18)
    at Module.load (internal/modules/cjs/loader.js:620:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
    at Function.Module._load (internal/modules/cjs/loader.js:552:3)
    at Module.require (internal/modules/cjs/loader.js:657:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (E:\programming\Node_Addons\urlparser_addon\parseurl_use.js:1:80)
    at Module._compile (internal/modules/cjs/loader.js:721:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:732:10)
    at Module.load (internal/modules/cjs/loader.js:620:32)

Again, I'm using a home-built v11.6.0. I downloaded the source code for v11.6.0 from the Node.js website. It requires a node.dll file which is there with node.exe in the same directory.

[gireeshpunathil]

@DragonOsman - I would ideally start with the sample addon and then build logic incrementally. In this case, I guess the key question is how to export new function. Please see if https://github.com/nodejs/help/issues/1662#issuecomment-502601082 helps you. Also, you don't need main function in the addon.

hope this helps

[gireeshpunathil]

ping @DragonOsman

[gireeshpunathil]

inactive, closing. please re-open if there is anything outstanding here