Support headless browser and webview to login to a site

[winterdl]

The v3 plugin system is great, I could add new site easily, but it is great if the core support more functions for crawling:

• Some sites using encrypted script to generate content, or they are using cloudflare to prevent bot, if we could using a headless browser, we could pass the block.

• Some sites protect their content behind membership (vip member), if we could open a webview for user login, after user loggedin to that site, we could share the cookie with script.

[gsioteam]

Ok, I have a headless browser implementation in other project, I will implement it in Kinoko.

The headless browser api will look like this:

let webView = new HiddenWebView({
      // Replace the resource in the web site.
      resourceReplacements: [{
          test:'jwplayer\.js',
          resource: this.loadString('my_jwplayer.js'),
          mimeType: 'text/javascript',
      }]
  });
  let cleanUp = () => {
      this.webView = null;
  }; 
  webView.load(src);
  // Receive message 
  webView.onmessage = (ev) => {
      let event = ev.event;
      let data = ev.data;
      switch (event) {
          case 'complete': {
              let items = [];
              for (let source of data.sources) {
                  items.push({
                      title: source.label,
                      url: source.file
                  });
              }
              resolve(items);
              cleanUp();
              break;
          }
      }
  };
  // Hold the reference otherwise the webView will be free before it callback.
  this.webView = webView;

[gsioteam]

The webview api is added in v4.1.0, I put the usage here.

[winterdl]

Thank you for your support, I will test the headless webview.