ERR_INVALID_URL with --spider flag

[nachtfunke]

Hello!

I am running into a weird problem - as soon as I use the --spider flag, I am getting errors:

GlyphHanger Fetch Error:  TypeError [ERR_INVALID_URL]: Invalid URL
    at new NodeError (node:internal/errors:371:5)
    at onParseError (node:internal/url:552:9)
    at new URL (node:internal/url:628:5)
    at normalizeUrl (/Users/user/.nvm/versions/node/v17.3.0/lib/node_modules/glyphhanger/node_modules/normalize-url/index.js:98:17)
    at SpiderPig.cleanupHref (/Users/user/.nvm/versions/node/v17.3.0/lib/node_modules/glyphhanger/node_modules/@zachleat/spider-pig/SpiderPig.js:22:10)
    at SpiderPig.fetchLocalUrls (/Users/user/.nvm/versions/node/v17.3.0/lib/node_modules/glyphhanger/node_modules/@zachleat/spider-pig/SpiderPig.js:118:16)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async MultipleSpiderPig.fetchUrls (/Users/user/.nvm/versions/node/v17.3.0/lib/node_modules/glyphhanger/src/MultipleUrlSpiderPig.js:50:17)
    at async /Users/user/.nvm/versions/node/v17.3.0/lib/node_modules/glyphhanger/cmd.js:88:5 {
  input: 'http:javascript:void(0);',
  code: 'ERR_INVALID_URL'
}

I thought it might have to do something with node 17, so I used nvm to switch to node 16, but the error persistent. I am not sure how to proceed here.

[zuzanasumlanska]

Hello, I have similar problem. The HTML page with telephone number will drop the error:

GlyphHanger Fetch Error:  TypeError [ERR_INVALID_URL]: Invalid URL: http:tel:+420731159973
    at onParseError (internal/url.js:258:9)
    at new URL (internal/url.js:334:5)
    at normalizeUrl (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/node_modules/normalize-url/index.js:98:17)
    at SpiderPig.cleanupHref (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/node_modules/@zachleat/spider-pig/SpiderPig.js:22:10)
    at SpiderPig.fetchLocalUrls (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/node_modules/@zachleat/spider-pig/SpiderPig.js:118:16)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async MultipleSpiderPig.fetchUrls (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/src/MultipleUrlSpiderPig.js:50:17)
    at async /Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/cmd.js:88:5 {
  input: 'http:tel:+420731159973',
  code: 'ERR_INVALID_URL'
}

Most likely the spider doesn't know how to handle links with telephone number or javascript in it.

[ali-baykal]

Hello, I have similar problem. The HTML page with telephone number will drop the error:

GlyphHanger Fetch Error:  TypeError [ERR_INVALID_URL]: Invalid URL: http:tel:+420731159973
    at onParseError (internal/url.js:258:9)
    at new URL (internal/url.js:334:5)
    at normalizeUrl (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/node_modules/normalize-url/index.js:98:17)
    at SpiderPig.cleanupHref (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/node_modules/@zachleat/spider-pig/SpiderPig.js:22:10)
    at SpiderPig.fetchLocalUrls (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/node_modules/@zachleat/spider-pig/SpiderPig.js:118:16)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async MultipleSpiderPig.fetchUrls (/Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/src/MultipleUrlSpiderPig.js:50:17)
    at async /Users/zuzanasumlanska/.nvm/versions/node/v12.22.12/lib/node_modules/glyphhanger/cmd.js:88:5 {
  input: 'http:tel:+420731159973',
  code: 'ERR_INVALID_URL'
}

Most likely the spider doesn't know how to handle links with telephone number or javascript in it.

I can confirm, that telephone links in the page cause glyphhanger to crash. I was about to open an issue for that.

[ShahriarKh]

Seems like this should be added to spider-pig: https://github.com/zachleat/spider-pig/blob/7d1e28d641891000c3b97550d31cc107cb0ea781/SpiderPig.js#L46-L52

Update: I opened a pull request https://github.com/zachleat/spider-pig/pull/9