wondering if there could support "Request HTTPS via Proxy"

[wangxi83]

module epxorted function has a parameter "options" that extends from "http".

noticed and confirmed from source code, that when pass parameter "url" and "options", "host and path" in "options" can take place with first parameter "url". it works well. so i wonder that - maybe i could pass options with "a Proxy Server's host and port" and "a Target's path" that worked fine in "node module http". just like: { host: "proxyserver", port: 1080, path: "http://some.host.needproxy", method:"GET", headers:{ Host:"host.needproxy" } } and it works!!

but, as i known , if i want to request a HTTPS via proxyserver, it could have 2 steps: 1,"connect" to 443 (or other) 2, get response use the socket tunnel. so when i try to do a https request via proxyserver, it didn't worked.

just wondering if there could support "Request HTTPS via Proxy" some day?

best regards

[wangxi83]

remote: Permission to denied to wangxi83. ^_^ just a happy path solution in attachment

this is only a happy path! but it can work

usage:

//from China that cannot access google.com (%_%)
//1. must speicify an url start with https
//2. need host to represent to the proxy server
//3. need a port represent to the proxy server
//4. no more others
 miniget("https://www.google.com", {
        host: '127.0.0.1',
        // proxy IP
        port: 1080,
        // proxy port
        method: 'GET'
    }).pipe(new Writer());

index.zip

[fent]

{ host: "proxyserver", port: 1080, path: "http://some.host.needproxy", method:"GET", headers:{ Host:"host.needproxy" } } and it works!!

but, as i known , if i want to request a HTTPS via proxyserver, it could have 2 steps: 1,"connect" to 443 (or other) 2, get response use the socket tunnel. so when i try to do a https request via proxyserver, it didn't worked.

Oh interesting. I didn't know https didn't work with the first method, and needed to connect. The attached file looks good, changes I'd make would basically be lint changes. I use eslint with these settings. Do you mind making a PR for it?

Otherwise I can try adding it in a few days, along with tests, and some documentation like an example or something in the readme.

[wangxi83]

Hi fent, it's great and pleased to receive your comment.

It's a nice idea that --- "Otherwise I can try adding it in a few days, along with tests, and some documentation like an example or something in the readme."

And, would you please explain some more about the "2nd Parameter"? It's because if some guy offer the parameter and give a 'path' field, it will certainly take place the "1st Parameter" which is "url". That could make some one like me confused, may be ^_^.

Any way, Thanks for your great project.

[fent]

And, would you please explain some more about the "2nd Parameter"? It's because if some guy offer the parameter and give a 'path' field, it will certainly take place the "1st Parameter" which is "url". That could make some one like me confused, may be ^_^.

Ah, good point. I wanted to give the most flexibility to this module with the 2nd parameter. I thought about making it like the http.request function

https://nodejs.org/api/http.html#http_http_request_options_callback

But the issue with that is, if you're not looking to modify any of the url's attributes (such as path), but instead want to customize another request option (like headers), you'll have to provide a url as a parsed object. And it's very common to have the url as a string.

let myurl = 'https://google.com';
let options = require('url').parse(myurl);
options.headers = { 'My-Header': '42' };
miniget(options);

One solution to this, is to allow for a url param in the options, that can be in the form of a string

let myurl = 'https://google.com';
let options = {
  url: myurl,
  headers: { 'My-Header': '42' },
};
miniget(options);

[wangxi83]

Sounds great.

In another way, it could be a way that, the miniget function could "make a jugdement" on 1st paramter. If it is in the form of string and URL, all things go on as current api (but the "path" in "options" will not be used or effective) , otherwise, 1st parameter is an Option object.

[fent]

Have you tried using https-proxy-agent?

I'm thinking it would be better to have users that want to use a proxy install that and use it as the agent. Rather than add the functionality to this module, and add onto the size of it. What do you think?