different proxy for different url's

[ofsahin]

Hello,

is it possible to use different proxies for different url's? set them up while queuing them?

i tried the options() method but i get curl error 35.

 'options'   =>  array(
 CURLOPT_USERAGENT   =>  'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36',
CURLOPT_SSL_VERIFYPEER   =>  0,
CURLOPT_SSL_VERIFYHOST   =>  0,
CURLOPT_PROXYTYPE   =>  $c['PROXY_TYPE'],
CURLOPT_PROXY   =>  $c['PROXY_HOST'],
 CURLOPT_PROXYPORT   =>  $c['PROXY_PORT'],
CURLOPT_PROXYUSERPWD   =>  $c['PROXY_USER'].":".$c['PROXY_PASS'],

i also tried to set the proxy to false and re-set it again.

$curl->proxy(false);
$curl->proxy($c["PROXY_HOST"], $c["PROXY_PORT"], $c["PROXY_USER"], $c["PROXY_PASS"]);

in this case, requests are made from the same proxy always.

[ofsahin]

i figured it for http connections. setting proxy type was the problem. just sending proxy ip port user and pass works. for https connections i still get CURLE_SSL_CONNECT_ERROR could it be using the same certifiicate to reach the same webpage from different proxies?

[stefangabos]

So you are saying that setting CURLOPT_PROXYTYPE results in an error, right?

As far as the errors related to https, downloading and using the CA bundle from Mozilla should fix the issues. Alternatively, disabling peer verification should also fix the issue. See the docs.

If you can provide me with a few proxies that I can test with, I will look into the issue with the CURLOPT_PROXYTYPE

[stefangabos]

i never used CURLOPT_PROXYTYPE so I just read the docs now and from what i understand fixing the ssl issues should fix the issues with specifying CURLOPT_PROXYTYPE

[stefangabos]

On how to use the CA bundle, have a look at any of the examples

[ofsahin]

ok. i think i figured most of it. there are two different websites with https. i use the script to check if proxies have access to certain websites. n11.com gittigidiyor.com

if i send requests through proxy one by one with my old script no problem. it works.

when i send 4 requests to each of them without proxy no problem with below code.

if i send the requests with proxy, n11.com gives ssl error. i am really not sure why.

<?php

// make sure error reporting is on
ini_set('display_errors', 1);  
error_reporting(E_ALL);

// make sure cache folder exists and is writable
if (!is_dir('cache') || !is_writable('cache')) trigger_error('the "cache" folder must be present and be writable in the "examples" folder', E_USER_ERROR);

// make sure CA bundle exists
elseif (!file_exists('cacert.pem')) trigger_error('"cacert.pem" file was not found', E_USER_ERROR);

$proxy="45.61.147.135:1520;user;8irrzDTDWSjk
88.198.69.215:2320;user;8irrzWSjk
95.46.198.25:19620;user;8irrzWSjk
95.46.198.26:23820;user;8irrzWSjk"

$ps = array();

if ($proxy) {
    $temp = explode("\n",$proxy);
    $i=0;
    foreach ($temp as $p) {
        //echo $key . $p . "\n";
        $split1   = explode(';',$p); // parameters
        $split2   = explode(':',$split1[0]); // Separate IP and port
        $ps[$i]['PROXY_HOST'] = trim($split2[0]); // Proxy server address
        $ps[$i]['PROXY_PORT'] = trim($split2[1]); // Proxy server port
        $ps[$i]['PROXY_USER'] = trim($split1[1]); // Username
        $ps[$i]['PROXY_PASS'] = trim($split1[2]); // Password
        $ps[$i]['PROXY_TYPE'] = "CURLPROXY_HTTP"); // Type
        $i++;
    }
}

// include the library
require './Zebra_cURL-master/Zebra_cURL.php';

// instantiate the Zebra_cURL class
$curl = new Zebra_cURL();
$curl->ssl(false);
$curl->queue();

foreach ($ps as $c) {

    //var_dump($c);
    $http_headers = array(
        'Pragma: no-cache',
        'Cache-Control: no-cache',
        'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36',
        'Accept-Language: en-US,en;q=0.7,tr;q=0.3',
    );

    $curl_params = array(
        // mandatory!
        'url'       =>  'https://www.n11.com',
        //'url'       =>  'https://www.gittigidiyor.com',

        // optional, used to set any cURL option
        // in the same way you would set with the options() method
        'options'   =>  array(
            //CURLOPT_USERAGENT   =>  'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36',
            CURLOPT_SSL_VERIFYPEER   =>  0,
            CURLOPT_SSL_VERIFYHOST   =>  0,
            //CURLOPT_PROXYTYPE   =>  $c['PROXY_TYPE'],
            //CURLOPT_PROXY   =>  $c['PROXY_HOST'],
            //CURLOPT_PROXYPORT   =>  $c['PROXY_PORT'],
            //CURLOPT_PROXYUSERPWD   =>  $c['PROXY_USER'].":".$c['PROXY_PASS'],
            CURLOPT_CONNECTTIMEOUT      =>  30,
            CURLOPT_TIMEOUT             =>  30,
            CURLOPT_HTTPHEADER   =>  $http_headers,
            CURLOPT_FOLLOWLOCATION   =>  1,
        ),

    );

    $curl->get($curl_params, function($result) use ($c) {
        // everything went well at cURL level
        if ($result->response[1] == CURLE_OK) {
            // if server responded with code 200 (meaning that everything went well)
            // see https://httpstatus.es/ for a list of possible response codes
            if ($result->info['http_code'] == 200) {
                if ($result->body !== '') {

                    print_r($c["PROXY_HOST"]);
                    echo " q: ";
                    //print_r($result->body);
                    print_r($result->info['original_url']);

                    echo "\n";
                    //$pos = strpos($result->body, "gittigidiyor.com");
                    $pos = strpos($result->body, "https://www.n11.com/destek-merkezi");

                    if (is_int($pos)) {
                        echo $c["PROXY_HOST"].":".$c["PROXY_PORT"].";".$c["PROXY_USER"].";".$c["PROXY_PASS"]."\n";
                    } else {
                        echo $c["PROXY_HOST"].":".$c["PROXY_PORT"].";".$c["PROXY_USER"].";".$c["PROXY_PASS"]." problem"."\n";
                        //echo $result->body;
                        //echo "a".$pos."a"."\n";
                    }

                    //var_dump($result);
                    //echo " a"."\n";                       
                } else {
                    echo "response body empty";
                }
                // show the server's response code
            } else {

                trigger_error('Server responded with code ' . $result->info['http_code'], E_USER_ERROR);
            }

        } else trigger_error('cURL responded with: ' . $result->response[0], E_USER_ERROR);

    }); 
}
print_r($curl);
// execute queued requests
$curl->start();

exit;

[ofsahin]

So you are saying that setting CURLOPT_PROXYTYPE results in an error, right?

As far as the errors related to https, downloading and using the CA bundle from Mozilla should fix the issues. Alternatively, disabling peer verification should also fix the issue. See the docs.

If you can provide me with a few proxies that I can test with, I will look into the issue with the CURLOPT_PROXYTYPE

i checked. CURLOPT_PROXYTYPE is not the problem

but it is ssl handshake related. with only this particular site.

CURLE_SSL_CONNECT_ERROR (35)

A problem occurred somewhere in the SSL/TLS handshake. You really want the error buffer and read the message there as it pinpoints the problem slightly more. Could be certificates (file formats, paths, permissions), passwords, and others.

[stefangabos]

I am unable to access those URLs - you're probably using a VPN Nevertheless, I googled it and one of the issues might be the used SSL version - see here; i'm not sure what the default is, but try using different values

[stefangabos]

this is another related one

[ofsahin]

I am unable to access those URLs - you're probably using a VPN Nevertheless, I googled it and one of the issues might be the used SSL version - see here; i'm not sure what the default is, but try using different values

ssl version solved it. thanks. also i have to say, clean code, amazing documentation. 👍

[stefangabos]

Awesome!