sachin-sankar / swiftshadow

Free IP Proxy rotator library for python
https://sachin-sankar.github.io/swiftshadow/
GNU General Public License v3.0
72 stars 5 forks source link

how to use #5

Closed unfairDude closed 4 months ago

unfairDude commented 1 year ago

hi, how to use this with proxychain or oxdork for example

sachin-sankar commented 1 year ago

Hello there currently there is no way to access the proxy list (validated and collected by swiftshadow) but that is something i can add right away.

But i need your help with this intergration.

Where exactly is the proxy list for proxychain located?

unfairDude commented 1 year ago

Hello there currently there is no way to access the proxy list (validated and collected by swiftshadow) but that is something i can add right away.

But i need your help with this intergration.

Where exactly is the proxy list for proxychain located?

its located in the proxychains4.conf file

sachin-sankar commented 1 year ago

is the directory where it is stored python accesible?

unfairDude commented 1 year ago

proxychain is a c project to route traffic using proxies.

Actually i came here because am using a python scraper and im getting the 429 error.

It would be cool if you can create a tool that use ur library, switching proxy on every rewuest

sachin-sankar commented 1 year ago

That is exactly what I am trying to create here. Can you run some test code in the same directory where the config is stored? Whoukd greatly help

sachin-sankar commented 1 year ago

run this code

with open('proxychains4.conf','r') as file:
    print(file.read())
ShrirajHegde commented 1 year ago

I don't think proxychains is the way to do it. AFAIK, Proxychain works as shim to programs compiled with libc and will require restart to swap out the proxy.

A better solution would be setting the environment variable all_proxy=proto://ip:port which is used by requests library.

Here's a proof of concept:

import requests
import time
from swiftshadow.swiftshadow import Proxy
import os

prox = Proxy(autoRotate=True)

from swiftshadow.swiftshadow import Proxy

print(requests.get("http://ip.me").text.strip())
while True:
    os.environ["all_proxy"] = f"http://{prox.proxy()['http']}"
    print(requests.get("http://ip.me").text.strip())
    time.sleep(1)

You can see the rotated IP being used by requests.

One of the side effects is, even proxychains starts making requests using the proxies, which shouldn't be a problem since the proxies are verified to be working anyway. Else proxychains might have to manually bypass proxy which doesn't seem to be a necessity.

Another solution is using proxies keyword arguement with requests:

import requests

proxies = {
  "http": "",
  "https": "",
}

requests.get("http://example.org", proxies=proxies)

But this will require adding this keyword arguments to function calls, which isn't really suitable while using libraries.

unfairDude commented 1 year ago

This is exactly what im talkin about

On Tue, May 2, 2023, 4:47 PM Shriraj Hegde @.***> wrote:

I don't think proxychains is the way to do it. AFAIK, Proxychain works as shim to programs compiled with libc and will require restart to swap out the proxy.

A better solution would be setting the environment variable all_proxy=proto://ip:port which is used by requests library.

Here's a proof of concept:

import requests import time from swiftshadow.swiftshadow import Proxy import os

prox = Proxy(autoRotate=True)

from swiftshadow.swiftshadow import Proxy

print(requests.get("http://ip.me").text.strip()) while True: os.environ["all_proxy"] = f"http://{prox.proxy()['http']}" print(requests.get("http://ip.me").text.strip()) time.sleep(1)

You can see the rotated IP being used by requests.

One of the side effects is, even proxychains starts making requests using the proxies, which shouldn't be a problem since the proxies are verified to be working anyway. Else proxychains might have to manually bypass proxy which doesn't seem to be a necessity.

Another solution is using proxies keyword arguement with requests:

import requests

proxies = { "http": "", "https": "", }

requests.get("http://example.org", proxies=proxies)

But this will require adding this keyword arguments to function calls, which isn't really suitable while using libraries.

— Reply to this email directly, view it on GitHub https://github.com/sachin-sankar/swiftshadow/issues/5#issuecomment-1531613828, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYPABN7PPE6XZ7SES6RCBQ3XEENBZANCNFSM6AAAAAAXMH3D6Q . You are receiving this because you authored the thread.Message ID: @.***>

sachin-sankar commented 1 year ago

Hey @ShrirajHegde thanks for the help.

After seeing this i am planning to add a new feature to set the environment variable when the proxy instance is created.

Example:

from swiftshadow.swiftshadow import Proxy

swfit = Proxy(setEnv=True,autoRotate=True)

This will set the environment variable once. But to rotate the proxy i am thinking we can implement some kind of a loop.

Also i read in this article that proxychains can use a list of proxies given in the .conf file. We can implement a function to edit this config file and add the lists too.

ShrirajHegde commented 1 year ago

Regarding proxychain config file, the default system file is at /etc/proxychains.conf and requires root permission to edit, which is not convenient.

But proxychains has a CLI option to specify the config file.

So the better option is to write a proxychains compatible config file in the working or home directory and use that with proxychains -f config_file.

sachin-sankar commented 1 year ago

Great idea. By any means do you have the default proxychains config file?

ShrirajHegde commented 1 year ago

It is here. proxychains.conf

The standard format is protocol ip port user pass

Some useful options to set would be

  1. random_chain so that a random proxy is selected among the available.
  2. chain_len = 1 so that multiple proxies are not used
  3. proxy_dns to resolve DNS at the proxy instead of local

It is important to remember that this proxychains not work on statically linked programs. Ex: Chromium based browsers and Go programs are statically linked.

sachin-sankar commented 1 year ago

Thanks i will implement a class to set this file upon initialisation with the proxies