modrzew / pokeminer

Pokemon location scraper
MIT License
220 stars 62 forks source link

AttributeError when trying to use a proxy (Windows 10) #272

Open tcrouch199205 opened 8 years ago

tcrouch199205 commented 8 years ago

I'm running into this issue after setting the proxy in the config;

Traceback (most recent call last):

    File "worker.py", line 420, in <module>

        spawn_workers(workers, status_bar=args.status_bar)

    File "worker.py", line 354, in spawn_workers

        start_worker(worker_no, points[worker_no])

    File "worker.py", line 339, in start_worker

        points=points

    File "worker.py", line 92, in __init__

        self.api.set_proxy(config.PROXIES)

    File "D:\Program Files\Python27\lib\site-packages\pgoapi\pgoapi.py", line 127, in __getattr__

        raise AttributeError

AttributeError

I've looked over the several posts related to proxies here already, and followed the instructions as they were given. I've run "pip install requests[shocks]", "pip install pysocks" and "pip install win_inet_pton". I've added "import win_inet_pton", and "import requests" to both the config.py and the pgoapi.py. I've tried using the changes listed HERE. I'm at a loss for what else I could be missing that would cause this issue.

Aiyubi commented 8 years ago

Did you run in it without the proxy and it was working?

This looks like there is an error in the way you wrote your proxy in the config.py

Since you use windows I recommend using notepad++ or similar to look at the file since it could be some encoding error and windows tools tend to be weird with encodings.

Also is there any special char like & in your url?

tcrouch199205 commented 8 years ago

It runs just fine without the proxy, and I've been using notepad++. I took the format that others have used, but perhaps I did make an error. Here's what I've got;

{'http': 'socks5://66.16.127:80', 'https': 'socks5://66.16.127:80'}

Aiyubi commented 8 years ago

obvious ones

tcrouch199205 commented 8 years ago

I've tried several IP's (of correct length, with a port included) to no avail, and my variable is called PROXIES.

Here's exactly what is in my config " PROXIES = {'http': 'socks5://96.27.57.234:10200', 'https': 'socks5://96.27.57.234:10200'}

leclaida commented 8 years ago

where is the pgoapi.py that you edited? paste the path here.

tcrouch199205 commented 8 years ago

D:\Program Files\pokeminer-0.5.4\src\pgoapi\pgoapi.py

leclaida commented 8 years ago

yeah so you can see in the error that the pgoapi that it is calling is here: D:\Program Files\Python27\lib\site-packages\pgoapi\pgoapi.py

this got installed from pip install -r requirements.txt

try to either: 1) update pgoapi with pip (by editing the requirements.txt file to include newer commit of pgoapi) or 2) go to your python folder and update the pgoapi.py there

lemme know if this works or if you need more help.

tcrouch199205 commented 8 years ago

I've updated the pgoapi within the python folder, and I'm getting this new error;

Traceback (most recent call last):
  File "worker.py", line 109, in run
    provider=service,
  File "D:\Program Files\Python27\lib\site-packages\pgoapi\src\pgoapi\pgoapi\pgoapi.py", line 181, in login
    response = self.app_simulation_login()
  File "D:\Program Files\Python27\lib\site-packages\pgoapi\src\pgoapi\pgoapi\pgoapi.py", line 158, in app_simulation_login
    response = request.call()
  File "D:\Program Files\Python27\lib\site-packages\pgoapi\src\pgoapi\pgoapi\pgoapi.py", line 241, in call
    response = request.request(self._api_endpoint, self._req_method_list, self.get_position())
  File "D:\Program Files\Python27\lib\site-packages\pgoapi\src\pgoapi\pgoapi\rpc_api.py", line 127, in request
    request_proto = self._build_main_request(subrequests, player_position)
  File "D:\Program Files\Python27\lib\site-packages\pgoapi\src\pgoapi\pgoapi\rpc_api.py", line 215, in _build_main_request
    for key in self.device_info:
TypeError: 'NoneType' object is not iterable
leclaida commented 8 years ago

haha i'm giving you more errors. now it likely won't run even w/o proxies.

so i can tell you updated the pgoapi to the latest because now it is looking for a device_info.

open up the new pgoapi.py (in D:\Program Files\Python27.....) and around line 71 you should see:

self.device_info = device_info

change this to

self.device_info = None

if that doesnt work try:

self.device_info = {}

that may or may not make it work again. to spoof an actual device, and 100% make it work again, this should set you up for spoofing a device:

https://github.com/pogodevorg/pgoapi/pull/108

an example from that thread:

at top of file, put:

import uuid

at line 71, where self.device_info = device_info WAS (delete it):

self.device_info = {} self.device_info['device_id'] = uuid.uuid4().hex self.device_info['device_brand'] = "Apple" self.device_info['device_model'] = "iPhone" self.device_info['device_model_boot'] = "iPhone8,2" self.device_info['hardware_manufacturer'] = "Apple" self.device_info['hardware_model'] = "N66AP" self.device_info['firmware_brand'] = "iPhone OS" self.device_info['firmware_type'] = "9.3.3"

EDIT: or if you dont want to do all of this, delete the pgoapi folder then change the requirements.txt in pokeminer to:

-e git+https://github.com/keyphact/pgoapi.git@8c1c17637be0aa679d92e582e6c4dd1370a3ac00#egg=pgoapi

tcrouch199205 commented 8 years ago

Using "self.device_info = {}" seems to have worked. Thanks for the help again!

leclaida commented 8 years ago

i edited my comment in case you wanted to have a look. right now you are sending blank device ids to the server.

tcrouch199205 commented 8 years ago

What's the reasoning for sending device ID's to the server. Just to make the accounts look less suspicious?

leclaida commented 8 years ago

yeah that is exactly why

tcrouch199205 commented 8 years ago

Alright, I've added those new lines and it's still working. Once again, thanks for the help.

tcrouch199205 commented 8 years ago

I've started having a new problem, where my workers often receive [LOGIN FAILED] while using a proxy now. I've tried 10+ different proxies and the problem persists through all of them. When I stop using proxies, the problem is non-existent. I haven't changed anything since I got the proxies working 2 days ago. Any ideas?

leclaida commented 8 years ago

can you paste the lines from your worker.log that pertain to this?

tcrouch199205 commented 8 years ago

[2016-08-27 13:09:30,119][ worker-15][ INFO][L 788] Starting new HTTPS connection (1): pgorelease.nianticlabs.com [2016-08-27 13:09:39,255][ worker-1][ INFO][L 277] Server seems to be busy or offline - try again! [2016-08-27 13:09:39,255][ worker-1][ INFO][L 284] Cleanup of request! [2016-08-27 13:09:39,255][ worker-1][ INFO][L 169] Finished RPC login sequence (app simulation) [2016-08-27 13:09:39,255][ worker-1][ INFO][L 197] Login failed!

leclaida commented 8 years ago

from what i can tell, you're not receiving any data when using your proxy. either the proxies you use are overloaded or they aren't configured correctly.

tcrouch199205 commented 8 years ago

So would the incorrect configuration be on my end, or the proxy's end?

leclaida commented 8 years ago

your end. can you load webpages using these proxies?

tcrouch199205 commented 8 years ago

Just tried, and none of them seem to be loading a webpage. I did have a few working proxies a few days ago, but those gave the same LOGIN FAILED the next day as well.

leclaida commented 8 years ago

sorry, but it looks like the proxies you use are either overloaded and not finishing requests or just not active anymore. try to find more working proxies and use those.

gunawanputra commented 8 years ago

run worker.py --log-level DEBUG to get more detail from log file. make sure you use socks proxy, not http proxy. Did all worket get the same error?

tcrouch199205 commented 8 years ago

I've got one proxy (sort of) working now. It takes about a minute for each scanner to successfully login, and then it begins the scan. However, they frequently display the LOGIN FAILED again for a bit and then resume scanning. They do this mid-cycle so I don't think it's just sleeping. I've been using this off and on since I found it, but I'm not sure if all of the points are getting scanned or not. All of the workers get the same error right off the bat, but then the LOGIN FAILED happens seemingly at random amongst them.

gunawanputra commented 8 years ago

There are a lot of free socks proxies you can find online, but you need to test them first. Have you read #252?

You'll never get your points 100% scanned with current worker.py. Simple server throttle on request is enough to make your worker restart and scan from the first point. Imagine if this happens when worker in 80% percentage