Second-Hand-Friends / kleinanzeigen-bot

A dilligent command line tool to publish ads on kleinanzeigen.de
GNU Affero General Public License v3.0
216 stars 44 forks source link

IP got blocked! #152

Closed lamyergeier closed 1 year ago

lamyergeier commented 1 year ago

Any suggestions if it's possible to unblock the IP?

PhilK-7 commented 1 year ago

The same happened to me at some point. Are you by any chance using a VPN? If so, you could switch to another IP. If not, try another method e.g. a proxy. The website defense measures have gotten more aggressive in recent time

Bastilla123 commented 1 year ago

This has nothing to do with IP Address. I can with same credentials and on same device with same IP Login with Firefox and not with Google Chrome. I have feeling, that header is another. I try same with selenium firefox driver...

Can someone confirm that Google Chrome is blocked but Firefox works?

Bastilla123 commented 1 year ago

This is very strange when i use this bot i get everytime my IP are blocked when i manual login then i can login.... i solve captcha with bot everytime manual... so selenium makes it another then i am make manual ... strange

lamyergeier commented 1 year ago

This is what happens to me: when i use the bot it says that my ip is blocked but i can log in manually!

Lennix commented 1 year ago

It may be due to sec-ch-ua header not being set by chromium and thats checked by the firewall. I changed the user-agent to firefox and its working for me now.

- --user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:108.0) Gecko/20100101 Firefox/108.0 Add this under browser.arguments in the config.yaml

lamyergeier commented 1 year ago

@Lennix I am using Ubuntu 22.10.; Firefox version 108.0.2

From [What Is My User Agent? - WhatIsMyIP.com®](https://www.whatismyip.com/user-agent/), I found my User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) Gecko/20100101 Firefox/108.0

So I did as you did in your config file, but I get following error

[INFO] Logging to [/home/lamy/Documents/Mobile/Data/Task/Todo/Sales/kleinanzeigen-bot.log]...
[INFO] App version: 0.1.dev1+geca782c
[INFO] Loading config from [/home/lamy/opt/Kleinanzeigen/config.yaml]...
[INFO]  -> found 138 categories
[INFO] Searching for ad config files...
[INFO]  -> found 2 ad config files
[INFO] Loading ad from [/home/lamy/Desktop/Task/Todo/Sales/Keyboard/ad_Keyboard.yaml]...
[INFO] Loading ad from [/home/lamy/Desktop/Task/Todo/Sales/Macbook/ad_Laptop.yaml]...
[INFO] Loaded 2 ads
[INFO] Creating WebDriver session...
[INFO] Using Browser: CHROMIUM 109.0.5414 [/snap/bin/chromium]
[INFO]  -> Custom chrome argument: --user-agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) Gecko/20100101 Firefox/108.0
[INFO]  -> Custom chrome argument: --disable-dev-shm-usage
[INFO]  -> Custom chrome argument: --no-sandbox
[INFO]  -> Chrome binary location: /snap/bin/chromium
[ERROR] WebDriverException: Message: unknown error: Chrome failed to start: exited normally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /snap/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x55908c299303 <unknown>
#1 0x55908c06dd37 <unknown>
#2 0x55908c096157 <unknown>
#3 0x55908c092330 <unknown>
#4 0x55908c0d34a6 <unknown>
#5 0x55908c0ca753 <unknown>
#6 0x55908c09da14 <unknown>
#7 0x55908c09eb7e <unknown>
#8 0x55908c2e832e <unknown>
#9 0x55908c2ebc0e <unknown>
#10 0x55908c2ce610 <unknown>
#11 0x55908c2ecc23 <unknown>
#12 0x55908c2c0545 <unknown>
#13 0x55908c30d6a8 <unknown>
#14 0x55908c30d836 <unknown>
#15 0x55908c328d13 <unknown>
#16 0x7f3660490402 <unknown>

[369781] Failed to execute script '__main__' due to unhandled exception!
van-shorty commented 1 year ago

define in the config file: user_data_dir: "" # see https://github.com/chromium/chromium/blob/main/docs/user_data_dir.md profile_name: "" --> normaly default

then chrome opens with the default profile as you use when starting chrome from windows, then everything works. the problem is, that every time you start the tool chrome create a new session profile and then the same ip sends multiple session requests to ebay kleinanzeigen and that will be blocked.

lamyergeier commented 1 year ago
In the IP range 2001:a61:3a3a:2602:9e3e:4e47:3bd2:a062 there have recently been several unsafe attempts to use our platform.

This may also have been attempted by other people. Therefore, to prevent fraud, this IP range has been temporarily excluded from the use of eBay classifieds. Please just try again later.

More information: IP range blocked on eBay classifieds

Ref#: 0.8ba02417.1677842740.7c2888b2 IP#: 2001:a61:3a3a:2602:9e3e:4e47:3bd2:a062
jeromewh commented 1 year ago

I'm experiencing the same error. The suggested workarounds by @van-shorty and @Lennix did not work. Is there any information I can provide?

van-shorty commented 1 year ago

did you define this in your config?

binary_location: C:\Program Files\Google\Chrome\Application\chrome.exe

jeromewh commented 1 year ago

Weird... I tried it again (I just uncommented the lines from my previous tries) and now it works

Here is my current configuration:

browser:
  arguments:
    - --disable-dev-shm-usage
    - --no-sandbox
    - --user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:108.0) Gecko/20100101 Firefox/108.0
  binary_location: 'C:\Program Files\Google\Chrome\Application\chrome.exe'
  extensions: []
  use_private_window: true
  user_data_dir: 'C:\Users\username\AppData\Local\Google\Chrome\User Data'
  profile_name: "Default"

I'll post again if get any more errors.

Nuc1eoN commented 1 year ago

I got blocked after just a few tries, even though I was using --user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:108.0) Gecko/20100101 Firefox/108.0 as suggested.

Maybe the sleep times between request should be randomized so it looks more organic?

Nuc1eoN commented 1 year ago

Interesting. I have published an ad with kleinanzeigen-bot, and the ad got retroactively deleted, without notice. Only about one hour after publishing.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If the issue is still valid, please add a respective comment to prevent this issue from being closed automatically. Thank you for your contributions.