Closed tandomayo closed 2 years ago
If you ran 'main.py', try running 'flathunt.py' instead.
I ran "flathunt.py" instead......I think its working.
Here is my config.yaml file just to be sure:
# Enable verbose mode (print DEBUG log messages)
# verbose: true
# Should the bot endlessly looop through the URLs?
# Between each loop it waits for <sleeping_time> seconds.
# Note that Ebay will (temporarily) block your IP if you
# poll too often - don't lower this below 600 seconds if you
# are crawling Ebay.
loop:
active: yes
sleeping_time: 600
# Location of the Database to store already seen offerings
# Defaults to the current directory
#database_location: /path/to/database
# List the URLs containing your filter properties below.
# Currently supported services: www.immobilienscout24.de,
# www.immowelt.de, www.wg-gesucht.de, and www.ebay-kleinanzeigen.de.
# List the URLs in the following format:
# urls:
# - https://www.immobilienscout24.de/Suche/...
# - https://www.wg-gesucht.de/...
urls:
- https://www.ebay-kleinanzeigen.de/stadt/koeln/
# Define filters to exclude flats that don't meet your critera.
# Supported filters include 'max_rooms', 'min_rooms', 'max_size', 'min_size',
# 'max_price', 'min_price', and 'excluded_titles'.
#
# 'excluded_titles' takes a list of regex patterns that match against
# the title of the flat. Any matching titles will be excluded.
# More to Python regex here: https://docs.python.org/3/library/re.html
#
# Example:
# filters:
# excluded_titles:
# - "wg"
# - "zwischenmiete"
# min_price: 700
# max_price: 1000
# min_size: 50
# max_size: 80
# max_price_per_square: 1000
#filters:
# There are often city districts in the address which
# Google Maps does not like. Use this blacklist to remove
# districts from the search.
#blacklist:
# - Innenstadt
# If an expose includes an address, the bot is capable of
# displaying the distance and time to travel (duration) to
# some configured other addresses, for specific kinds of
# travel.
#
# Available kinds of travel ('gm_id') can be found in the
# Google Maps API documentation, but basically there are:
# - "bicycling"
# - "transit" (public transport)
# - "driving"
# - "walking"
#
# The example configuration below includes a place for
# "John", located at the main train station of munich.
# Two kinds of travel (bicycle and transit) are requested,
# each with a different label. Furthermore a place for
# "Jane" is included, located at the given destination and
# with the same kinds of travel.
#durations:
# - name: John
# destination: Hauptbahnhof, München
# modes:
# - gm_id: transit
# title: "Öff."
# - gm_id: bicycling
# title: "Rad"
# - name: Jane
# destination: Karlsplatz, München
# modes:
# - gm_id: transit
# title: "Öff."
# - gm_id: driving
# title: "Auto"
# Multiline message (yes, the | is supposed to be there),
# to format the message received from the Telegram bot.
#
# Available placeholders:
# - {title}: The title of the expose
# - {rooms}: Number of rooms
# - {price}: Price for the flat
# - {durations}: Durations calculated by GMaps, see above
# - {url}: URL to the expose
#message: |
# {title}
# Zimmer: {rooms}
# Größe: {size}
# Preis: {price}
# Ort: {address}
# {url}
# Calculating durations requires access to the Google Maps API.
# Below you can configure the URL to access the API, with placeholders.
# The URL should most probably just kept like that.
# To use the Google Maps API, an API key is required. You can obtain one
# without costs from the Google App Console (just google for it).
# Additionally, to enable the API calls in the code, set the 'enable' key to True
#google_maps_api:
# key: YOUR_API_KEY
# url: https://maps.googleapis.com/maps/api/distancematrix/json?origins={origin}&destinations={dest}&mode={mode}&sensor=true&key={key}&arrival_time={arrival}
# enable: False
# Register at 2captcha and enter your API key below. you will also
# have to install a Chrome Web Driver and write below the path to
# the executable.
#captcha:
# api_key: YOUR_API_KEY
# driver_path: YOUR_CHROME_DRIVER_PATH
# driver_arguments:
# - "--headless"
# Sending messages using Telegram requires a Telegram Bot configured.
# Telegram.org offers a good documentation about how to create a bot.
# Once you read it, will make sense. Still: bot_token should hold the
# access token of your bot and receiver_ids should list the client ids
# of receivers. Note that those receivers are required to already have
# started a conversation with your bot.
#
# telegram:
# bot_token: 160165XXXXXXX....
# receiver_ids:
# - 12345....
# - 67890....
telegram:
bot_token: //
receiver_ids: //
# If you are running the web interface, you can configure Login with Telegram support
# Follow the instructions here to register your domain with the Telegram bot:
# https://core.telegram.org/widgets/login
#
# website:
# bot_name: bot_name_xxx
# domain: flathunter.example.com
# session_key: SomeSecretValue
# listen:
# host: 127.0.0.1
# port: 8080
# If you are deploying to google cloud,
# uncomment this and set it to your project id. More info in the readme.
# google_cloud_project_id: my-flathunters-project-id
# For websites like idealista.it, there are anti-crawler measures that can be
# circumvented using proxies.
# use_proxy_list: True
You'll need to provide a Kleinanzeigen URL that shows you the listings that match your criteria. Right now, your URL (https://www.ebay-kleinanzeigen.de/stadt/koeln/
) only points to the landing page for Cologne. Could you try something like https://www.ebay-kleinanzeigen.de/s-wohnung-mieten/koeln/c203l945
and report whether you still run into issues?
So I changed it and now it doesn't work. @fkalbe :(
C:\Users\T\PycharmProjects\flathunter\venv\Scripts\python.exe C:/Users/T/PycharmProjects/flathunter/flathunt.py
[2021/10/19 18:45:43|config.py |INFO ]: Using config C:\Users\T\PycharmProjects\flathunter/config.yaml
Traceback (most recent call last):
File "C:/Users/T/PycharmProjects/flathunter/flathunt.py", line 89, in <module>
main()
File "C:/Users/T/PycharmProjects/flathunter/flathunt.py", line 86, in main
launch_flat_hunt(config)
File "C:/Users/T/PycharmProjects/flathunter/flathunt.py", line 46, in launch_flat_hunt
hunter.hunt_flats()
File "C:\Users\T\PycharmProjects\flathunter\flathunter\hunter.py", line 42, in hunt_flats
for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
File "C:\Users\T\PycharmProjects\flathunter\flathunter\sender_telegram.py", line 32, in process_expose
self.send_msg(message)
File "C:\Users\T\PycharmProjects\flathunter\flathunter\sender_telegram.py", line 39, in send_msg
for chat_id in self.receiver_ids:
TypeError: 'int' object is not iterable
Process finished with exit code 1
It looks like there might a problem with your config file. Have you made sure that the Telegram Receiver-IDs have been entered in the correct format? You could also post your config file here for us to take a look on. Just make sure that you censor any secrets/passwords by replacing a few characters with asterix.
@fkalbe - you were right! I changed the format of the receiver id and now the code runs but a new problem arises - I do not get any messages on my telegram bot.
Here is the config file
# Enable verbose mode (print DEBUG log messages)
# verbose: true
# Should the bot endlessly looop through the URLs?
# Between each loop it waits for <sleeping_time> seconds.
# Note that Ebay will (temporarily) block your IP if you
# poll too often - don't lower this below 600 seconds if you
# are crawling Ebay.
loop:
active: yes
sleeping_time: 700
# Location of the Database to store already seen offerings
# Defaults to the current directory
#database_location: /path/to/database
# List the URLs containing your filter properties below.
# Currently supported services: www.immobilienscout24.de,
# www.immowelt.de, www.wg-gesucht.de, and www.ebay-kleinanzeigen.de.
# List the URLs in the following format:
# urls:
# - https://www.immobilienscout24.de/Suche/...
# - https://www.wg-gesucht.de/...
urls:
- https://www.ebay-kleinanzeigen.de/s-wohnung-mieten/koeln/c203l945
# Define filters to exclude flats that don't meet your critera.
# Supported filters include 'max_rooms', 'min_rooms', 'max_size', 'min_size',
# 'max_price', 'min_price', and 'excluded_titles'.
#
# 'excluded_titles' takes a list of regex patterns that match against
# the title of the flat. Any matching titles will be excluded.
# More to Python regex here: https://docs.python.org/3/library/re.html
#
# Example:
# filters:
# excluded_titles:
# - "wg"
# - "zwischenmiete"
# min_price: 700
# max_price: 1000
# min_size: 50
# max_size: 80
# max_price_per_square: 1000
#filters:
# There are often city districts in the address which
# Google Maps does not like. Use this blacklist to remove
# districts from the search.
#blacklist:
# - Innenstadt
# If an expose includes an address, the bot is capable of
# displaying the distance and time to travel (duration) to
# some configured other addresses, for specific kinds of
# travel.
#
# Available kinds of travel ('gm_id') can be found in the
# Google Maps API documentation, but basically there are:
# - "bicycling"
# - "transit" (public transport)
# - "driving"
# - "walking"
#
# The example configuration below includes a place for
# "John", located at the main train station of munich.
# Two kinds of travel (bicycle and transit) are requested,
# each with a different label. Furthermore a place for
# "Jane" is included, located at the given destination and
# with the same kinds of travel.
#durations:
# - name: John
# destination: Hauptbahnhof, München
# modes:
# - gm_id: transit
# title: "Öff."
# - gm_id: bicycling
# title: "Rad"
# - name: Jane
# destination: Karlsplatz, München
# modes:
# - gm_id: transit
# title: "Öff."
# - gm_id: driving
# title: "Auto"
# Multiline message (yes, the | is supposed to be there),
# to format the message received from the Telegram bot.
#
# Available placeholders:
# - {title}: The title of the expose
# - {rooms}: Number of rooms
# - {price}: Price for the flat
# - {durations}: Durations calculated by GMaps, see above
# - {url}: URL to the expose
#message: |
# {title}
# Zimmer: {rooms}
# Größe: {size}
# Preis: {price}
# Ort: {address}
# {url}
# Calculating durations requires access to the Google Maps API.
# Below you can configure the URL to access the API, with placeholders.
# The URL should most probably just kept like that.
# To use the Google Maps API, an API key is required. You can obtain one
# without costs from the Google App Console (just google for it).
# Additionally, to enable the API calls in the code, set the 'enable' key to True
#google_maps_api:
# key: YOUR_API_KEY
# url: https://maps.googleapis.com/maps/api/distancematrix/json?origins={origin}&destinations={dest}&mode={mode}&sensor=true&key={key}&arrival_time={arrival}
# enable: False
# Register at 2captcha and enter your API key below. you will also
# have to install a Chrome Web Driver and write below the path to
# the executable.
#captcha:
# api_key: YOUR_API_KEY
# driver_path: YOUR_CHROME_DRIVER_PATH
# driver_arguments:
# - "--headless"
# Sending messages using Telegram requires a Telegram Bot configured.
# Telegram.org offers a good documentation about how to create a bot.
# Once you read it, will make sense. Still: bot_token should hold the
# access token of your bot and receiver_ids should list the client ids
# of receivers. Note that those receivers are required to already have
# started a conversation with your bot.
#
# telegram:
# bot_token: 160165XXXXXXX....
# receiver_ids:
# - 12345....
# - 67890....
telegram:
bot_token: 21xxxxxxxxx
receiver_ids:
- 2xxxxxxx
# If you are running the web interface, you can configure Login with Telegram support
# Follow the instructions here to register your domain with the Telegram bot:
# https://core.telegram.org/widgets/login
#
# website:
# bot_name: bot_name_xxx
# domain: flathunter.example.com
# session_key: SomeSecretValue
# listen:
# host: 127.0.0.1
# port: 8080
# If you are deploying to google cloud,
# uncomment this and set it to your project id. More info in the readme.
# google_cloud_project_id: my-flathunters-project-id
# For websites like idealista.it, there are anti-crawler measures that can be
# circumvented using proxies.
# use_proxy_list: True
Have you added your bot to your telegram chats? If I remember correctly, you also need to message your bot first with /start
, otherwise it won't be able to send you anything. If this doesn't help, you could try to delete the processed_ids.db
file in the flathunter root directory to reset any already processed listings. Afterwards, restart the bot and check if this helped.
I had added my bot and started it. I tried what you suggested with the processed_ids but it still does not work. I know the code compiles as it shows statements like the following:
[2021/11/07 22:25:32|hunter.py |INFO ]: New offer: Suche Wg In Köln
But I just dont receive them on Telegram.
I suspect this might be an issue with Telegram, rather the bot not sending messages. To rule this out, you could use the Telegram API to send a test message using the following URL:
https://api.telegram.org/bot<YOUR_BOT_TOKEN>/sendMessage?chat_id=<YOUR_RECEIVER_ID>&parse_mode=Markdown&text=test
Simply replace <YOUR_BOT_TOKEN>
and <YOUR_RECEIVER_ID>
with your own information. Afterwards, copy the URL into the address bar of your browser and hit enter. Your browser should indicate whether the message was send successfully and you should receive the test message on your phone. If this doesn't work, then something is wrong with your the setup of your Telegram bot.
I tried what you said and the bot works. I got the test
message.
I also tried looking at my indentation and with pycharm it automatically takes care of it.
I am really lost as to why I am not getting these messages and kinda tired....been working on this code for 3 hours now and still no luck 😞
Sorry to hear that 😕 My next idea would be to run the crawler in verbose mode, which gives us additional debug information, including whether any messages were sent. In your config file, just uncomment the second line (by removing the #) so that it says:
[...]
verbose: true
[...]
Feel free to post the logs after running the bot again (but be careful because your API key might be in there).
API key? Is that what I am missing?
Have a look at the logs:
[2021/11/14 18:06:10|abstract_crawler.py|DEBUG ]: Got search URL https://www.ebay-kleinanzeigen.de/s-wohnung-mieten/koeln/c203l945
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50667 Köln Altstadt
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 75 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: Gesuch
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50933 Köln Braunsfeld
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 4 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 108 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51103 Kalk
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 45 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50996 Rodenkirchen
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 1 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 37,50 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50674 Köln Altstadt
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 85 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51103 Höhenberg
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 54 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51149 Porz
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 1 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 45 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50733 Nippes
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 70 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51145 Porz
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 1 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 67 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51103 Höhenberg
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 1 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 23 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50737 Weidenpesch
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50823 Ehrenfeld
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 86 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50674 Köln Altstadt
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 60 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50939 Köln Klettenberg
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 1 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 42 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50767 Heimersdorf
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 105 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50733 Nippes
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 70 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50767 Heimersdorf
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 80 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50735 Nippes
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 90 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50937 Köln Sülz
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 1 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50823 Ehrenfeld
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2,5 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 63 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50670 Köln Altstadt
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 4 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 90 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50999 Sürth
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 54 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50969 Zollstock
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 4 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 95,33 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 51069 Köln Dellbrück
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 63 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50931 Lindenthal
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 45 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50677 Köln Altstadt
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 2 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 50937 Köln Sülz
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 3 Zimmer
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: 95 m²
[2021/11/14 18:06:10|crawl_ebaykleinanzeigen.py|DEBUG ]: extracted: 27
[2021/11/14 18:06:10|abstract_crawler.py|DEBUG ]: Number of found entries: 27
[2021/11/14 18:06:10|idmaintainer.py |DEBUG ]: is_processed(1926412974)
[2021/11/14 18:06:10|idmaintainer.py |DEBUG ]: is_processed(1905268842)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932683476)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932672619)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932643498)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932643046)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932563690)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932518515)
[2021/11/14 18:06:11|idmaintainer.py |DEBUG ]: is_processed(1932400525)
[2021/11/14 18:06:12|idmaintainer.py |DEBUG ]: is_processed(1932380006)
[2021/11/14 18:06:12|idmaintainer.py |DEBUG ]: is_processed(1932301683)
[2021/11/14 18:06:12|idmaintainer.py |DEBUG ]: is_processed(1932271813)
[2021/11/14 18:06:12|idmaintainer.py |DEBUG ]: is_processed(1932216464)
[2021/11/14 18:06:12|idmaintainer.py |DEBUG ]: is_processed(1932166520)
[2021/11/14 18:06:12|idmaintainer.py |DEBUG ]: is_processed(1852770243)
[2021/11/14 18:06:13|idmaintainer.py |DEBUG ]: is_processed(1928137559)
[2021/11/14 18:06:13|idmaintainer.py |DEBUG ]: is_processed(1852733347)
[2021/11/14 18:06:13|idmaintainer.py |DEBUG ]: is_processed(1932101501)
[2021/11/14 18:06:13|idmaintainer.py |DEBUG ]: is_processed(1918686524)
[2021/11/14 18:06:13|idmaintainer.py |DEBUG ]: is_processed(1931997783)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931960166)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931916867)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931840734)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931837597)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931792927)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931718758)
[2021/11/14 18:06:14|idmaintainer.py |DEBUG ]: is_processed(1931665814)
Let me know if you can help.
By API-Key I meant your Telegram bot token. However, I don't think that's the issue right now. Instead, the crawler is not even trying to send any messages, because it has seen (and processed) all these listings before. Please try once again to delete your processed_ids.db
file, restart the crawler and post the debug logs.
Hey @fkalbe - I understood the config file was wrong and was able to get the code working.
Thank you so much for the support and help! I will keep you posted if anything goes wrong.
Very new to coding so I need a bit of help (using pycharm).
Tried running the code according to the instructions. I get an error saying: "C:\Users......... Traceback (most recent call last): File "C:/Users.......", line 30, in
listen = config['website']['listen']
File "C:\Users.......", line 43, in getitem
return self.config[value]
KeyError: 'website'
Process finished with exit code 1"
Trying to understand if I am putting in the website incorrectly?
"urls: