Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
Hello!
It so happened that I need to track the status of three accounts on the same site. The sensor settings are absolutely identical, except for the login/password pair. If I am working with only one account at a time, then the data is retrieved correctly. After I add the second and/or third sensor, all the added sensors get the same value, although they are different in my personal accounts.
I was able to combine all the accounts into one personal account, found a link to switch between accounts, but still all sensors get the same value. In other words, there is no difference whether I log into my personal account under one user and switch between accounts or log into my personal account under different credentials. As far as I understand, after reading the value from one account, multiscrape does not have time to log out of the personal account, tries to log in under another account, remaining in the personal account of the previous account and reads data from it.
I'm sorry if I didn't write it clearly.
Debug log
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape] КрасноярскЭнергоСбыт # Setting up multiscrape with config:
{'resource': 'https://lkfl.krsk-sbit.ru/index.php?setcurrentaccount&account=103113674650', 'name': 'КрасноярскЭнергоСбыт', 'scan_interval': datetime.timedelta(seconds=10800), 'log_response': True, 'form_submit': {'resource': 'https://lkfl.krsk-sbit.ru/login', 'submit_once': True, 'method': 'POST', 'select': 'form', 'input': {'login': 'admin', 'password': '***'}, 'input_filter': [], 'verify_ssl': True, 'timeout': 10, 'resubmit_on_error': True, 'variables': []}, 'sensor': [{'unique_id': '43ed0650-d03b-458c-bb0b-aa5e11a75a1d', 'name': 'КрасноярскЭнергоСбыт', 'force_update': True, 'device_class': <SensorDeviceClass.MONETARY: 'monetary'>, 'unit_of_measurement': 'RUB', 'select': Template<template=(.user-page__score-int) renders=0>, 'value_template': Template<template=({{ (value.split(" ")[0] | float) if value is search("\d+") else None }}) renders=0>, 'on_error': {'log': 'warning', 'value': 'none'}, 'extract': 'text'}], 'list_separator': ',', 'method': 'GET', 'timeout': 10, 'verify_ssl': True, 'parser': 'lxml'}
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.file] КрасноярскЭнергоСбыт # Log responses enabled, creating logging folder: /config/multiscrape/krasnoiarskenergosbyt/
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Initializing http wrapper
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Initializing form submitter
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Initializing http wrapper
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Creating scraper
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Initializing scraper
2024-10-21 11:16:24.129 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Creating ContentRequestManager
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Creating coordinator
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Scan interval is 3:00:00
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.service] КрасноярскЭнергоСбыт # Setting up multiscrape configuration level services
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape] КрасноярскЭнергоСбыт # Setting up multiscrape with config:
{'resource': 'https://lkfl.krsk-sbit.ru/index.php?setcurrentaccount&account=133500912060', 'name': 'КрасноярскЭнергоСбыт', 'scan_interval': datetime.timedelta(seconds=10800), 'log_response': True, 'form_submit': {'resource': 'https://lkfl.krsk-sbit.ru/login', 'submit_once': True, 'method': 'POST', 'select': 'form', 'input': {'login': 'admin', 'password': '***'}, 'input_filter': [], 'verify_ssl': True, 'timeout': 10, 'resubmit_on_error': True, 'variables': []}, 'sensor': [{'unique_id': '142ba7bd-6d8f-435c-803b-2ddc8b55710a', 'name': 'КрасноярскЭнергоСбыт', 'force_update': True, 'device_class': <SensorDeviceClass.MONETARY: 'monetary'>, 'unit_of_measurement': 'RUB', 'select': Template<template=(.user-page__score-int) renders=0>, 'value_template': Template<template=({{ (value.split(" ")[0] | float) if value is search("\d+") else None }}) renders=0>, 'on_error': {'log': 'warning', 'value': 'none'}, 'extract': 'text'}], 'list_separator': ',', 'method': 'GET', 'timeout': 10, 'verify_ssl': True, 'parser': 'lxml'}
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.file] КрасноярскЭнергоСбыт # Log responses enabled, creating logging folder: /config/multiscrape/krasnoiarskenergosbyt/
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Initializing http wrapper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Initializing form submitter
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Initializing http wrapper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Creating scraper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Initializing scraper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Creating ContentRequestManager
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Creating coordinator
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Scan interval is 3:00:00
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.service] КрасноярскЭнергоСбыт # Setting up multiscrape configuration level services
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape] КрасноярскЭнергоСбыт # Setting up multiscrape with config:
{'resource': 'https://lkfl.krsk-sbit.ru/index.php?setcurrentaccount&account=102102001480', 'name': 'КрасноярскЭнергоСбыт', 'scan_interval': datetime.timedelta(seconds=10800), 'log_response': True, 'form_submit': {'resource': 'https://lkfl.krsk-sbit.ru/login', 'submit_once': True, 'method': 'POST', 'select': 'form', 'input': {'login': 'admin', 'password': '***'}, 'input_filter': [], 'verify_ssl': True, 'timeout': 10, 'resubmit_on_error': True, 'variables': []}, 'sensor': [{'unique_id': 'f9c48ae1-d65e-4a29-97a7-c38bb87e6ceb', 'name': 'КрасноярскЭнергоСбыт', 'force_update': True, 'device_class': <SensorDeviceClass.MONETARY: 'monetary'>, 'unit_of_measurement': 'RUB', 'select': Template<template=(.user-page__score-int) renders=0>, 'value_template': Template<template=({{ (value.split(" ")[0] | float) if value is search("\d+") else None }}) renders=0>, 'on_error': {'log': 'warning', 'value': 'none'}, 'extract': 'text'}], 'list_separator': ',', 'method': 'GET', 'timeout': 10, 'verify_ssl': True, 'parser': 'lxml'}
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.file] КрасноярскЭнергоСбыт # Log responses enabled, creating logging folder: /config/multiscrape/krasnoiarskenergosbyt/
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Initializing http wrapper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Initializing form submitter
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Initializing http wrapper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Creating scraper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Initializing scraper
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Creating ContentRequestManager
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Creating coordinator
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Scan interval is 3:00:00
2024-10-21 11:16:24.130 DEBUG (MainThread) [custom_components.multiscrape.service] КрасноярскЭнергоСбыт # Setting up multiscrape configuration level services
2024-10-21 11:16:25.606 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Setting up sensor
2024-10-21 11:16:25.606 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Added sensor to HA
2024-10-21 11:16:25.606 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Restoring previous state: 0.0
2024-10-21 11:16:25.606 INFO (MainThread) [homeassistant.components.sensor] Setting up multiscrape.sensor
2024-10-21 11:16:25.607 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Setting up sensor
2024-10-21 11:16:25.607 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Added sensor to HA
2024-10-21 11:16:25.607 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Restoring previous state: 0.0
2024-10-21 11:16:25.607 INFO (MainThread) [homeassistant.components.sensor] Setting up multiscrape.sensor
2024-10-21 11:16:25.607 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Setting up sensor
2024-10-21 11:16:25.607 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Added sensor to HA
2024-10-21 11:16:25.607 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Restoring previous state: 0.0
2024-10-21 11:16:36.954 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Home assistant started, triggering scrape on startup
2024-10-21 11:16:36.955 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # New run: start (re)loading data from resource
2024-10-21 11:16:36.955 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Deleting logging files from previous run
2024-10-21 11:16:36.955 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Home assistant started, triggering scrape on startup
2024-10-21 11:16:36.955 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # New run: start (re)loading data from resource
2024-10-21 11:16:36.955 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Deleting logging files from previous run
2024-10-21 11:16:36.956 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Home assistant started, triggering scrape on startup
2024-10-21 11:16:36.956 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # New run: start (re)loading data from resource
2024-10-21 11:16:36.956 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Deleting logging files from previous run
2024-10-21 11:16:37.477 ERROR (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Error deleting files from previous run: [Errno 2] No such file or directory: '/config/multiscrape/krasnoiarskenergosbyt/form_page_request_headers.txt'
2024-10-21 11:16:37.477 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Starting with form-submit
2024-10-21 11:16:37.477 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Requesting page with form from: https://lkfl.krsk-sbit.ru/login
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing form_page-request with a GET to url: https://lkfl.krsk-sbit.ru/login with headers: {} and cookies: None.
2024-10-21 11:16:37.478 ERROR (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Error deleting files from previous run: [Errno 2] No such file or directory: '/config/multiscrape/krasnoiarskenergosbyt/form_page_soup.txt'
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Starting with form-submit
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Requesting page with form from: https://lkfl.krsk-sbit.ru/login
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing form_page-request with a GET to url: https://lkfl.krsk-sbit.ru/login with headers: {} and cookies: None.
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Starting with form-submit
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Requesting page with form from: https://lkfl.krsk-sbit.ru/login
2024-10-21 11:16:37.478 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing form_page-request with a GET to url: https://lkfl.krsk-sbit.ru/login with headers: {} and cookies: None.
2024-10-21 11:16:37.618 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: form_page_request_headers.txt
2024-10-21 11:16:37.618 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: form_page_request_headers.txt
2024-10-21 11:16:37.622 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: form_page_request_headers.txt
2024-10-21 11:16:38.852 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:39.297 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: form_page_response_headers.txt
2024-10-21 11:16:39.297 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: form_page_response_cookies.txt
2024-10-21 11:16:39.297 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: form_page_response_body.txt
2024-10-21 11:16:39.369 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:39.404 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:39.416 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Parse page with form with BeautifulSoup parser lxml
2024-10-21 11:16:39.489 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: form_page_response_cookies.txt
2024-10-21 11:16:39.489 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: form_page_response_headers.txt
2024-10-21 11:16:39.489 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: form_page_response_headers.txt
2024-10-21 11:16:39.489 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: form_page_response_body.txt
2024-10-21 11:16:39.489 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: form_page_response_body.txt
2024-10-21 11:16:39.504 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: form_page_response_cookies.txt
2024-10-21 11:16:39.511 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # The page with the form parsed by BeautifulSoup has been written to file: form_page_soup.txt
2024-10-21 11:16:39.511 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Try to find form with selector form
2024-10-21 11:16:39.512 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Form looks like this:
<form action="https://lkfl.krsk-sbit.ru/login" method="post">
<div class="form-card">
<div class="form-group">
<label class="text__desc text__desc--small" for="login">Адрес электронной почты или номер мобильного телефона</label>
<input autocomplete="new-login" class="form-control ym-disable-keys" id="login" name="login" required="" type="text" value=""/>
<div class="invalid-feedback">
</div>
</div>
<div class="form-group password">
<label class="text__desc text__desc--small" for="password">Пароль</label>
<input autocomplete="new-password" class="form-control ym-disable-keys" id="password" name="password" required="" type="password" value=""/>
<a class="password-control" href="#"></a>
<div class="valid-feedback">
Текст ошибки
</div>
</div>
<div class="form-card__inner col-sm-4">
<div class="form-check form-check-inline">
<input class="form-check-input" id="remember" name="remember" type="checkbox" value="option1"/>
<label class="form-check-label" for="remember">Запомнить меня</label>
</div>
<button class="btn btn-send" type="submit">Войти</button>
</div>
<button class="btn btn-change restore-password" style="margin-right: 20px" type="button">Забыли пароль?</button>
<button class="btn btn-change registration" type="button">Зарегистрироваться</button>
</div>
<div class="reg-type">
<span class="reg-type__text">Или войти через соцсети:</span>
<ul class="reg-type__list">
<li>
<a class="reg-type__link" href="javascript:void(window.open('/index.php?route=extension/module/sociallogin/vk','_blank', 'width=700, height=400, top='+((screen.height-470)/2)+',left='+((screen.width-860)/2)+', resizable=yes, scrollbars=no, status=yes'))">
<!-- "source/partials/svg/vk-icon.svg" -->
<svg fill="none" height="30" viewbox="0 0 30 30" width="30" xmlns="http://www.w3.org/2000/svg">
<path d="M15 0C6.716 0 0 6.716 0 15c0 8.284 6.716 15 15 15 8.284 0 15-6.716 15-15 0-8.284-6.716-15-15-15zm7.609 16.622c.699.683 1.439 1.326 2.066 2.08.278.333.54.677.74 1.065.285.553.027 1.159-.467 1.192l-3.073-.001c-.794.066-1.426-.255-1.958-.797-.425-.433-.82-.894-1.229-1.342a2.871 2.871 0 00-.553-.492c-.42-.272-.783-.189-1.023.249-.245.445-.3.939-.324 1.434-.033.725-.252.914-.979.948-1.554.073-3.028-.163-4.398-.946-1.208-.69-2.144-1.666-2.959-2.77-1.587-2.152-2.803-4.514-3.895-6.944-.246-.547-.066-.84.538-.85a85.522 85.522 0 013.009-.002c.408.007.678.24.835.625.542 1.334 1.206 2.603 2.039 3.78.222.313.448.625.77.846.357.245.628.164.796-.233.107-.253.154-.523.177-.794.08-.927.09-1.854-.049-2.777-.086-.578-.41-.951-.987-1.06-.293-.056-.25-.165-.108-.332.248-.29.48-.47.944-.47h3.472c.547.107.67.353.744.904l.003 3.859c-.006.213.107.845.49.985.307.101.51-.145.693-.34.833-.883 1.426-1.926 1.957-3.005.235-.476.437-.969.634-1.462.146-.365.373-.545.785-.539l3.343.004c.099 0 .199.001.296.018.564.097.718.339.544.889-.275.863-.808 1.583-1.329 2.304-.558.772-1.154 1.517-1.707 2.293-.508.709-.468 1.066.163 1.681z" fill="#4E77A7"></path>
</svg>
<!-- /"source/partials/svg/vk-icon.svg" -->
</a>
</li>
<li>
<a class="reg-type__link" href="javascript:void(window.open('/index.php?route=extension/module/sociallogin/ok','_blank', 'width=700, height=400, top='+((screen.height-470)/2)+',left='+((screen.width-860)/2)+', resizable=yes, scrollbars=no, status=yes'))">
<!-- "source/partials/svg/ok-icon.svg" -->
<svg fill="none" height="30" viewbox="0 0 30 30" width="30" xmlns="http://www.w3.org/2000/svg">
<path d="M15.014 12.433a2.568 2.568 0 002.568-2.591A2.573 2.573 0 0015 7.262a2.577 2.577 0 00-2.587 2.61 2.574 2.574 0 002.601 2.56z" fill="#EE8208"></path>
<path d="M15 0C6.716 0 0 6.716 0 15c0 8.284 6.716 15 15 15 8.284 0 15-6.716 15-15 0-8.284-6.716-15-15-15zm.021 4.562c2.922.008 5.267 2.397 5.252 5.351-.014 2.886-2.398 5.225-5.315 5.215-2.887-.01-5.255-2.403-5.237-5.295a5.292 5.292 0 015.3-5.271zm5.918 12.642c-.646.663-1.424 1.143-2.287 1.478-.817.317-1.711.476-2.597.582.134.146.197.217.28.3 1.203 1.209 2.41 2.411 3.608 3.623.408.413.493.925.269 1.404-.246.525-.796.87-1.336.833-.342-.024-.608-.193-.845-.432-.907-.913-1.832-1.809-2.72-2.739-.26-.27-.383-.219-.612.016-.912.94-1.84 1.865-2.772 2.784-.42.413-.918.488-1.404.252-.516-.25-.845-.777-.82-1.308.018-.358.194-.632.44-.877 1.19-1.188 2.377-2.379 3.563-3.569.079-.079.152-.163.267-.285-1.619-.17-3.078-.568-4.328-1.545-.155-.121-.315-.238-.456-.375-.547-.525-.602-1.127-.17-1.746.37-.53.992-.672 1.638-.367.125.058.244.133.358.211 2.329 1.6 5.527 1.644 7.865.072.231-.177.479-.323.766-.396.557-.144 1.078.062 1.376.55.343.557.339 1.101-.083 1.534z" fill="#EE8208"></path>
</svg>
<!-- /"source/partials/svg/ok-icon.svg" -->
</a>
</li>
</ul>
</div>
</form>
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Finding all input fields in form
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Found the following input fields: {'login': '', 'password': '', 'remember': 'option1'}
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Found form action https://lkfl.krsk-sbit.ru/login and method post
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Merged input fields with input data in config. Result: {'login': 'admin', 'password': '***', 'remember': 'option1'}
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Determined the url to submit the form to: https://lkfl.krsk-sbit.ru/login
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Submitting the form
2024-10-21 11:16:39.513 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing form_submit-request with a post to url: https://lkfl.krsk-sbit.ru/login with headers: {} and cookies: <Cookies[<Cookie OCSESSID=302f3d8162d6d0e1ec844db33c for lkfl.krsk-sbit.ru />, <Cookie language=ru-ru for lkfl.krsk-sbit.ru />, <Cookie currency=RUB for lkfl.krsk-sbit.ru />, <Cookie session-cookie=18005cb702a421962dbcaa5f18991a24def884165319a5fd81c910ed6c11f8f1ae05d0d5843b412473b6f699c80da35f for lkfl.krsk-sbit.ru />]>.
2024-10-21 11:16:39.545 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Parse page with form with BeautifulSoup parser lxml
2024-10-21 11:16:39.594 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Parse page with form with BeautifulSoup parser lxml
2024-10-21 11:16:39.654 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # The page with the form parsed by BeautifulSoup has been written to file: form_page_soup.txt
2024-10-21 11:16:39.654 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Try to find form with selector form
2024-10-21 11:16:39.655 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Form looks like this:
<form action="https://lkfl.krsk-sbit.ru/login" method="post">
<div class="form-card">
<div class="form-group">
<label class="text__desc text__desc--small" for="login">Адрес электронной почты или номер мобильного телефона</label>
<input autocomplete="new-login" class="form-control ym-disable-keys" id="login" name="login" required="" type="text" value=""/>
<div class="invalid-feedback">
</div>
</div>
<div class="form-group password">
<label class="text__desc text__desc--small" for="password">Пароль</label>
<input autocomplete="new-password" class="form-control ym-disable-keys" id="password" name="password" required="" type="password" value=""/>
<a class="password-control" href="#"></a>
<div class="valid-feedback">
Текст ошибки
</div>
</div>
<div class="form-card__inner col-sm-4">
<div class="form-check form-check-inline">
<input class="form-check-input" id="remember" name="remember" type="checkbox" value="option1"/>
<label class="form-check-label" for="remember">Запомнить меня</label>
</div>
<button class="btn btn-send" type="submit">Войти</button>
</div>
<button class="btn btn-change restore-password" style="margin-right: 20px" type="button">Забыли пароль?</button>
<button class="btn btn-change registration" type="button">Зарегистрироваться</button>
</div>
<div class="reg-type">
<span class="reg-type__text">Или войти через соцсети:</span>
<ul class="reg-type__list">
<li>
<a class="reg-type__link" href="javascript:void(window.open('/index.php?route=extension/module/sociallogin/vk','_blank', 'width=700, height=400, top='+((screen.height-470)/2)+',left='+((screen.width-860)/2)+', resizable=yes, scrollbars=no, status=yes'))">
<!-- "source/partials/svg/vk-icon.svg" -->
<svg fill="none" height="30" viewbox="0 0 30 30" width="30" xmlns="http://www.w3.org/2000/svg">
<path d="M15 0C6.716 0 0 6.716 0 15c0 8.284 6.716 15 15 15 8.284 0 15-6.716 15-15 0-8.284-6.716-15-15-15zm7.609 16.622c.699.683 1.439 1.326 2.066 2.08.278.333.54.677.74 1.065.285.553.027 1.159-.467 1.192l-3.073-.001c-.794.066-1.426-.255-1.958-.797-.425-.433-.82-.894-1.229-1.342a2.871 2.871 0 00-.553-.492c-.42-.272-.783-.189-1.023.249-.245.445-.3.939-.324 1.434-.033.725-.252.914-.979.948-1.554.073-3.028-.163-4.398-.946-1.208-.69-2.144-1.666-2.959-2.77-1.587-2.152-2.803-4.514-3.895-6.944-.246-.547-.066-.84.538-.85a85.522 85.522 0 013.009-.002c.408.007.678.24.835.625.542 1.334 1.206 2.603 2.039 3.78.222.313.448.625.77.846.357.245.628.164.796-.233.107-.253.154-.523.177-.794.08-.927.09-1.854-.049-2.777-.086-.578-.41-.951-.987-1.06-.293-.056-.25-.165-.108-.332.248-.29.48-.47.944-.47h3.472c.547.107.67.353.744.904l.003 3.859c-.006.213.107.845.49.985.307.101.51-.145.693-.34.833-.883 1.426-1.926 1.957-3.005.235-.476.437-.969.634-1.462.146-.365.373-.545.785-.539l3.343.004c.099 0 .199.001.296.018.564.097.718.339.544.889-.275.863-.808 1.583-1.329 2.304-.558.772-1.154 1.517-1.707 2.293-.508.709-.468 1.066.163 1.681z" fill="#4E77A7"></path>
</svg>
<!-- /"source/partials/svg/vk-icon.svg" -->
</a>
</li>
<li>
<a class="reg-type__link" href="javascript:void(window.open('/index.php?route=extension/module/sociallogin/ok','_blank', 'width=700, height=400, top='+((screen.height-470)/2)+',left='+((screen.width-860)/2)+', resizable=yes, scrollbars=no, status=yes'))">
<!-- "source/partials/svg/ok-icon.svg" -->
<svg fill="none" height="30" viewbox="0 0 30 30" width="30" xmlns="http://www.w3.org/2000/svg">
<path d="M15.014 12.433a2.568 2.568 0 002.568-2.591A2.573 2.573 0 0015 7.262a2.577 2.577 0 00-2.587 2.61 2.574 2.574 0 002.601 2.56z" fill="#EE8208"></path>
<path d="M15 0C6.716 0 0 6.716 0 15c0 8.284 6.716 15 15 15 8.284 0 15-6.716 15-15 0-8.284-6.716-15-15-15zm.021 4.562c2.922.008 5.267 2.397 5.252 5.351-.014 2.886-2.398 5.225-5.315 5.215-2.887-.01-5.255-2.403-5.237-5.295a5.292 5.292 0 015.3-5.271zm5.918 12.642c-.646.663-1.424 1.143-2.287 1.478-.817.317-1.711.476-2.597.582.134.146.197.217.28.3 1.203 1.209 2.41 2.411 3.608 3.623.408.413.493.925.269 1.404-.246.525-.796.87-1.336.833-.342-.024-.608-.193-.845-.432-.907-.913-1.832-1.809-2.72-2.739-.26-.27-.383-.219-.612.016-.912.94-1.84 1.865-2.772 2.784-.42.413-.918.488-1.404.252-.516-.25-.845-.777-.82-1.308.018-.358.194-.632.44-.877 1.19-1.188 2.377-2.379 3.563-3.569.079-.079.152-.163.267-.285-1.619-.17-3.078-.568-4.328-1.545-.155-.121-.315-.238-.456-.375-.547-.525-.602-1.127-.17-1.746.37-.53.992-.672 1.638-.367.125.058.244.133.358.211 2.329 1.6 5.527 1.644 7.865.072.231-.177.479-.323.766-.396.557-.144 1.078.062 1.376.55.343.557.339 1.101-.083 1.534z" fill="#EE8208"></path>
</svg>
<!-- /"source/partials/svg/ok-icon.svg" -->
</a>
</li>
</ul>
</div>
</form>
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Finding all input fields in form
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Found the following input fields: {'login': '', 'password': '', 'remember': 'option1'}
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Found form action https://lkfl.krsk-sbit.ru/login and method post
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Merged input fields with input data in config. Result: {'login': 'admin', 'password': '***', 'remember': 'option1'}
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Determined the url to submit the form to: https://lkfl.krsk-sbit.ru/login
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Submitting the form
2024-10-21 11:16:39.656 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing form_submit-request with a post to url: https://lkfl.krsk-sbit.ru/login with headers: {} and cookies: <Cookies[<Cookie OCSESSID=c9bf24210fbb44205fbb16922a for lkfl.krsk-sbit.ru />, <Cookie language=ru-ru for lkfl.krsk-sbit.ru />, <Cookie currency=RUB for lkfl.krsk-sbit.ru />, <Cookie session-cookie=18005cb7036ce3ad2dbcaa5f18991a24fa74aa6c099fdf873286566a6850c28a81af62fc9f6428ac5d95555c62861cae for lkfl.krsk-sbit.ru />]>.
2024-10-21 11:16:39.657 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: form_submit_request_headers.txt
2024-10-21 11:16:39.657 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_body written to file: form_submit_request_body.txt
2024-10-21 11:16:39.657 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_cookies written to file: form_submit_request_cookies.txt
2024-10-21 11:16:39.657 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # The page with the form parsed by BeautifulSoup has been written to file: form_page_soup.txt
2024-10-21 11:16:39.657 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Try to find form with selector form
2024-10-21 11:16:39.658 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Form looks like this:
<form action="https://lkfl.krsk-sbit.ru/login" method="post">
<div class="form-card">
<div class="form-group">
<label class="text__desc text__desc--small" for="login">Адрес электронной почты или номер мобильного телефона</label>
<input autocomplete="new-login" class="form-control ym-disable-keys" id="login" name="login" required="" type="text" value=""/>
<div class="invalid-feedback">
</div>
</div>
<div class="form-group password">
<label class="text__desc text__desc--small" for="password">Пароль</label>
<input autocomplete="new-password" class="form-control ym-disable-keys" id="password" name="password" required="" type="password" value=""/>
<a class="password-control" href="#"></a>
<div class="valid-feedback">
Текст ошибки
</div>
</div>
<div class="form-card__inner col-sm-4">
<div class="form-check form-check-inline">
<input class="form-check-input" id="remember" name="remember" type="checkbox" value="option1"/>
<label class="form-check-label" for="remember">Запомнить меня</label>
</div>
<button class="btn btn-send" type="submit">Войти</button>
</div>
<button class="btn btn-change restore-password" style="margin-right: 20px" type="button">Забыли пароль?</button>
<button class="btn btn-change registration" type="button">Зарегистрироваться</button>
</div>
<div class="reg-type">
<span class="reg-type__text">Или войти через соцсети:</span>
<ul class="reg-type__list">
<li>
<a class="reg-type__link" href="javascript:void(window.open('/index.php?route=extension/module/sociallogin/vk','_blank', 'width=700, height=400, top='+((screen.height-470)/2)+',left='+((screen.width-860)/2)+', resizable=yes, scrollbars=no, status=yes'))">
<!-- "source/partials/svg/vk-icon.svg" -->
<svg fill="none" height="30" viewbox="0 0 30 30" width="30" xmlns="http://www.w3.org/2000/svg">
<path d="M15 0C6.716 0 0 6.716 0 15c0 8.284 6.716 15 15 15 8.284 0 15-6.716 15-15 0-8.284-6.716-15-15-15zm7.609 16.622c.699.683 1.439 1.326 2.066 2.08.278.333.54.677.74 1.065.285.553.027 1.159-.467 1.192l-3.073-.001c-.794.066-1.426-.255-1.958-.797-.425-.433-.82-.894-1.229-1.342a2.871 2.871 0 00-.553-.492c-.42-.272-.783-.189-1.023.249-.245.445-.3.939-.324 1.434-.033.725-.252.914-.979.948-1.554.073-3.028-.163-4.398-.946-1.208-.69-2.144-1.666-2.959-2.77-1.587-2.152-2.803-4.514-3.895-6.944-.246-.547-.066-.84.538-.85a85.522 85.522 0 013.009-.002c.408.007.678.24.835.625.542 1.334 1.206 2.603 2.039 3.78.222.313.448.625.77.846.357.245.628.164.796-.233.107-.253.154-.523.177-.794.08-.927.09-1.854-.049-2.777-.086-.578-.41-.951-.987-1.06-.293-.056-.25-.165-.108-.332.248-.29.48-.47.944-.47h3.472c.547.107.67.353.744.904l.003 3.859c-.006.213.107.845.49.985.307.101.51-.145.693-.34.833-.883 1.426-1.926 1.957-3.005.235-.476.437-.969.634-1.462.146-.365.373-.545.785-.539l3.343.004c.099 0 .199.001.296.018.564.097.718.339.544.889-.275.863-.808 1.583-1.329 2.304-.558.772-1.154 1.517-1.707 2.293-.508.709-.468 1.066.163 1.681z" fill="#4E77A7"></path>
</svg>
<!-- /"source/partials/svg/vk-icon.svg" -->
</a>
</li>
<li>
<a class="reg-type__link" href="javascript:void(window.open('/index.php?route=extension/module/sociallogin/ok','_blank', 'width=700, height=400, top='+((screen.height-470)/2)+',left='+((screen.width-860)/2)+', resizable=yes, scrollbars=no, status=yes'))">
<!-- "source/partials/svg/ok-icon.svg" -->
<svg fill="none" height="30" viewbox="0 0 30 30" width="30" xmlns="http://www.w3.org/2000/svg">
<path d="M15.014 12.433a2.568 2.568 0 002.568-2.591A2.573 2.573 0 0015 7.262a2.577 2.577 0 00-2.587 2.61 2.574 2.574 0 002.601 2.56z" fill="#EE8208"></path>
<path d="M15 0C6.716 0 0 6.716 0 15c0 8.284 6.716 15 15 15 8.284 0 15-6.716 15-15 0-8.284-6.716-15-15-15zm.021 4.562c2.922.008 5.267 2.397 5.252 5.351-.014 2.886-2.398 5.225-5.315 5.215-2.887-.01-5.255-2.403-5.237-5.295a5.292 5.292 0 015.3-5.271zm5.918 12.642c-.646.663-1.424 1.143-2.287 1.478-.817.317-1.711.476-2.597.582.134.146.197.217.28.3 1.203 1.209 2.41 2.411 3.608 3.623.408.413.493.925.269 1.404-.246.525-.796.87-1.336.833-.342-.024-.608-.193-.845-.432-.907-.913-1.832-1.809-2.72-2.739-.26-.27-.383-.219-.612.016-.912.94-1.84 1.865-2.772 2.784-.42.413-.918.488-1.404.252-.516-.25-.845-.777-.82-1.308.018-.358.194-.632.44-.877 1.19-1.188 2.377-2.379 3.563-3.569.079-.079.152-.163.267-.285-1.619-.17-3.078-.568-4.328-1.545-.155-.121-.315-.238-.456-.375-.547-.525-.602-1.127-.17-1.746.37-.53.992-.672 1.638-.367.125.058.244.133.358.211 2.329 1.6 5.527 1.644 7.865.072.231-.177.479-.323.766-.396.557-.144 1.078.062 1.376.55.343.557.339 1.101-.083 1.534z" fill="#EE8208"></path>
</svg>
<!-- /"source/partials/svg/ok-icon.svg" -->
</a>
</li>
</ul>
</div>
</form>
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Finding all input fields in form
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Found the following input fields: {'login': '', 'password': '', 'remember': 'option1'}
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Found form action https://lkfl.krsk-sbit.ru/login and method post
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Merged input fields with input data in config. Result: {'login': 'admin', 'password': '***', 'remember': 'option1'}
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Determined the url to submit the form to: https://lkfl.krsk-sbit.ru/login
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Submitting the form
2024-10-21 11:16:39.659 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing form_submit-request with a post to url: https://lkfl.krsk-sbit.ru/login with headers: {} and cookies: <Cookies[<Cookie OCSESSID=655d20f422a8d8d4a3b72fc668 for lkfl.krsk-sbit.ru />, <Cookie language=ru-ru for lkfl.krsk-sbit.ru />, <Cookie currency=RUB for lkfl.krsk-sbit.ru />, <Cookie session-cookie=18005cb7029c82372dbcaa5f18991a2438b038ccc148ee5ca7848469881e2707171436e42833e366db551860591e8dff for lkfl.krsk-sbit.ru />]>.
2024-10-21 11:16:39.680 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: form_submit_request_headers.txt
2024-10-21 11:16:39.680 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_body written to file: form_submit_request_body.txt
2024-10-21 11:16:39.680 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_cookies written to file: form_submit_request_cookies.txt
2024-10-21 11:16:39.680 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: form_submit_request_headers.txt
2024-10-21 11:16:39.680 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_body written to file: form_submit_request_body.txt
2024-10-21 11:16:39.693 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_cookies written to file: form_submit_request_cookies.txt
2024-10-21 11:16:40.322 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:40.335 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: form_submit_response_cookies.txt
2024-10-21 11:16:40.336 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: form_submit_response_headers.txt
2024-10-21 11:16:40.343 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: form_submit_response_body.txt
2024-10-21 11:16:40.345 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Form seems to be submitted successfully (to be sure, use log_response and check file). Now continuing to retrieve target page.
2024-10-21 11:16:40.345 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing page-request with a GET to url: https://lkfl.krsk-sbit.ru/index.php?setcurrentaccount&account=103113674650 with headers: {} and cookies: <Cookies[<Cookie OCSESSID=302f3d8162d6d0e1ec844db33c for lkfl.krsk-sbit.ru />]>.
2024-10-21 11:16:40.351 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_cookies written to file: page_request_cookies.txt
2024-10-21 11:16:40.351 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: page_request_headers.txt
2024-10-21 11:16:40.408 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:40.414 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: form_submit_response_headers.txt
2024-10-21 11:16:40.421 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: form_submit_response_cookies.txt
2024-10-21 11:16:40.422 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: form_submit_response_body.txt
2024-10-21 11:16:40.423 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Form seems to be submitted successfully (to be sure, use log_response and check file). Now continuing to retrieve target page.
2024-10-21 11:16:40.423 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing page-request with a GET to url: https://lkfl.krsk-sbit.ru/index.php?setcurrentaccount&account=102102001480 with headers: {} and cookies: <Cookies[<Cookie OCSESSID=c9bf24210fbb44205fbb16922a for lkfl.krsk-sbit.ru />]>.
2024-10-21 11:16:40.425 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: page_request_headers.txt
2024-10-21 11:16:40.426 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_cookies written to file: page_request_cookies.txt
2024-10-21 11:16:40.446 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:40.449 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: form_submit_response_headers.txt
2024-10-21 11:16:40.450 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: form_submit_response_body.txt
2024-10-21 11:16:40.451 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: form_submit_response_cookies.txt
2024-10-21 11:16:40.452 DEBUG (MainThread) [custom_components.multiscrape.form] КрасноярскЭнергоСбыт # Form seems to be submitted successfully (to be sure, use log_response and check file). Now continuing to retrieve target page.
2024-10-21 11:16:40.452 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Executing page-request with a GET to url: https://lkfl.krsk-sbit.ru/index.php?setcurrentaccount&account=133500912060 with headers: {} and cookies: <Cookies[<Cookie OCSESSID=655d20f422a8d8d4a3b72fc668 for lkfl.krsk-sbit.ru />]>.
2024-10-21 11:16:40.453 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_headers written to file: page_request_headers.txt
2024-10-21 11:16:40.453 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # request_cookies written to file: page_request_cookies.txt
2024-10-21 11:16:44.759 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:44.770 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: page_response_headers.txt
2024-10-21 11:16:44.770 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: page_response_body.txt
2024-10-21 11:16:44.772 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: page_response_cookies.txt
2024-10-21 11:16:44.774 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Loading the content in BeautifulSoup.
2024-10-21 11:16:44.918 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # page_soup written to file: page_soup.txt
2024-10-21 11:16:44.918 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Data successfully refreshed. Sensors will now start scraping to update.
2024-10-21 11:16:44.918 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 7.964 seconds (success: True)
2024-10-21 11:16:44.918 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Start scraping to update sensor
2024-10-21 11:16:44.927 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Tag selected: <span class="user-page__score-int" style="color: #1667a3">0 <span class="rubble">₽</span></span>
2024-10-21 11:16:44.927 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Selector result: 0 ₽
2024-10-21 11:16:44.927 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Applying value_template on selector result
2024-10-21 11:16:44.927 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Final selector value: 0.0 of type <class 'float'>
2024-10-21 11:16:44.928 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Selected: 0.0
2024-10-21 11:16:44.928 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Sensor updated and state written to HA
2024-10-21 11:16:44.993 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:44.999 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: page_response_headers.txt
2024-10-21 11:16:45.001 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: page_response_cookies.txt
2024-10-21 11:16:45.004 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: page_response_body.txt
2024-10-21 11:16:45.004 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Loading the content in BeautifulSoup.
2024-10-21 11:16:45.098 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # page_soup written to file: page_soup.txt
2024-10-21 11:16:45.099 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Data successfully refreshed. Sensors will now start scraping to update.
2024-10-21 11:16:45.099 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 8.143 seconds (success: True)
2024-10-21 11:16:45.099 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Start scraping to update sensor
2024-10-21 11:16:45.105 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Tag selected: <span class="user-page__score-int" style="color: #1667a3">0 <span class="rubble">₽</span></span>
2024-10-21 11:16:45.105 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Selector result: 0 ₽
2024-10-21 11:16:45.105 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Applying value_template on selector result
2024-10-21 11:16:45.106 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Final selector value: 0.0 of type <class 'float'>
2024-10-21 11:16:45.106 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Selected: 0.0
2024-10-21 11:16:45.106 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Sensor updated and state written to HA
2024-10-21 11:16:46.025 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # Response status code received: 200
2024-10-21 11:16:46.035 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_headers written to file: page_response_headers.txt
2024-10-21 11:16:46.071 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_cookies written to file: page_response_cookies.txt
2024-10-21 11:16:46.072 DEBUG (MainThread) [custom_components.multiscrape.http] КрасноярскЭнергоСбыт # response_body written to file: page_response_body.txt
2024-10-21 11:16:46.087 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # Loading the content in BeautifulSoup.
2024-10-21 11:16:46.176 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # page_soup written to file: page_soup.txt
2024-10-21 11:16:46.176 DEBUG (MainThread) [custom_components.multiscrape.coordinator] КрасноярскЭнергоСбыт # Data successfully refreshed. Sensors will now start scraping to update.
2024-10-21 11:16:46.176 DEBUG (MainThread) [custom_components.multiscrape.coordinator] Finished fetching multiscrape data in 9.221 seconds (success: True)
2024-10-21 11:16:46.177 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Start scraping to update sensor
2024-10-21 11:16:46.181 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Tag selected: <span class="user-page__score-int" style="color: #1667a3">0 <span class="rubble">₽</span></span>
2024-10-21 11:16:46.181 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Selector result: 0 ₽
2024-10-21 11:16:46.181 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Applying value_template on selector result
2024-10-21 11:16:46.182 DEBUG (MainThread) [custom_components.multiscrape.scraper] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Final selector value: 0.0 of type <class 'float'>
2024-10-21 11:16:46.182 DEBUG (MainThread) [custom_components.multiscrape.sensor] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Selected: 0.0
2024-10-21 11:16:46.182 DEBUG (MainThread) [custom_components.multiscrape.entity] КрасноярскЭнергоСбыт # КрасноярскЭнергоСбыт # Sensor updated and state written to HA
whizkid79
commented
2 weeks ago
Version of the custom_component
8.0.2
Configuration
Describe the bug
Hello! It so happened that I need to track the status of three accounts on the same site. The sensor settings are absolutely identical, except for the login/password pair. If I am working with only one account at a time, then the data is retrieved correctly. After I add the second and/or third sensor, all the added sensors get the same value, although they are different in my personal accounts. I was able to combine all the accounts into one personal account, found a link to switch between accounts, but still all sensors get the same value. In other words, there is no difference whether I log into my personal account under one user and switch between accounts or log into my personal account under different credentials. As far as I understand, after reading the value from one account, multiscrape does not have time to log out of the personal account, tries to log in under another account, remaining in the personal account of the previous account and reads data from it. I'm sorry if I didn't write it clearly.
Debug log