Error during calendar page

[rsilvaberlin]

Hi

I saw the calendar and date available but nothing happening after that. There is a captcha step after date and dropdown selection. Could that be the reason?

The log seems to be saying null value at credentials. I am new here so any help would be great.

2023-05-22 09:26:21 ERROR TerminFinder Exception occurred during the process, quitting. org.openqa.selenium.TimeoutException: Expected condition failed: waiting for org.example.formhandlers.Section2ServiceSelectionHandler$$Lambda$640/0x000000080108a310@71e7bc01 (tried for 60 second(s) with 500 milliseconds interval) Build info: version: '4.8.1', revision: '8ebccac989' System info: os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '13.0', java.version: '17.0.7' Driver info: org.openqa.selenium.remote.RemoteWebDriver Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 113.0.5672.63, chrome: {chromedriverVersion: 113.0.5672.63 (0e1a4471d5ae..., userDataDir: /tmp/.org.chromium.Chromium...}, goog:chromeOptions: {debuggerAddress: localhost:35505}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: LINUX, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://172.17.0.2:4444/sessio..., se:cdpVersion: 113.0.5672.63, se:vnc: ws://172.17.0.2:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.17.0.2:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true} Session ID: 63aa7156ca12c096fb1b38f4805f3c21 at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:87) ~[selenium-support-4.8.1.jar:?] at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:231) ~[selenium-support-4.8.1.jar:?] at org.example.formhandlers.Section2ServiceSelectionHandler.selectCitizenshipValue(Section2ServiceSelectionHandler.java:97) ~[main/:?] at org.example.formhandlers.Section2ServiceSelectionHandler.fillForm(Section2ServiceSelectionHandler.java:71) ~[main/:?] at org.example.formhandlers.Section2ServiceSelectionHandler.fillAndSendForm(Section2ServiceSelectionHandler.java:59) ~[main/:?] at org.example.TerminFinder.fillAndSendFormWithExceptionHandling(TerminFinder.java:111) ~[main/:?] at org.example.TerminFinder.run(TerminFinder.java:86) ~[main/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:833) [?:?] 2023-05-22 09:26:23 INFO IoUtils error occurred during getting the secret 2023-05-22 09:26:23 INFO IoUtils File name :TerminFinder_2023-05-22_09:26:23_exception.html, TerminFinder_2023-05-22_09:26:23_exception.png 2023-05-22 09:26:23 INFO IoUtils Getting the page content 2023-05-22 09:26:24 INFO IoUtils Saving source code to file 2023-05-22 09:26:24 INFO IoUtils Saving screenshot 2023-05-22 09:26:24 INFO IoUtils Storing files to s3 2023-05-22 09:26:24 INFO IoUtils Saving has failed. Reason: java.lang.NullPointerException: Credentials must not be null. at software.amazon.awssdk.utils.Validate.notNull(Validate.java:119) ~[utils-2.20.61.jar:?] at software.amazon.awssdk.auth.credentials.StaticCredentialsProvider.(StaticCredentialsProvider.java:30) ~[auth-2.20.61.jar:?] at software.amazon.awssdk.auth.credentials.StaticCredentialsProvider.create(StaticCredentialsProvider.java:37) ~[auth-2.20.61.jar:?] at org.example.utils.IoUtils.savePage(IoUtils.java:204) ~[main/:?] at org.example.TerminFinder.run(TerminFinder.java:97) ~[main/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:833) [?:?]

[rsilvaberlin]

[rsilvaberlin]

[rsilvaberlin]

[ya-ning]

I have the same problem. The screenshots show the calendar page but nothing was selected

[rsilvaberlin]

@ya-ning can you share your log as well?

[ya-ning]

@rsilvaberlin it just happened again (screenshot shows the calendar page, with 22.05 being available), the logs are:

2023-05-22 17:30:18 INFO Section2ServiceSelectionHandler Calender page is not opened. Search count: 133. SearchCountWithCalenderOpened: 0 2023-05-22 17:30:28 INFO Section2ServiceSelectionHandler Starting to fill the form in section 2 2023-05-22 17:30:28 INFO Section2ServiceSelectionHandler Starting to selectCitizenshipValue 2023-05-22 17:31:28 ERROR TerminFinder Exception occurred during the process, quitting. org.openqa.selenium.TimeoutException: Expected condition failed: waiting for org.example.formhandlers.Section2ServiceSelectionHandler$$Lambda$649/0x000000080108b970@7b11236f (tried for 60 second(s) with 500 milliseconds interval) Build info: version: '4.8.1', revision: '8ebccac989' System info: os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '13.3.1', java.version: '17.0.7' Driver info: org.openqa.selenium.remote.RemoteWebDriver Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 112.0.5615.138, chrome: {chromedriverVersion: 112.0.5615.138 (2258f2fcaeb..., userDataDir: /tmp/.org.chromium.Chromium...}, goog:chromeOptions: {debuggerAddress: localhost:43541}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: LINUX, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://172.17.0.2:4444/sessio..., se:cdpVersion: 112.0.5615.138, se:vnc: ws://172.17.0.2:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.17.0.2:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true} Session ID: 01e28f457a1da2ac1ea890a7633ff081 at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:87) ~[selenium-support-4.8.1.jar:?] at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:231) ~[selenium-support-4.8.1.jar:?] at org.example.formhandlers.Section2ServiceSelectionHandler.selectCitizenshipValue(Section2ServiceSelectionHandler.java:97) ~[main/:?] at org.example.formhandlers.Section2ServiceSelectionHandler.fillForm(Section2ServiceSelectionHandler.java:71) ~[main/:?] at org.example.formhandlers.Section2ServiceSelectionHandler.fillAndSendForm(Section2ServiceSelectionHandler.java:59) ~[main/:?] at org.example.TerminFinder.fillAndSendFormWithExceptionHandling(TerminFinder.java:111) ~[main/:?] at org.example.TerminFinder.run(TerminFinder.java:86) ~[main/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:833) [?:?] 2023-05-22 17:31:30 INFO IoUtils error occurred during getting the secret 2023-05-22 17:31:30 INFO IoUtils File name :TerminFinder_2023-05-22_17:31:30_exception.html, TerminFinder_2023-05-22_17:31:30_exception.png 2023-05-22 17:31:30 INFO IoUtils Getting the page content 2023-05-22 17:31:32 INFO IoUtils Saving source code to file 2023-05-22 17:31:32 INFO IoUtils Saving screenshot 2023-05-22 17:31:34 INFO IoUtils Storing files to s3 2023-05-22 17:31:34 INFO IoUtils Saving has failed. Reason: java.lang.NullPointerException: Credentials must not be null. at software.amazon.awssdk.utils.Validate.notNull(Validate.java:119) ~[utils-2.20.61.jar:?] at software.amazon.awssdk.auth.credentials.StaticCredentialsProvider.(StaticCredentialsProvider.java:30) ~[auth-2.20.61.jar:?] at software.amazon.awssdk.auth.credentials.StaticCredentialsProvider.create(StaticCredentialsProvider.java:37) ~[auth-2.20.61.jar:?] at org.example.utils.IoUtils.savePage(IoUtils.java:204) ~[main/:?] at org.example.TerminFinder.run(TerminFinder.java:97) ~[main/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:833) [?:?] 2023-05-22 17:31:34 INFO TerminFinder page is saved 2023-05-22 17:31:34 INFO DriverUtils Initializing the driver. Host: localhost, try: 0 2023-05-22 17:32:10 INFO DriverUtils Driver is initialized. 2023-05-22 17:32:20 INFO TerminFinder Starting to getHomePage 2023-05-22 17:32:21 INFO TerminFinder Getting the URL: https://otv.verwalt-berlin.de/ams/TerminBuchen?lang=en 2023-05-22 17:32:23 INFO Section1MainPageHandler Starting to fill the form

[rsilvaberlin]

@ya-ning unfortunately I am not an expert, but it seems to be similar to my problem. I'll leave the bot running for few more days to see if it is only because of lack of available timeslot. Maybe someone else from the community can shows us some light on this.

[yilmaznaslan]

The bot is broken due to the latest update on the website, until fix is implemented

[rsilvaberlin]

Thanks @yilmaznaslan.

[yassin64b]

With the newest code I get this output and the bot stops, but no confirmation email from LEA so it was not successful. Before updating today I still got a sound notification and similar error to above, now I also don't get a sound.

2023-10-11 18:30:04 INFO Section2ServiceSelectionHandler Page is loaded
2023-10-11 18:30:04 INFO Section2ServiceSelectionHandler Starting to isDateSelectionOpened
2023-10-11 18:30:07 INFO Section2ServiceSelectionHandler Value of the ACTIVE_STEP is: 3Date selection
2023-10-11 18:30:07 INFO Section2ServiceSelectionHandler There is no error message
2023-10-11 18:30:07 INFO Section2ServiceSelectionHandler Calender page is opened and there is no alert
2023-10-11 18:30:07 INFO Section3DateSelectionHandler Starting to: listDatesAndClickToFirstAvailable
2023-10-11 18:30:07 INFO Section3DateSelectionHandler Available date: Day: 12, Month: 9 Year: 2023
2023-10-11 18:30:07 INFO Section3DateSelectionHandler Available date: Day: 12, Month: 9 Year: 2023
2023-10-11 18:30:07 INFO AppointmentFinder End of process
2023-10-11 18:30:09 INFO IoUtils error occurred during getting the secret
2023-10-11 18:30:09 INFO IoUtils File name :date_selection_success_2023-10-11_18:30:09.html, date_selection_success_2023-10-11_18:30:09.png
2023-10-11 18:30:09 INFO IoUtils Getting the page content
2023-10-11 18:30:09 INFO IoUtils Saving source code to file
2023-10-11 18:30:09 INFO IoUtils Saving screenshot
2023-10-11 18:30:09 INFO IoUtils S3 is not  enabled
2023-10-11 18:30:09 INFO Main Appointments found

[yilmaznaslan]

Yes because of captcha you don't get email but it only plays a sound once termin page is opened, Idk why it didn't play sound

[allentiak]

@yilmaznaslan Shall we either rename this or closing it in favor of #83?