Can't make it run in iteration (retry)

[arielmoatti]

Thanks! I've update the file appointment.test.js with mailslurp API key, inbox_id, name and phone. I run it in Docker Desktop and it runs, once, with the usual "Leider sind aktuell keine Termine für ihre Auswahl verfügbar." result. But I have no idea how to make this run over and over (with the 60 seconds delay and Terminsuche wiederholen button) since "retries" field is no loger there. Even if I added it, it doesn't have a timeout, so it reached "limit" immediately. What am I doning wrong?

[daniel-cortez-stevenson]

Hey @arielmoatti - I've decided that this code is no longer encompasses retry logic. The idea with that is that I'm not opinionated about orchestration of the code. It just does the thing when asked.

That being said, to get the functionality you're seeking: add this snippet to /tests/appointment.test.js before the test definition test() is called and after test is imported and initialized:

test.beforeEach(async ({}, testInfo) => {
  if (testInfo.retry > 0) {
    const waitSeconds = parseInt(process.env.RETRY_WAIT_SECONDS || "60");
    logger.info(`Waiting ${waitSeconds} seconds then retrying ...`);
    await sleep(waitSeconds * 1000);
  }
});

test.afterEach(async ({ context }, testInfo) => {
 await context.close()
 if (testInfo.status !== testInfo.expectedStatus) {
   logger.warn(`Appointment booking failed: ${testInfo.error.message}`);
   if (testInfo.error.message === "Rate limit exceeded") {
     const waitSeconds = parseInt(process.env.RETRY_WAIT_SECONDS_BLOCKED || "600");
     logger.info(`Waiting ${waitSeconds} seconds because we were rate limited ...`);
     await sleep(waitSeconds * 1000);
   }
 }
});

function sleep(ms = 120000) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

You can then play with RETRY_WAIT_SECONDS and RETRY_WAIT_SECONDS_BLOCKED. However the default values here are OK (usually).

Let me know if that works for you.

EDIT: Don't forget about defining the sleep function. Also notice logic to use test.beforeEach and test.afterEach

[arielmoatti]

Thanks, Daniel! ok, I've added this snippet, still nothing. Then I added the line: retries: 1000, to playwright.config.js and now finally I see Docker waits 60 seconds and retries! Nevertheless, I either get "terminvereinbarung/termin/taken/" and wait 60 seconds, or "Zu viele Zugriffe". I am trying to book a simple appointment for Anmeldung, and in another image I'm trying "Fahrerlaubnis - Umschreibung einer ausländischen Fahrerlaubnis aus einem Nicht-EU/EWR-Land (Drittstaat/Anlage 11)" and get the same results.

I was really hoping that this app would do the trick, but the situation in their system is so lousy, there must be another way for me to get a Termin...

[daniel-cortez-stevenson]

I either get "terminvereinbarung/termin/taken/" and wait 60 seconds, or "Zu viele Zugriffe".

It's normal to wait a long time at night or on weekends for an appointment to open up. Mornings around 9-11 AM and evenings 5-7 PM are usually best.

Let it run and increase RETRY_WAIT_SECONDS to "90" or "120" so you don't have to wait 10 mins when you are rate-limited. Alternatively, get a backconnect proxy like Bright Data (I've used them professionally), and you won't ever get blocked (even without this snippet!).

I was really hoping that this app would do the trick, but the situation in their system is so lousy, there must be another way for me to get a Termin...

We're competing with a lot of other bots. I even heard that there are shady folks selling appointments for cash.

btw with your driver's license appointment, make sure to show the bureaucrat your supporting docs (like when you got the license & driving record) even if they don't ask. They forgot to ask me when I went, and then I got a letter months late that I had to go again to show them the docs I had brought to the original meeting in the first place.

[arielmoatti]

Thanks again for those ideas! Ok, I registered at Bright Data and set it up (hopefully, correctly). I commented out the snippet, added the retriesparameter and set at 100. I sat back and watched it run (for about 5 minutes) and at the end? Docker gave me a huge list in the Logs, 90% of those were "Zu viele Zugriffe" and the rest were "...terminvereinbarung/termin/taken/" So, I'm not even sure my strategy is right.

As for the license, thanks for the tips! Yes, I'll have everything at hand. I actually wanted to ask: To book any other type of Termin, I need to put in the exact "name" string, as it appears on the website? E.G. "Fahrerlaubnis - Umschreibung einer ausländischen Fahrerlaubnis aus einem Nicht-EU/EWR-Land (Drittstaat/Anlage 11)" and not, for instance, its code? 327537? Since I have so far zero luck, I don't even know if I made any mistake in the whole chain...

Moreover, my ideal Termin would be to have both (Anmeldung & Umschreibung) in the same day, in the same place, and in sequential order, so I can get rid of these two in 1 hour (!) - until today, my only luck was to open two browsers in incognito and do that... now, even this fails. Any ideas? I appreciate everything you share here :) Ariel

[daniel-cortez-stevenson]

set it up (hopefully, correctly).

Could you share your playwright.config.js file (please remove secret values)? Did you set PROXY_URL? Would you share your logs?

To book any other type of Termin, I need to put in the exact "name" string, as it appears on the website? E.G. "Fahrerlaubnis - Umschreibung einer ausländischen Fahrerlaubnis aus einem Nicht-EU/EWR-Land (Drittstaat/Anlage 11)"

Yes. Exactly as it appears at https://service.berlin.de/dienstleistungen/.

Moreover, my ideal Termin would be to have both (Anmeldung & Umschreibung) in the same day, in the same place, and in sequential order

I would get the first appointment and then use env vars to limit the time, date & place of the second appointment ... but maybe there is something more clever. Let me think.

[arielmoatti]

Thanks. I sent you my PROXY_URL setting privately per email. As for getting 2 consecutive appointments... I'm afraid that by the time it finds me the 1st Anmeldung, when I'm not near the computer, the next available slot will be already booked. This needs to happen simultaneously, with Anmeldung BEFORE Umschreibung, same date, same place, next available slot.

That's a tought mission.

[daniel-cortez-stevenson]

As for getting 2 consecutive appointments... I'm afraid that by the time it finds me the 1st Anmeldung, when I'm not near the computer, the next available slot will be already booked. This needs to happen simultaneously, with Anmeldung BEFORE Umschreibung, same date, same place, next available slot.

Are you suggesting that the calendar appointments are universal?

My belief is that appointments are made available per service. So the calendars are not the same between services.

Maybe what you're suggesting is true or partially true. We'd need to do some research unless you can definitely say.

Perhaps appointment slot batches are made available at the same time for every calendar. Maybe they release similar dates/times per service to each individual calendar. We could leverage that.

My belief is that when you cancel an appointment it is made available in the calendar for the appointment service you booked. We definitely see instances of single appointments being available often enough that this should be true (releasing individual appointments via a manual mechanism would not scale).

Any thoughts @arielmoatti ?

[arielmoatti]

My belief is that appointments are made available per service. So the calendars are not the same between services.

Yes, I'm sure. At least for those two I mentioned. A friend of mine tried it for himself with two browser windows (incognito or different user profile, since the system is cookie based) and requested these two different Termins, he found a day in March, one slot at 11:00 and the second at 12:00. He booked them and he's done! So envy... I couldn't replicate this... I usually never see a calendar at all (and wait 1:00 minute), and when a miracle happens, and I go for the one available day - it then shows me an empty page (no Amt nor time slots). That's why I'm counting on a bot, to do that tidious job for me... The logic is, that the same Beamter can handle all these requests. They just want it under a designated "Termin" (......). I cannot guaranty about other types of services, and definitely the Ausländerbehörde is a whole different story.

[daniel-cortez-stevenson]

OK so this thread has gotten a bit off topic. @arielmoatti I think we solved the retry issue. If you continue to have issues, create a new one :)