everything crashes after 1 or 2 hours even after restarting

[fabihoe]

everything crashes after 1-2 hours and I have to make a subtle change for my search request (for instance, raise the rent 50€) for it to run again for 1-2 hours. This is probably a protective mechanism from immoscout. Do you have an idea for a soultion?

[mustafabayar]

Is this different issue than https://github.com/mustafabayar/immo_trakt/issues/6? Can you show me the stack trace?

[mustafabayar]

Also can you tell me what frequency did you set in the config?

[fabihoe]

thanks for your reply!

2022-04-08T08:56:26.280526+00:00 app[worker.1]: 2022/04/08 08:56:26 json: cannot unmarshal object into Go struct field .searchResponseModel.resultlist.resultlist.resultlistEntries.resultlistEntry of type []struct { ID string "json:\"@id\""; Publishdate string "json:\"@publishDate\""; ResultlistRealEstate struct { ID string "json:\"@id\""; Title string "json:\"title\""; Price struct { Value float32 "json:\"value\""; Currency string "json:\"currency\"" } "json:\"price\""; LivingSpace float32 "json:\"livingSpace\""; NumberOfRooms float32 "json:\"numberOfRooms\""; CalculatedTotalRent struct { Totalrent struct { Value float32 "json:\"value\""; Currency string "json:\"currency\"" } "json:\"totalRent\"" } "json:\"calculatedTotalRent\"" } "json:\"resultlist.realEstate\"" }
2022-04-08T08:56:26.405335+00:00 heroku[worker.1]: Process exited with status 1
2022-04-08T08:56:26.500688+00:00 heroku[worker.1]: State changed from up to crashed

my frequency is set to 1m and my search link looks like that : "https://www.immobilienscout24.de/Suche/de/berlin/berlin/wohnung-mieten?haspromotion=false&numberofrooms=2.0-&price=-1800.0&exclusioncriteria=projectlisting,swapflat&pricetype=rentpermonth&geocodes=110000000702,110000000801,110000000406,110000000101,110000000102,110000000103,110000000202,110000000301,110000000401,110000000104,110000000105,110000000402,110000000106,110000000601,110000000701&sorting=2&enteredFrom=result_list"

[fabihoe]

Is this different issue than #6? Can you show me the stack trace?

it's the same problem I guess

[mustafabayar]

Just to experiment, can you try running the app with url from shape drawing view? From here:

Something like this: https://www.immobilienscout24.de/Suche/shape/wohnung-mieten?shape=eXttX0lneWBwQX5vQXl2QGRoQWt9QWJ7QGlScGlDbEVibkBpYUB8dUB9cENpQGFiRGdmQGVfQG1zQmFOa29Ac3RAdW5Bd0l1Tm9yQHZKd19DcGJAd2tCYEBwd0J1SmhsQW5SfmtAZUd9c0ZyQF9EbG1BY3lBfnFAc31Cc0FlfkVvekFjQWdkQXhlQWd1QWNBY3tAeGpFa2tAanBAX3BBaENrWn1wQ21ad1hjZkBiQXdkQ25tSnFDanFDX25AemlAb0VuY0BsZ0B0eURmUXhiSG50QH52QWRkQWxjQDt1aHhfSWdxenBBaEJfSFJfVA..&haspromotion=false&numberofrooms=2.0-&price=-1800.0&exclusioncriteria=projectlisting,swapflat&pricetype=rentpermonth&sorting=2&enteredFrom=result_list&viewMode=MAP#/?boundingBox=52.390103%2C13.196589%2C52.609138%2C13.581111

If it works without issue, then it means result from these two different endpoints are not always identical.

[mustafabayar]

Update: I was able to reproduce the issue using your search url. I will try to find the problem.

[fabihoe]

Thanks for your effort! I have now tried the other shape link and it seems to work without crashing. However, the WBS filter doesn't seem to work but that's no big issue.

[valentinabhm]

I ran into the same problem and found the issue. If it is sending a request for a page with just one entry the API returns an object instead of an array with one entry. So it fails to unmarshal the json as it expects an array for resultListEntry

[mustafabayar]

I ran into the same problem and found the issue. If it is sending a request for a page with just one entry the API returns an object instead of an array with one entry. So it fails to unmarshal the json as it expects an array for resultListEntry

Aha! Didn't had chance to look deeper but this explains the problem. I guess I need to define more flexible way of parsing the response. Will check this. Thank you.

[mustafabayar]

Thanks for your effort! I have now tried the other shape link and it seems to work without crashing. However, the WBS filter doesn't seem to work but that's no big issue.

Do you mean you set exclude_wbs to true in the config but you still get offers which have WBS in the title? Interesting

[fabihoe]

yes

[mustafabayar]

Fixed this issue in https://github.com/mustafabayar/immo_trakt/commit/7d811854a3e01468a92b3098bf8949c6a4e1710a

Kudos to @valentinabhm for pointing out the root cause.

Let me know if you experience similar issues.