orangecoding / fredy

:heart: Fredy - [F]ind [R]eal [E]states [D]amn Eas[y] - Fredy will constantly search for new listings on sites like Immoscout or Immowelt and send new results to you, so that you can focus on more important things in life ;)
http://www.orange-coding.net
MIT License
231 stars 58 forks source link

"Unexpected end of file" when trying to scrape Immowelt #69

Closed ULTRA-VAGUE closed 1 year ago

ULTRA-VAGUE commented 1 year ago

Whenever I try to setup Immowelt I get this error code.

"Error while trying to scrape data. Received error: unexpected end of file"

It doesn´t scrape anything off of Immowelt, even with a new Config and Database .json file. I´ve checked the formatting and there doesn´t seem to be any issues there.

How can I resolve this?

This is the specific String I used with fredy: https://www.immowelt.de/liste/berlin/wohnungen/mieten?ami=30&d=true&pma=700&sd=DESC&sf=TIMESTAMP&sp=1

orangecoding commented 1 year ago

Thanks for the report. I'll check.

orangecoding commented 1 year ago

Mhm. it works totally fine for me. Can you do me a favour, go into

/test/provider/testProvider.json

and replace the existing immowelt with your url in here:

  "immowelt": {
    "url": "https://www.immowelt.de/liste/duesseldorf/wohnungen/kaufen?d=true&rmi=3&sd=DESC&sf=TIMESTAMP&sp=1",
    "enabled": true
  },

then run yarn run test. All Tests are passing for me.

ULTRA-VAGUE commented 1 year ago

all tests are also passing for me, however the issue still persists. I don´t think there is an issue with the string by itself.

ULTRA-VAGUE commented 1 year ago

to add to that: I have created seperate jobs for every provider, when I disable immowelt everything starts working normally again.

orangecoding commented 1 year ago

If the test works, everything else should work as well,as the test is only using the stuff that is also being used while doing 'real' work. I'll check further

orangecoding commented 1 year ago

Hi @ULTRA-VAGUE Can you pull the latest master, run npm install or yarn (or whatever you use) and try again? See https://github.com/orangecoding/fredy/commit/7d0ec72a0c3b6aaf8ef77866aa12864584288c53

ULTRA-VAGUE commented 1 year ago

Tried it again on my vps. That seems to have fixed it! Thank you :)