API reversing vs Scarping?

[nerder]

Description

I think that most of the modern web for house searching will have directly API that we can call to get the info we want in a already formatted way. I think we should investigate this solution.

Pros

1. More stable, since API changes are less common then UI changes
2. Pre-formatted data format
3. Cleaner architecture

Cons

1. API limits?
2. BOT detection?
3. Cost of reversing the API?

Extra toughts

I imagine an architecture where we can have a ModernNormalizer for each provider that in in charge of converting the different models for each provider into our common one, in this way we can manipulate the data easily in the backend. The scraping part is an eventual fallback in case we don't have a public accessible API, or in case that reverting it will be to expensive.

Refs

5

[oscard0m]

To me would have sense to evaluate this path and discard the potential cons. I assign this to myself if you are ok with it and depending on the pros/cons for idealista and fotocasa we can decide what to do here! @nerder