search

scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.28k stars 1.41k forks source link

Infinite page loading #838

Closed AndrejCro closed 5 years ago

AndrejCro commented 6 years ago

On some sites portia application gets halted (cannot add new start urls or sample pages) because the page is still loading (infinitely).

How to prevent this from happening?

ayushmankoul commented 6 years ago

Hey, @CroCoder If you can provide an example of the issue that will be a very great clue to understanding the problem and work on it.

PaoloC68 commented 6 years ago

I try to set as start page this one https://society6.com/tapestries and I have the same behaviour

PaoloC68 commented 6 years ago

@ayushmankoul It seems that you have a quite serious problem to fix

screen shot 2018-03-13 at 00 29 18
ayushmankoul commented 6 years ago

Thank You @PaoloC68 for responding.The given website is based on react JS which is not supported by Portia as of now. reference. I am interested in contributing to Portia as part of GSoC 2018 on this Increase Crawling Performance through page clustering.Any guidance from you will be a great help to me.