search

dipnl / wappalyzer

Identify technology on websites.
https://www.wappalyzer.com
2 stars 0 forks source link

betere debugging mogelijkheid door requests failures te loggen #15

Open simonbesters opened 4 months ago

simonbesters commented 4 months ago

We zien nu niet goed waarom een website niet laadt of dat we maar enkele techs krijgen. Door de geskipte/failed requests ook te loggen vinden we sneller de oorzaak:

Driver.js regel 614:

    page.on('requestfailed', request => {
      this.log(`Skipped: ${request.resourceType()} --- ${request.url()}`);
    });

Voorbeeld:

log | driver | Launching browser (attempt 1)...
log | driver | Setting storage...
log | driver | Navigate to https://ltc.andes.nl/
log | driver | Skipped: stylesheet --- https://ltc.andes.nl/ltc/Content/_Shared/css/bootstrap.min.css
log | driver | Skipped: stylesheet --- https://ltc.andes.nl/ltc/Content/_Shared/css/fonts.css
log | driver | Skipped: stylesheet --- https://ltc.andes.nl/ltc/Content/_Shared/css/main.css
log | driver | Skipped: stylesheet --- https://ltc.andes.nl/ltc/Content/Home/css/custom.css
log | driver | Skipped: image --- https://ltc.andes.nl/ltc/Content/_Shared/images/logo_head.png
log | driver | Skipped: image --- https://ltc.andes.nl/ltc/Content/_Shared/images/logo_footer.png
log | driver | Skipped: image --- https://ltc.andes.nl/ltc/Content/_Shared/images/linkedin.png
log | driver | Skipped: image --- https://ltc.andes.nl/ltc/Content/_Shared/images/youtube.png
log | driver | Skipped: image --- https://ltc.andes.nl/ltc/Content/_Shared/images/scroll-to-top.svg
log | driver | Skipped: ping --- https://region1.google-analytics.com/g/collect?v=2&tid=G-KSBQD6RMLX&gtm=45je42t1v9115351067za220&_p=1709656793227&gcd=13l3l3l2l1&npa=1&dma_cps=sypham&dma=1&cid=606872255.1709656793&ul=en-us&sr=3440x1440&ir=1&uaa=&uab=&uafvl=&uamb=0&uam=&uap=&uapv=&uaw=0&pscdl=noapi&_eu=EAAI&_s=1&sid=1709656793&sct=1&seg=0&dl=https%3A%2F%2Fltc.andes.nl%2Fltc%2FAccount%2FLogin%3FReturnUrl%3D%252fltc%252f&dr=https%3A%2F%2Fltc.andes.nl%2F&dt=Inloggen%20LTC%20Platform%20-%20LTC%20Portal&en=page_view&_fv=1&_ss=1&tfd=382
simonbesters commented 4 months ago

We kunnen eventueel ook alle requests loggen:

    const session = await page.target().createCDPSession();
    await session.send('Network.enable');
    session.on('Network.requestWillBeSent', ({ request, type }) => {
      if (!request.url.startsWith('data:')) {
          console.log(`Sent: ${type} ${request.url}`);
        }
    });

Maar dat zegt niets over of deze requests ook slagen.