search

sitespeedio / sitespeed.io

sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.
https://www.sitespeed.io/
MIT License
4.73k stars 601 forks source link

Using --block options #2074

Closed senthilr2k closed 5 years ago

senthilr2k commented 6 years ago

Hi,

Could you please help in using --block option for blocking few domains in my url. Please let know how i can specify multiple domain names that has to ignored

Thanks

soulgalore commented 6 years ago

Hi @senthilr2k it should work that you add the domain with block parameter. If you wanna block multiple domains you add the block param multiple times: sitespeed.io https://en.wikipedia.org/wiki/Main_Page --block upload.wikimedia.org

multiple domains: sitespeed.io https://en.wikipedia.org/wiki/Main_Page --block upload.wikimedia.org --block login.wikimedia.org

Please let me know if it works for you! Best Peter

senthilr2k commented 6 years ago

Hi Thanks. When i used --block domainname, i am still seeing sitespeed making request for those urls and able to see those in the waterfall graph in sitespeed output

soulgalore commented 6 years ago

Hi @senthilr2k can you please share exactly how you run and screenshots so I can try to reproduce.

Thanks Peter

senthilr2k commented 6 years ago

I am using docker. sitespeed/sitespeed.io -b chrome --block ss7.vzw.com https://www.verizonwireless.com

soulgalore commented 6 years ago

ok, when I run docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io --block ss7.vzw.com https://www.verizonwireless.com -n 1 I cannot see a request to that domain in the HAR? Switched changing to --block ss71.vzw.com I see that domain is blocked (and a request to the ss7 domain popped up in the HAR).

senthilr2k commented 6 years ago

I am using the same but have browsertime proxy. Iam seeing request is made to scache.vzw.com even though i have added --block scache.vzw.com . I am seeing the request made in har as well as water fall graph. I also tried blocking ss7.vzw.com but same result. Is it like i am using a different version of sitespeed. How can i check the version in docker

soulgalore commented 6 years ago

just run with —version. best is to always specify the version when running docker with :7.0.3 ... sorry for not doing that in my example!

soulgalore commented 6 years ago

btw your log should also show which version you are using.

softwareklinic commented 6 years ago

Hey Senthil --- I'm here to post exactly same question :) I tried it again using 6.4.1 sitespeed.io and am seeing one particular domain which I want to block... still loads all resources from that domain...

Peter - do u want me to try out another sitespeed version?

softwareklinic commented 6 years ago

Tried using 7.1.2 sitespeed docker as well.. the same outcome - doesn't block here is a sample command

and a screenshot showing it's not being blocked

docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io https://www.verizonwireless.com -n 1 --mobile --browsertime.block tags.tiqcdn.com

image

softwareklinic commented 6 years ago

Also, let me add something more --- couple of months back using the same 6.4.1 version i have indeed seen it blocking.. but something was changed in the way these tags are fired... after which it stopped blocking.... But, could be a similar issue for companies using tag injection tools such as Tealium etc... if moving from head to body etc... or async or defer... being used for loading these?

soulgalore commented 6 years ago

@softwareklinic let me try to reproduce tonight, thanks for sharing exact how you run :)

The blocking happens in the web extension so it shouldn't matter how the request is done (see https://github.com/sitespeedio/browsertime-extension/blob/master/background.js).

softwareklinic commented 6 years ago

Only more thing is we are behind proxy so we are using browsertime proxy parameters to reach out to internet... again as I said it was certainly working with same docker 6.4.1 -- but not anymore... so our business team moving around the external vendor tags from head to body or using async -- might have caused it.. just in case if this input is useful for your diagnosis...

soulgalore commented 6 years ago

Could it be the proxy that mess things up? When I tried it seems to work:

docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io https://www.verizonwireless.com -n 1 --mobile --browsertime.block tags.tiqcdn.com

screen shot 2018-06-20 at 3 36 56 pm
softwareklinic commented 6 years ago

I think it could be proxy... let me try it out without PROXY... using hotspot and get back... If its proxy.. do you think we will need a fix in code? or is it fixable?

On Wed, Jun 20, 2018 at 9:38 AM, Peter Hedenskog notifications@github.com wrote:

Could it be the proxy that mess things up? When I tried it seems to work:

docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/ sitespeed.io https://www.verizonwireless.com -n 1 --mobile --browsertime.block tags.tiqcdn.com

[image: screen shot 2018-06-20 at 3 36 56 pm] https://urldefense.proofpoint.com/v2/url?u=https-3A__user-2Dimages.githubusercontent.com_540757_41665437-2De1c12174-2D749f-2D11e8-2D9d35-2D4746ff2a4146.png&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=9BseYMtKEBfGNEXp3jVdpQ9R1gedy5URNVLqSaUimto&s=jmOEB0Wmb3uWM9hs3NQ91kwG_QE2IXIy_pt1rlSm5bo&e=

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sitespeedio_sitespeed.io_issues_2074-23issuecomment-2D398773469&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=9BseYMtKEBfGNEXp3jVdpQ9R1gedy5URNVLqSaUimto&s=lf0gT6H3gdYn_qME2aVgE2mkEDzrDs7EI8d_M9V6Z_I&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AHMCiXtzCCYrPUygHOOsaFoOXrdEFw5Rks5t-2Dl5pgaJpZM4UcveB&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=9BseYMtKEBfGNEXp3jVdpQ9R1gedy5URNVLqSaUimto&s=roNtZ8MRX8mLFroG3cX-cAbBPgiYzBCzBcqy5AYFv9A&e= .

-- Regards, Keyur Cell: 2147845203

softwareklinic commented 6 years ago

CONFIRMED - its the proxy... i moved around the parameters as well but no luck... so as far as proxy is present --- its not working... So, now I remember that when i initially tried it out --- i tried from home.. but never worked from our internal server... atleast we know now for sure.

softwareklinic commented 6 years ago

@soulgalore Can this be patched quickly?

soulgalore commented 6 years ago

@softwareklinic I'm not sure if it is in (our) extension or in Chrome/Firefox or where the problem is?

softwareklinic commented 6 years ago

will try to find that

On Thu, Jun 28, 2018 at 2:38 PM Peter Hedenskog notifications@github.com wrote:

@softwareklinic https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_softwareklinic&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=l1FY-Jh5FsPiXIqYMwewXfiUa3LxtiGDCfrB4-NwrxU&s=yF1r3lhcdr1tyCTaVCG3IMCYurvxLbnf0YBKzwn3QeM&e= I'm not sure if it is in (our) extension or in Chrome/Firefox or where the problem is?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sitespeedio_sitespeed.io_issues_2074-23issuecomment-2D401149318&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=l1FY-Jh5FsPiXIqYMwewXfiUa3LxtiGDCfrB4-NwrxU&s=MQ1Lg-mFM14HkNEzcPmszNq4dR1GeoSRthEeloFt4ic&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AHMCiYfCLmJBnDWFVwJHkk-5FxO-2DrjNsMhks5uBTC4gaJpZM4UcveB&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=l1FY-Jh5FsPiXIqYMwewXfiUa3LxtiGDCfrB4-NwrxU&s=yBXWjcz2yhL4rkKznrSHu5LQ3zx-w3invKCqg50eIZI&e= .

-- Regards, Keyur Cell: 2147845203

softwareklinic commented 6 years ago

@soulgalore check this when u have a chance

https://github.com/Codeception/CodeceptJS/issues/561

Does this relate or feel connected by any means

softwareklinic commented 6 years ago

tried this on local browser with proxy enabled will try this on without proxy as well and see if any difference...

chrome://net-internals/#events

soulgalore commented 6 years ago

Does this relate or feel connected by any means

No. I don't think so.

Let explain better how it works: We use a WebExtension to block the traffic, so either we have setup the blocking wrong there (see for example https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/webRequest/onAuthRequired), like we need to give more permissions but I'm not sure, or it is a browser bug.

softwareklinic commented 6 years ago

The issue i'm seeing is with both FF and Chrome.. not just related to FF

soulgalore commented 5 years ago

So was this a problem running with a proxy and try block? Then it's a upstream issue for both browsers.

softwareklinic commented 5 years ago

Will try again and let u know

Regards Keyur

On Jan 26, 2019, at 1:27 PM, Peter Hedenskog notifications@github.com wrote:

So was this a problem running with a proxy and try block? Then it's a upstream issue for both browsers.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sitespeedio_sitespeed.io_issues_2074-23issuecomment-2D457858271&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=Vuw2jAx4U23oOU8OTQJYns9DDVK7rN-iSsPMMbxmj10&s=P3nVTyYUmd0VvIJmiQimCSLRODGZFqzSZqwi679iDVo&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AHMCibvPHdl-2DbrV-5F3ubH7DSCCQCgmUofks5vHKw5gaJpZM4UcveB&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=Vuw2jAx4U23oOU8OTQJYns9DDVK7rN-iSsPMMbxmj10&s=hXqZRbRfMENutG0z3SdWBD5SMNbkrAlQuEJnwkaH6qg&e= .

soulgalore commented 5 years ago

In 9.0 we implemented --blockDomainsExcept that doesn't use the extension to block, so hopefully it will work better see https://www.sitespeed.io/documentation/sitespeed.io/thirdparty/#block-all-3rd-parties

Best Peter