sitespeedio / sitespeed.io

sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.
https://www.sitespeed.io/
MIT License
4.75k stars 603 forks source link

Get same result for use --multi script #2915

Closed EasilyTest closed 4 years ago

EasilyTest commented 4 years ago

hi @soulgalore ,i have a problem when use --multi to run script npm :sitespeed.io --preScript login.js --multi list.js

then get a same result for pages.html. it look like just show the result(Summary.Metric/RUM Timing Metrics/screenshots) of root path url example: http://example.com/example/#/example/example.do http://example.com/example/#/example/example1.do

soulgalore commented 4 years ago

Hi @zxf-nobug can you share more more info so I can reproduce?

EasilyTest commented 4 years ago

Hi @soulgalore ,This is what I tested: If the page has more than one url like http://example.com/example/#/example/example.do http://example.com/example/#/example/example1.do Just the interface is different,but when run Scripts 'sitespeed.io --preScript login.js --multi list.js' ‘login.js’ is my login information,and ’list.js’ is the list to execute: module.exports = async function(context, commands) { await commands.measure.start('http://example.net/xxx/#/xxx/interface1','alias'); return commands.measure.start('http://example.net/xxx/#/xxx/interface2','alias'); }; When the script is finished,it will get the same result on pages.html,Looks like only recorded the last result,If the urls are not similar the result is true.There is the correct test result of the corresponding URL in the har file, but it is not shown(including the screenshot),

I want to get test results of multiple similar URLs

Is there any way to solve this problem?

softwareklinic commented 4 years ago

Like its mentioned in documentation you have to use a dummy query parameter to get it measured

Test the same page multiple times within the same run # If you for some reason want to test the same URL within the same run multiple times, it will not work out of the box since the current version create the result files using the URL. For example testing https://www.sitespeed.io/ two times, will break since the second access will try to overwrite the first one.

But there is a hack you can do. If you add a dummy query parameter (and give the page an alias) you can test them twice a

module.exports = async function(context, commands) { await commands.measure.start('https://www.sitespeed.io/', 'HomePage');

// Do something smart that then make you need to test the same URL again
// ...

return commands.navigate('https://www.sitespeed.io/?dummy', 'BackToHomepage');

};

EasilyTest commented 4 years ago

Hi @softwareklinic ,Thanks for your help,But I don't think it's what I want.

I tried to use the method in the document to get some data in pages.html. I think the data is correct,But I click to enter the Page summary the result will be the same. Although the URL is similar, but actually the interface and page data are totally different. I should get a totally different HTML and screenshot. Even if the data will not change, the screenshot is completely consistent, which is obviously incorrect. Moreover, the correct data has been obtained in browsertime.har, When nodejs is running, see some performance results I want in console .

Could you provide some ideas or solutions?

softwareklinic commented 4 years ago

I can send you a sample JS -- if that helps ?

softwareklinic commented 4 years ago

You can try to check below script and that should help... Let me know -- or If you can share your script (masking the datapoints) -- I can take a look at that as well.

module.exports = async function(context, commands) { // Navigate to a URL, but do not measure the URL

try {

    const webdriver = context.selenium.webdriver;
    const driver = context.selenium.driver;

    await commands.navigate('https://www.domain.com/test.html');

    await commands.measure.start('SSO Login >> MDN Selection');
    await commands.addText.byId('xxxxxxxxx', 'IDToken1');
    await commands.addText.byId('xxxxxxxxx', 'IDToken2');

    await commands.click.byIdAndWait('login-submit');
    await commands.wait.byTime(30000);
    await commands.measure.stop();

    await commands.click.byIdAndWait('check0');

    await commands.measure.start('Continue >> Smartphones');
    await commands.click.bySelectorAndWait('#page > div > div:nth-child(104) > div.aem-page.page > div > div.aem-container.aem-Grid.aem-Grid--12.aem-Grid--default--12 > div > div.aem-container.aem-Grid.aem-Grid--12.aem-Grid--default--12 > div > div.grid > div > div > div.Row__StyledRow-sc-1oqylrf-0.fWtJzy.pad12.onlyLeftPad > button');
    await commands.wait.byTime(20000);

    // Below code is to extract the session id cookie and add as a extra measurement on sitespeed summary
    const ig_session = driver.manage().getCookie('digital_ig_session') 
    const am_id = driver.manage().getCookie('amID')
    Promise.all([ig_session, am_id]).then(function(values) {
        var jsonString = JSON.stringify(values,null, 2);

    commands.measure.add('session_id', values[0].value);
    commands.measure.add('am_id', values[1].value);

    });              

    await commands.wait.byTime(35000);
    //await commands.wait.byXpath("/html/body/div[2]/div/div[61]/div[1]/div/div[1]/div/div[1]/div/div[1]/section/div/div[3]/div[2]/div/div/form/div[1]/div[1]/div/div/input",30000);
    await commands.measure.stop();

    return; //commands.measure.stop();

} catch (e) { // We try/catch so we will catch if the the input fields can't be found // The error is automatically logged in Browsertime an rethrown here // We could have an alternative flow ... // else we can just let it cascade since it catched later on and reported in // the HTML throw e; } };

softwareklinic commented 4 years ago

Most important thing is measure start and stop - -which will let you define custom name for the step.

EasilyTest commented 4 years ago

Hi @softwareklinic You may not understand what I'm trying to say. I use '--preScript login.js --multi list.js' In the nodejs(v10.12.0) environment. Let '--multi list.js' get the URL data after login through ‘--preScript login.js’, So I can measure some URLs after I need to login .Although the previous URLs are similar, they are actually completely different pages and interfaces.

Data is normal in page.html, but click to enter Page summary(SUMMARY/Waterfall/Browser Metrics/Coach/PageXray/Third party All of these menus are the last, I think the right data is in Har, they are not shown ),path:/sitespeed-result/url/2020-03-12-17-24-29/pages/url/data/browsertime.har.

But I only got last result(including screenshots),There are two different pages, I should get different screenshots instead of the same screenshots. For example, it's like you use Google to measure, but only get the final result .

softwareklinic commented 4 years ago

I use only one script to measure all urls 1st I use signon and that one i don’t measure and all the subsequent urls I do measure.

You can individually measure each url by using start and stop.

Again if you share script snippet it will be easier to solve.

Regards Keyur

On Mar 23, 2020, at 2:08 AM, Tester_Z notifications@github.com wrote:



Hi @softwareklinic https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_softwareklinic&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=omeAPtW91tsXJSmgi_X5Bo6w5hMe6jc0v3dvkom8cRA&s=cUZiywWb-QI7xbDOUjh1DABRigfvnxNmVqnRqk-Qcdo&e= You may not understand what I'm trying to say. I use '--preScript login.js --multi list.js' In the nodejs(v10.12.0) environment. Let '--multi list.js' get the URL data after login through ‘--preScript login.js’, So I can measure some URLs after I need to login .Although the previous URLs are similar, they are actually completely different pages and interfaces.

But I only got last result(including screenshots),There are two different pages, I should get different screenshots instead of the same screenshots. For example, it's like you use Google to measure, but only get the final result .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sitespeedio_sitespeed.io_issues_2915-23issuecomment-2D602422540&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=omeAPtW91tsXJSmgi_X5Bo6w5hMe6jc0v3dvkom8cRA&s=MAQxlBU-Zp3nhU4j1-i01NhmWmgmk176nI2SYWzuUD0&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABZQFCOY3ZEFOSMSHN374FLRI4DHJANCNFSM4LFSDM3A&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=omeAPtW91tsXJSmgi_X5Bo6w5hMe6jc0v3dvkom8cRA&s=Azuk1fZ40AH2lX9nNleYWjInpzoAR0zB_gfDN4EyLbk&e= .

EasilyTest commented 4 years ago

I use only one script to measure all urls 1st I use signon and that one i don’t measure and all the subsequent urls I do measure. You can individually measure each url by using start and stop. Again if you share script snippet it will be easier to solve. Regards Keyur On Mar 23, 2020, at 2:08 AM, Tester_Z notifications@github.com wrote:  Hi @softwareklinic https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_softwareklinic&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=omeAPtW91tsXJSmgi_X5Bo6w5hMe6jc0v3dvkom8cRA&s=cUZiywWb-QI7xbDOUjh1DABRigfvnxNmVqnRqk-Qcdo&e= You may not understand what I'm trying to say. I use '--preScript login.js --multi list.js' In the nodejs(v10.12.0) environment. Let '--multi list.js' get the URL data after login through ‘--preScript login.js’, So I can measure some URLs after I need to login .Although the previous URLs are similar, they are actually completely different pages and interfaces. But I only got last result(including screenshots),There are two different pages, I should get different screenshots instead of the same screenshots. For example, it's like you use Google to measure, but only get the final result . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sitespeedio_sitespeed.io_issues_2915-23issuecomment-2D602422540&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=omeAPtW91tsXJSmgi_X5Bo6w5hMe6jc0v3dvkom8cRA&s=MAQxlBU-Zp3nhU4j1-i01NhmWmgmk176nI2SYWzuUD0&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABZQFCOY3ZEFOSMSHN374FLRI4DHJANCNFSM4LFSDM3A&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=omeAPtW91tsXJSmgi_X5Bo6w5hMe6jc0v3dvkom8cRA&s=Azuk1fZ40AH2lX9nNleYWjInpzoAR0zB_gfDN4EyLbk&e= .

Because I use the local system, I am not allowed to send scripts to you. I will try to reproduce this problem on other websites.This problem looks like clicking on different interfaces on the same page.

If it succeeds, i will reply to you.Thank you for your help!

EasilyTest commented 4 years ago

Hi @softwareklinic Let me give you an example. My script is like this:

module.exports = async function(context, commands) { await commands.measure.start('http://domain/activity/#/adManage/carouselManage','1'); await commands.measure.start('http://domain/activity/#/adManage/entryPositionManage','2'); return commands.measure.start('http://domain/activity/#/adManage/appStartPage','3'); };

There is a symbol '#' in the URL,I'm not sure if it's this symbol that just got the last piece of data.After ‘http://domain/activity/#/adManage/’ are some of my interfaces, which are actually displayed on different pages

regards

soulgalore commented 4 years ago

@zxf-nobug do you run with the --spa flag see https://www.sitespeed.io/documentation/sitespeed.io/spa/? By default everything after a # isn't used in the file name. If you enable --spa the full name is used.

EasilyTest commented 4 years ago

@soulgalore @softwareklinic I am indebted for the honour of your assistance. run with the --spa got the right data.I'll close this issuse.

softwareklinic commented 4 years ago

You are very welcome.

http://www.verizon.com

Keyur Shah

Associate Fellow - Enterprise Architecture 2147845203

"Defects & waste slow down the progress" - Identify, measure and eliminate.

On Thu, Mar 26, 2020 at 9:04 PM Tester_Z notifications@github.com wrote:

@soulgalore https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_soulgalore&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=zok7iHTnf3Az6SoMA_aASW1LKr-WbXOMswcii_-G8wI&s=yqweclJDomzJKJXJ9QdrBmAR_tbHdVujr3rZA5UaTWQ&e= @softwareklinic https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_softwareklinic&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=zok7iHTnf3Az6SoMA_aASW1LKr-WbXOMswcii_-G8wI&s=gL9ZnS2-4wUHKu_wtJJ12MWm-HtJB2mS-UJ_U4oUXQY&e= I am indebted for the honour of your assistance. run with the --spa got the right data.I'll close this issuse.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sitespeedio_sitespeed.io_issues_2915-23issuecomment-2D604776592&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=zok7iHTnf3Az6SoMA_aASW1LKr-WbXOMswcii_-G8wI&s=SpE-ufi5l3Lt56TB0K-eqLfjbl7iEh1kaNS6huGEM-Y&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABZQFCPDTZVZASXAYB2HJP3RJQCSHANCNFSM4LFSDM3A&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=QdU9ynK_Pna3a4TlNm0IqSkTvAoOvgSYGNrfxhJcJgM&m=zok7iHTnf3Az6SoMA_aASW1LKr-WbXOMswcii_-G8wI&s=cXY4-uiiiAirQbIqW7r-g-GvkvDCpJHIxSVBxSeZVbQ&e= .