search

mozilla / bugbot

A Mozilla release management tool to send reminders to Firefox developers and improve Bugzilla metadata
BSD 3-Clause "New" or "Revised" License
41 stars 68 forks source link

Add new rule for monitoring performance alert resolution changes. #2535

Closed gmierz closed 1 week ago

gmierz commented 3 weeks ago

This PR adds a new rule to monitor perfalert resolution changes. It will send out an email which provides a summary of all the performance alerts which have had their resolution changed within the last day. It includes the resolution comment, and author in the email as well.

gmierz commented 3 weeks ago

Sample output from a local dry run:

$ python -m bugbot.rules.perfalert_resolved_regression
2024-11-05 17:08:33,648 - INFO - Run rule perfalert_resolved_regression.py
2024-11-05 17:08:33,652 - DEBUG - Starting new HTTPS connection (1): bugzilla.mozilla.org:443
2024-11-05 17:08:38,904 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bug?include_fields=id&include_fields=resolution&include_fields=summary&include_fields=groups&f3=creation_ts&o3=greaterthan&v3=2024-10-01T00%3A00%3A00Z&f1=regressed_by&o1=isnotempty&f2=keywords&o2=allwords&v2=regression&v2=perf-alert&f4=resolution&o4=changedafter&v4=2024-11-05&f5=status_whiteboard&o5=notsubstring&v5=%5Bno-nag%5D&product=JSS&product=Focus&product=DevTools&product=NSS&product=Firefox+Build+System&product=Testing&product=Toolkit&product=Firefox&product=Firefox+for+iOS&product=WebExtensions&product=Core&product=Fenix&product=Release+Engineering&product=Developer+Infrastructure&product=Remote+Protocol&product=NSPR&product=GeckoView&product=External+Software+Affecting+Firefox&count_only=1 HTTP/11" 200 None
2024-11-05 17:08:38,909 - DEBUG - Starting new HTTPS connection (1): bugzilla.mozilla.org:443
2024-11-05 17:08:43,205 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bug?include_fields=id&include_fields=resolution&include_fields=summary&include_fields=groups&f3=creation_ts&o3=greaterthan&v3=2024-10-01T00%3A00%3A00Z&f1=regressed_by&o1=isnotempty&f2=keywords&o2=allwords&v2=regression&v2=perf-alert&f4=resolution&o4=changedafter&v4=2024-11-05&f5=status_whiteboard&o5=notsubstring&v5=%5Bno-nag%5D&product=JSS&product=Focus&product=DevTools&product=NSS&product=Firefox+Build+System&product=Testing&product=Toolkit&product=Firefox&product=Firefox+for+iOS&product=WebExtensions&product=Core&product=Fenix&product=Release+Engineering&product=Developer+Infrastructure&product=Remote+Protocol&product=NSPR&product=GeckoView&product=External+Software+Affecting+Firefox&limit=100&order=bug_id&offset=0 HTTP/11" 200 None
2024-11-05 17:08:43,210 - DEBUG - Starting new HTTPS connection (1): bugzilla.mozilla.org:443
2024-11-05 17:08:43,820 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bug?id=1925081%2C1927104%2C1928089%2C1928943%2C1929201&include_fields=id HTTP/11" 200 None
2024-11-05 17:08:43,823 - DEBUG - Starting new HTTPS connection (1): bugzilla.mozilla.org:443
2024-11-05 17:08:43,825 - DEBUG - Starting new HTTPS connection (2): bugzilla.mozilla.org:443
2024-11-05 17:08:44,434 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bug/1925081/comment?ids=1927104&ids=1928089&ids=1928943&ids=1929201&include_fields=text&include_fields=creation_time&include_fields=author HTTP/11" 200 None
2024-11-05 17:08:44,436 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bug/1925081/history?ids=1927104&ids=1928089&ids=1928943&ids=1929201 HTTP/11" 200 None
2024-11-05 17:08:44,445 - DEBUG - Starting new HTTPS connection (1): bugzilla.mozilla.org:443
2024-11-05 17:08:45,048 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bitly/shorten?url=https%3A%2F%2Fbugzilla.mozilla.org%2Fbuglist.cgi%3Finclude_fields%3Did%26include_fields%3Dresolution%26include_fields%3Dsummary%26include_fields%3Dgroups%26f3%3Dcreation_ts%26o3%3Dgreaterthan%26v3%3D2024-10-01T00%253A00%253A00Z%26f1%3Dregressed_by%26o1%3Disnotempty%26f2%3Dkeywords%26o2%3Dallwords%26v2%3Dregression%26v2%3Dperf-alert%26f4%3Dresolution%26o4%3Dchangedafter%26v4%3D2024-11-05%26f5%3Dstatus_whiteboard%26o5%3Dnotsubstring%26v5%3D%255Bno-nag%255D%26product%3DJSS%26product%3DFocus%26product%3DDevTools%26product%3DNSS%26product%3DFirefox%2BBuild%2BSystem%26product%3DTesting%26product%3DToolkit%26product%3DFirefox%26product%3DFirefox%2Bfor%2BiOS%26product%3DWebExtensions%26product%3DCore%26product%3DFenix%26product%3DRelease%2BEngineering%26product%3DDeveloper%2BInfrastructure%26product%3DRemote%2BProtocol%26product%3DNSPR%26product%3DGeckoView%26product%3DExternal%2BSoftware%2BAffecting%2BFirefox HTTP/11" 200 None
2024-11-05 17:08:45,053 - INFO - 
****************************
* DRYRUN: not sending mail *
****************************
Receivers: ['calixte@mozilla.com', 'sylvestre@mozilla.com', 'mcastelluccio@mozilla.com', 'smujahid@mozilla.com', 'jwalker@mozilla.com', 'perfalert-activity@mozilla.com', 'sparky@mozilla.com']
Message:
Content-Type: multipart/mixed; boundary="===============7554564531139704757=="
MIME-Version: 1.0
From: xxx@xxxx.xxx
To: calixte@mozilla.com, sylvestre@mozilla.com, mcastelluccio@mozilla.com, smujahid@mozilla.com, jwalker@mozilla.com, perfalert-activity@mozilla.com, sparky@mozilla.com
Subject: [bugbot] PerfAlert regressions whose resolution has changed recently for the 2024-11-05
Cc: 
Bcc: 
X-Mailer: bugbot

--===============7554564531139704757==
Content-Type: text/html; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html lang="en-us">
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <p>Hi there,</p>

        <p>The following bug list contains performance alerts whose resolution has changed in the last week:</p>
<table style="border:1px solid black;border-collapse:collapse;" border="1">
    <thead>
        <tr>
            <th>Bug</th>
            <th>Summary</th>
            <th>Resolution</th>
        </tr>
    </thead>
    <tbody>
        <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1929201">1929201</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>25.51 - 3.21% speedometer3 TodoMVC-Preact-Complex-DOM/CompletingAllItems/Async / speedometer3 wallclock-for-tracking-only + 9 more (Linux, OSX) regression on Wed October 23 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> I think given the regression/progression magnitude, and that the patch is a known performance trade-off (not strict improvement per se), we should not worry too much about those two regressions.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by emilio@crisal.io</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1928943">1928943</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>34316.95 - -3932% browser-console:parent-process memory / browser-console:parent-process memory (Linux) regression on Tue October 29 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> https://hg.mozilla.org/mozilla-central/rev/89128774a49c
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by smolnar@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1928089">1928089</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>1.22% Base Content Heap Unclassified (Linux) regression on Wed October 23 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> No resolution comment provided.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by rjesup@jesup.org</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927104">1927104</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>627.86 - 2.16% pdfpaint tiling-pattern-box.pdf / pdfpaint issue14990.pdf + 24 more (Windows) regression on Thu October 17 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> (In reply to Donal Meehan [:dmeehan] from comment #8)
> :calixte/:lsalman any updates on this?
> We are tracking this for Fx133 but next week is the final week of beta. 
> What's the next step here?

The issue was resolved between both bug 1926668, and further in bug 1928106 for 134, but it's up to Calixte whether bug 1928106 is worth uplifting to 133.

Otherwise, I am okay with this simply being wontfix/fix-optional for 133, as it's expected to be a mixed bag of improvements and regressions on a testsuite as wide as pdfpaint.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by lsalzman@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1925081">1925081</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>4.31% pdfpaint issue9534_reduced.pdf (Linux) regression on Mon October 7 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> Spoke with :rebeca after looking at the regressing android patch and the linux platform where the regression was noted I am closing this as an invalid bug, apologies for all the confusion!
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by aglavic@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
        </tr>
    </tbody>
</table>
        <p>
            Sincerely,
            <br>
            BugBot
        </p>
        <p>
                See the search query on <a href="https://mzl.la/3CdrbDk">Bugzilla</a>.
            </p>

        <p>
            See the source / report issues on the <a href="https://github.com/mozilla/bugbot/">GitHub repo</a>
        </p>
    </body>
</html>
--===============7554564531139704757==--
gmierz commented 2 weeks ago

I've added some needinfo functionality to this rule. It will now create a needinfo when we can't find a reason for why the bug was resolved (essentially from status changes that don't contain a comment). I also added a new column to the table. Here's the new output:

2024-11-07 17:36:47,651 - INFO - The bugs: 1927688
 will be autofixed with:
{'comment': {'body': 'A comment containing a reason for why the performance regression was resolved as INVALID could not be found. It should provided when the status of the bug is changed.\n\n:dmeehan, since you resolved this bug, could you provide a comment explaining the resolution? If one has already been provided, this needinfo can be ignored/removed.\n\nIf you need additional information/help, reach out in [#perftest](https://matrix.to/#/#perftest:mozilla.org), or [#perfsheriffs](https://matrix.to/#/#perfsheriffs:mozilla.org) on Element.\n\nFor more information, please visit [BugBot documentation](https://wiki.mozilla.org/BugBot#perfalert_resolved_regression.py).\n'}, 'flags': [{'name': 'needinfo', 'requestee': 'dmeehan@mozilla.com', 'status': '?', 'new': 'true'}]}
2024-11-07 17:36:47,664 - DEBUG - Starting new HTTPS connection (1): bugzilla.mozilla.org:443
2024-11-07 17:36:48,099 - DEBUG - https://bugzilla.mozilla.org:443 "GET /rest/bitly/shorten?url=https%3A%2F%2Fbugzilla.mozilla.org%2Fbuglist.cgi%3Finclude_fields%3Did%26include_fields%3Dresolution%26include_fields%3Dsummary%26include_fields%3Dgroups%26f3%3Dcreation_ts%26o3%3Dgreaterthan%26v3%3D2024-10-01T00%253A00%253A00Z%26f1%3Dregressed_by%26o1%3Disnotempty%26f2%3Dkeywords%26o2%3Dallwords%26v2%3Dregression%26v2%3Dperf-alert%26f4%3Dresolution%26o4%3Dchangedafter%26v4%3D2024-11-07%26f5%3Dstatus_whiteboard%26o5%3Dnotsubstring%26v5%3D%255Bno-nag%255D%26product%3DWebExtensions%26product%3DTesting%26product%3DCore%26product%3DJSS%26product%3DFenix%26product%3DFirefox%26product%3DRelease%2BEngineering%26product%3DRemote%2BProtocol%26product%3DNSS%26product%3DDevTools%26product%3DToolkit%26product%3DFocus%26product%3DFirefox%2Bfor%2BiOS%26product%3DGeckoView%26product%3DExternal%2BSoftware%2BAffecting%2BFirefox%26product%3DDeveloper%2BInfrastructure%26product%3DFirefox%2BBuild%2BSystem%26product%3DNSPR HTTP/11" 200 None
2024-11-07 17:36:48,104 - INFO - 
****************************
* DRYRUN: not sending mail *
****************************
Receivers: ['calixte@mozilla.com', 'sylvestre@mozilla.com', 'mcastelluccio@mozilla.com', 'smujahid@mozilla.com', 'jwalker@mozilla.com', 'perfalert-activity@mozilla.com', 'sparky@mozilla.com']
Message:
Content-Type: multipart/mixed; boundary="===============5440169953358115079=="
MIME-Version: 1.0
From: xxx@xxxx.xxx
To: calixte@mozilla.com, sylvestre@mozilla.com, mcastelluccio@mozilla.com, smujahid@mozilla.com, jwalker@mozilla.com, perfalert-activity@mozilla.com, sparky@mozilla.com
Subject: [bugbot][autofix] PerfAlert regressions whose resolution has changed recently for the 2024-11-07
Cc: 
Bcc: 
X-Mailer: bugbot

--===============5440169953358115079==
Content-Type: text/html; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html lang="en-us">
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <p>Hi there,</p>

        <p>The following bug list contains performance alerts whose resolution has changed in the last week:</p>
<table style="border:1px solid black;border-collapse:collapse;" border="1">
    <thead>
        <tr>
            <th>Bug</th>
            <th>Summary</th>
            <th>Status</th>
            <th>Resolution</th>
        </tr>
    </thead>
    <tbody>
        <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1929854">1929854</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>19.92 - 5.35% motionmark-1-3 cpuTimePageload / speedometer3 cpuTimePageload + 2 more (Linux) regression on Sat November 2 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> This was an expected reversion from the backout done in Bug 1927058. Turning the GPU process off-by-default for X11 undid the associated performance changes. And as we can see, they are tradeoffs. These are the ones that are worse off *without* a GPU process.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by bwerth@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>
                --- -> INVALID
            </td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1929227">1929227</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>7.88% cnn ContentfulSpeedIndex (OSX) regression on Wed October 30 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> https://hg.mozilla.org/mozilla-central/rev/a5b67007fef7
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by chorotan@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>
                --- -> FIXED
            </td>
        </tr>
    <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1928180">1928180</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>9.51% tresize (Linux) regression on Mon October 21 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> Yeah, seems like we're good here, lmk if anyone disagrees :)

*** This bug has been marked as a duplicate of bug 1927111 ***
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by emilio@crisal.io</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>
                DUPLICATE -> FIXED
            </td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927807">1927807</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>%X- %Y regression testing bug</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> This bug was used for testing.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by gmierz2@outlook.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>
                --- -> INVALID
            </td>
        </tr>
    <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927688">1927688</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>3.5% speedometer BackboneJS-TodoMVC/CompletingAllItems/Async (OSX) regression on Thu October 24 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> No resolution comment provided.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by dmeehan@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>
                --- -> INVALID
            </td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927476">1927476</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>0.16% installer size (OSX) regression on Mon October 21 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> https://hg.mozilla.org/mozilla-central/rev/8a9222246491
https://hg.mozilla.org/mozilla-central/rev/484ab36510c0
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by chorotan@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>
                --- -> FIXED
            </td>
        </tr>
    </tbody>
</table>
        <p>
            Sincerely,
            <br>
            BugBot
        </p>
        <p>
                See the search query on <a href="https://mzl.la/3AmGRUn">Bugzilla</a>.
            </p>

        <p>
            See the source / report issues on the <a href="https://github.com/mozilla/bugbot/">GitHub repo</a>
        </p>
    </body>
</html>
gmierz commented 2 weeks ago

Example of the table after the fixes:

2024-11-13 11:37:37,949 - INFO - 
****************************
* DRYRUN: not sending mail *
****************************
Receivers: ['calixte@mozilla.com', 'sylvestre@mozilla.com', 'mcastelluccio@mozilla.com', 'smujahid@mozilla.com', 'jwalker@mozilla.com', 'perfalert-activity@mozilla.com', 'sparky@mozilla.com']
Message:
Content-Type: multipart/mixed; boundary="===============5026421538734706422=="
MIME-Version: 1.0
From: xxx@xxxx.xxx
To: calixte@mozilla.com, sylvestre@mozilla.com, mcastelluccio@mozilla.com, smujahid@mozilla.com, jwalker@mozilla.com, perfalert-activity@mozilla.com, sparky@mozilla.com
Subject: [bugbot] PerfAlert regressions whose resolution has changed recently for the 2024-11-13
Cc: 
Bcc: 
X-Mailer: bugbot

--===============5026421538734706422==
Content-Type: text/html; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html lang="en-us">
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <p>Hi there,</p>

        <p>The following bug list contains performance alerts whose resolution has changed in the last week:</p>
<table style="border:1px solid black;border-collapse:collapse;" border="1">
    <thead>
        <tr>
            <th>Bug</th>
            <th>Summary</th>
            <th>Status</th>
            <th>Resolution</th>
        </tr>
    </thead>
    <tbody>
        <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1930747">1930747</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>8.62 - 7.33% cpstartup content-process-startup / cpstartup content-process-startup (Linux) regression on Thu October 31 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> Unfortunately we likely have to accept this regression, because we shouldn't keep deferring the update to a more recent ICU. Otherwise we run into cross-browser compatibility issues (e.g. bug 1909112 or bug 1919905) and we run behind in sec-updates (e.g. 1871635, 1871636, 1871623).
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by andrebargull@googlemail.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>--- -> WONTFIX</td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927052">1927052</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>2.07 - 1.93% speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync / speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync + 3 more (Windows) regression on Sat October 12 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> The update is no update -- and realistically, WONTFIX.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by mgaudet@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>--- -> WONTFIX</td>
        </tr>
    </tbody>
</table>
        <p>
            Sincerely,
            <br>
            BugBot
        </p>
        <p>
                See the search query on <a href="https://mzl.la/4ew8sjE">Bugzilla</a>.
            </p>

        <p>
            See the source / report issues on the <a href="https://github.com/mozilla/bugbot/">GitHub repo</a>
        </p>
    </body>
</html>

Example of a needinfo:

2024-11-13 11:41:05,737 - INFO - The bugs: 1929843
 will be autofixed with:
{'comment': {'body': 'A comment containing a reason for why the performance regression was resolved as INVALID could not be found. It should provided when the status of the bug is changed.\n\n:bacasandrei, since you resolved this bug, could you provide a comment explaining the resolution? If one has already been provided, this needinfo can be ignored/removed.\n\nIf you need additional information/help, reach out in [#perftest](https://matrix.to/#/#perftest:mozilla.org), or [#perfsheriffs](https://matrix.to/#/#perfsheriffs:mozilla.org) on Element.\n\nFor more information, please visit [BugBot documentation](https://wiki.mozilla.org/BugBot#perfalert_resolved_regression.py).\n'}, 'flags': [{'name': 'needinfo', 'requestee': 'bacasandrei@mozilla.com', 'status': '?', 'new': 'true'}]}
gmierz commented 2 weeks ago

Example run for the email:

****************************
* DRYRUN: not sending mail *
****************************
Receivers: ['calixte@mozilla.com', 'sylvestre@mozilla.com', 'mcastelluccio@mozilla.com', 'smujahid@mozilla.com', 'jwalker@mozilla.com', 'perfalert-activity@mozilla.com', 'sparky@mozilla.com']
Message:
Content-Type: multipart/mixed; boundary="===============4356218612956373498=="
MIME-Version: 1.0
From: xxx@xxxx.xxx
To: calixte@mozilla.com, sylvestre@mozilla.com, mcastelluccio@mozilla.com, smujahid@mozilla.com, jwalker@mozilla.com, perfalert-activity@mozilla.com, sparky@mozilla.com
Subject: [bugbot] PerfAlert regressions whose resolution has changed recently for the 2024-11-14
Cc: 
Bcc: 
X-Mailer: bugbot

--===============4356218612956373498==
Content-Type: text/html; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html lang="en-us">
    <head>
        <meta charset="utf-8">
    </head>
    <body>
        <p>Hi there,</p>

        <p>The following bug list contains performance alerts whose resolution has changed in the last week:</p>
<table style="border:1px solid black;border-collapse:collapse;" border="1">
    <thead>
        <tr>
            <th>Bug</th>
            <th>Summary</th>
            <th>Status</th>
            <th>Resolution</th>
        </tr>
    </thead>
    <tbody>
        <tr bgcolor="#E0E0E0"
            >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1930747">1930747</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>8.62 - 7.33% cpstartup content-process-startup / cpstartup content-process-startup (Linux) regression on Thu October 31 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> Unfortunately we likely have to accept this regression, because we shouldn't keep deferring the update to a more recent ICU. Otherwise we run into cross-browser compatibility issues (e.g. bug 1909112 or bug 1919905) and we run behind in sec-updates (e.g. 1871635, 1871636, 1871623).
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by andrebargull@googlemail.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>--- -> WONTFIX</td>
        </tr>
    <tr >
            <td>
                <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1927052">1927052</a>
            </td>
            <td>
                <table>
                    <tr>
                        <td>2.07 - 1.93% speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync / speedometer EmberJS-Debug-TodoMVC/CompletingAllItems/Sync + 3 more (Windows) regression on Sat October 12 2024</td>
                    </tr>
                    <tr>
                        <td>
                            <b>Resolution Comment:</b> The update is no update -- and realistically, WONTFIX.
                        </td>
                    </tr>
                    <tr>
                        <td>
                            <i>Resolved by mgaudet@mozilla.com</i>
                        </td>
                    </tr>
                </table>
            </td>
            <td>RESOLVED</td>
            <td>--- -> WONTFIX</td>
        </tr>
    </tbody>
</table>
        <p>
            Sincerely,
            <br>
            BugBot
        </p>
        <p>
                See the search query on <a href="https://mzl.la/3Z5GPtl">Bugzilla</a>.
            </p>

        <p>
            See the source / report issues on the <a href="https://github.com/mozilla/bugbot/">GitHub repo</a>
        </p>
    </body>
</html>

Example run for the needinfo comment:

2024-11-14 10:14:54,268 - INFO - The bugs: 1927120
 will be autofixed with:
{'comment': {'body': 'A comment containing a reason for why the performance regression was resolved as FIXED could not be found. It should provided when the status of the bug is changed.\n\n:dao, since you resolved this bug, could you provide a comment explaining the resolution? If one has already been provided, this needinfo can be ignored/removed.\n\nIf you need additional information/help, reach out in [#perftest](https://matrix.to/#/#perftest:mozilla.org), or [#perfsheriffs](https://matrix.to/#/#perfsheriffs:mozilla.org) on Element.\n\nFor more information, please visit [BugBot documentation](https://wiki.mozilla.org/BugBot#perfalert_resolved_regression.py).\n'}, 'flags': [{'name': 'needinfo', 'requestee': 'dao+bmo@mozilla.com', 'status': '?', 'new': 'true'}]}
2024-11-14 10:14:54,269 - INFO - The bugs: 1929843
 will be autofixed with:
{'comment': {'body': 'A comment containing a reason for why the performance regression was resolved as INVALID could not be found. It should provided when the status of the bug is changed.\n\n:bacasandrei, since you resolved this bug, could you provide a comment explaining the resolution? If one has already been provided, this needinfo can be ignored/removed.\n\nIf you need additional information/help, reach out in [#perftest](https://matrix.to/#/#perftest:mozilla.org), or [#perfsheriffs](https://matrix.to/#/#perfsheriffs:mozilla.org) on Element.\n\nFor more information, please visit [BugBot documentation](https://wiki.mozilla.org/BugBot#perfalert_resolved_regression.py).\n'}, 'flags': [{'name': 'needinfo', 'requestee': 'bacasandrei@mozilla.com', 'status': '?', 'new': 'true'}]}
gmierz commented 1 week ago

Thanks @suhaibmujahid! I've committed the requested change, and added an entry to the wiki: https://wiki.mozilla.org/BugBot#perfalert_resolved_regression.py