Explore automatically expiring uploaded profiles after 6 months

[digitarald]

Moving a prior discussion into this issue.

Uploaded profiles should expire automatically after 6 months and get deleted.

Why?

1. Data hygiene. Lean data practice:

   "try to keep [data] for only as long as necessary".

2. Half-life of profile relevance. Hypothesis: After 6 (or 12?) months (or 4/8+ Firefox releases) a profile of Firefox is unlikely to reflect the state of Firefox and/or the profiled website.

Cost savings on storage is a non-issue.

Why not?

1. Maybe one day there is a rare case that requires looking at an old profile?
2. Reference profiles uploaded for training purposes should be kept (can also be solved by uploading and loading from URL)

Possible investigations?

• Collect telemetry about the age of opened profiles
• Share opt-in option to limit expiration of a profile before uploading (pastebin style)
• Share opt-in option to extend the expiration of an uploaded profile
• Change expiration heuristic to time after last opening, so actively used profiles stay online.

Thoughts?

┆Issue is synchronized with this Jira Task

[past]

+1 from me.

[julienw]

In the investigations, we could add "send a mail to dev-platform to gather feedback".

I've mixed feelings because few things frustrate me more than a dead link... Old profiles can be useful for bug archeology (finding why some code is there).

But in the same time I totally understand the reasons for not keeping them.

[mstange]

I have looked at profiles older than 6 months many times. We have many performance issues that are still unfixed, sometimes after years. And for those that really are fixed, being able to see the original profile makes it easier to verify that they actually are fixed.

I also don't see much point in having dead links all over Bugzilla.

@jrmuizel's opinion on this is: "The work to do expiry is more cost than keeping the profiles."

Once we ship the new perf tool in the devtools, it'll probably make sense to have a separate profile storage for profiles from that source. And we can expire those profiles. But for profiles in Bugzilla, I'd really hope that we can keep them around.

[gregtatum]

I strongly agree with Markus here.

There seem to be two users, and they should be treated with different policies.

Profiles collected for Gecko hackers should live forever when they are attached to a bug, as they should be first class citizens of our version control and issue tracking process.

Profiles collected for general DevTools use (for webdevs) should have the best privacy controls people expect from Mozilla, and so should probably have a stricter default expiration policy.

[past]

From the reasons Harald provides, I think data hygiene is the most important one. Yes, old profiles can be useful and old bug reports could accumulate dead links, but this is true over time of any type of link on the web (test cases, broken sites, crash reports, you name it). Crash reports have very similar use cases and concerns, yet we don't keep them around forever for the same reasons. We could consider keeping profiles without any privacy-sensitive content for longer periods of time (assuming we can convince ourselves about that), but keeping around regular profiles with URLs is a long-term liability.

[digitarald]

Checking in with crash-stats, the data retention policy for raw crash pings is 6 months. After that only aggregates are kept (crashes per usage hours, per os, etc).

I also don't see much point in having dead links all over Bugzilla.

This a risk but is happening how to crash-stats as well.

There seem to be two users, and they should be treated with different policies.

This proposal does look like it is about webdevs, but there is a middle ground that we get from asking the external technical audience (sometimes called foxfooders) to report profiling issues. Technical enough to record profiles, we should not trust them to fully understand the amount of private data collected in profiles that the profiler cloud will keep forever.

@mstange @gregtatum following the argument of long-running bugs with profiles that stay relevant for analysis, what would be concerns with this: When a profile gets opened, reset the expiration timer.

[gregtatum]

When a profile gets opened, reset the expiration timer.

It's better than expiring after a short interval, and would lower the amount of stored profiles, but still has the downsides of broken links on valid information. I'd still like to find a path where Gecko engineers can have permanent profiles by default, even if we lock it down a bit for others.

[mikeconley]

I'm going to echo mstange's input here. I've definitely looked back at profiles > 6 months old. Having those go away would be unfortunate data loss. :/

[nbp]

I concur towards removing old profiles.

While I agree that profile are containing information which might be extracted even after 6 months, profiles are also a terrible way to communicate. Adding a link to a Gecko profile on bugzilla and 40 persons will find 20 different reasons on why the page is slow.

If profiles were to expire after X months/days, then profiles would become an investigation scene from which we draw conclusion in order to later take actions. Therefore having expiring profiles would be a way for developers to have either some step-to-reproduce or some investigation conclusion.

The world where profiles expire would make bugzilla a cleaner place to work with, instead of having 2 years old bugs opened without any investigation made. Such bug would either have an STR, some investigation work or closed as incomplete.

[zoepage]

To sum it up, the main issue why we want to delete the profiles after 6 months is data hygiene.

We assume users do not need to save their profiles in 99% of the time for 6 months. Sometimes someone needs just to share the profile really quick and knows already it can be deleted in 2 weeks. Sometimes they know already they will need it for the next year.

An idea that might help with keeping the important profiles available could be to give the user an option in the addon to set a duration for storing the profile.

Something like here

So before we get a proper profile management, this might be a good start as we would have to make the duration of storage variable anyway at some point.

Thoughts?

[fqueze]

I thought some more about this because I also often look at profiles that are more than 6 months old and would hate to have plenty of dead links in bugzilla, but I also think most profiles cointain plenty of information that shouldn't be retained forever.

I think profiles fall mostly in 2 categories:

• profiles captured when something seems to not be performing well, shared as a way to request that someone else looks at the data to figure out if there are bugs to be found. I think expiring these after 6 months would make a lot of sense.
• profiles in which a bug has already been identified, that are shared as a proof that the bug exists, and to provide information to the developer about what's happening and what needs fixing. These will typically be profiles from the first category that have already received some analysis by an engineer, and the url will usually point to a subset of the profile that highlights the problem.

Assuming my reasoning is correct, a good default behavior could be to make profiles expire after 6 months by default, but to provide a way to make the profile never expire each time we generate a new tiny url. I mean if I'm generating a tiny url for a subview of a profile that would otherwise have expired in 6 months, then the data shouldn't expire.

This means we would have expiring and non-expiring urls. It could be nice to make these urls different, so that when seeing a bug report we can notice when a valuable profile that could expire needs to be saved.

[digitarald]

We assume users do not need to save their profiles in 99% of the time for 6 months.

That is probably more true for 1 year. 6 months trips of many performance engineers that we talked to.

An idea that might help with keeping the important profiles available could be to give the user an option in the addon to set a duration for storing the profile.

The addon is probably not the best place, as profiles can also come from many other places (startup profiling, perfherder, imports, etc).

Assuming my reasoning is correct, a good default behavior could be to make profiles expire after 6 months by default, but to provide a way to make the profile never expire each time we generate a new tiny url.

The way I understand the first part is that we reset the profile expiry countdown each time somebody generates a bitly URL. The alternative I could see that you are describing an an option in the share dialog to set expiry to never? Could you help me understand?