search

Raku / user-experience

Identifying issues in and improving the Raku user experience
Artistic License 2.0
25 stars 5 forks source link

rakudobug@perl.org isn't easily discoverable #8

Closed timo closed 6 years ago

timo commented 8 years ago

A thread on the perl6-users mailing list prompted me to open this task.

A new user asked how to report bugs, another user suggested the rather time-consuming way of registering a bitcard account and using the "new ticket" form on rt.perl.org.

This is a nice way to lose potential bug reports. Not only is it long and involved, but it also requires users to create Yet Another Account.

We should make rakudobug@perl.org more prominent than just:

Thoughts?

CIAvash commented 8 years ago

I think we need a 'How to report bugs' page.

Maybe also mention it on FAQ.

moltar commented 8 years ago

We need to get with the modern times and allow people to "login with" (e.g. GitHub), or don't even ask for login. Ideally there should be smth like GitHub for tickets. RT is meh.

moritz commented 8 years ago

People love to hate RT, but I haven't yet seen a github project with 800+ issues that was able to handle them well.

zaucker commented 8 years ago

It is probably the frontend that people love or hate. And that could probably be worked around.

Doing it the Perl way probably means to have many different ways to submit and browse issues.

So I'd suggest that everybody complaining just opens an issue describing what s/he would prefer and as soon as there are no more pressing issues to resolve those issues about issue handling can be worked on.

My personal favorite for submitting issues is indeed eMail. Of course, just submitting issues by eMail without first checking will probably lead do multiple submissions. I wonder how big this problem is, though.

On Wed, 3 Feb 2016, Moritz Lenz wrote:

People love to hate RT, but I haven't yet seen a github project with 800+ issues that was able to handle them well.

Reply to this email directly or view it on GitHub: https://github.com/perl6/user-experience/issues/8#issuecomment-179181692

Oetiker+Partner AG tel: +41 62 775 9903 (direct) Fritz Zaucker +41 62 775 9900 (switch board) Aarweg 15 +41 79 675 0630 (mobile) CH-4600 Olten fax: +41 62 775 9905 Schweiz web: www.oetiker.ch

AlexDaniel commented 8 years ago

but I haven't yet seen a github project with 800+ issues that was able to handle them well.

What do you mean by “handle them well”? What's the problem? And how much should we care about developers looking through these bug reports?

I think that the most important thing is that it should be easy for users to report bugs. Everything else is less important.

And although I appreciate the ability to send an email to rakudobug@perl.org, let's face it – RT sucks. It is horrible. Even if you completely ignore the web interface and just send an email it is still far from being ok.

Much bigger issue is that although most people know how to report bugs, they still don't know how to comment on them… It is not obvious, and it is very energy consuming (compare that to just writing a comment into a textarea and pressing “submit”).

bitcard account? You've got to be kidding me. I've tried making one but it just doesn't work. RT is terribly broken. Every time I login it just disables everything, I can't even view the tickets while I'm logged in.

Open any bug report and click ｢Comment｣ button. You'll see this:

This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org

Time to display: 0.005367

»|« RT 4.0.18 Copyright 1996-2013 Best Practical Solutions, LLC.

Yeah, RT, fuck you too.

After 100 submitted tickets I can say that if you give me a chance to submit bugs on GitHub I'll be the happiest man. This, however, doesn't mean that we should move everything to GitHub. One way to do it is to mirror all bug reports to RT. This will make it more obvious how to report bugs (most developers know about GitHub, right?).

Anyway, I think that the current situation is not TIMTOWTDIish enough. To a point when it makes some people (me) cry. I hope that something could be done with it (but I don't think that there is a way to fix RT, so I'd much rather abandon the ship…).

tbrowder commented 8 years ago

On Wed, Feb 3, 2016 at 7:56 AM, Aleks-Daniel Jakimenko-Aleksejev notifications@github.com wrote:

but I haven't yet seen a github project with 800+ issues that was able to handle them well.

What do you mean by “handle them well”? What's the problem? And how much should we care about developers looking through these bug reports? ...

I say "amen" to most all Aleks says.

And I just looked, Bugzilla has a plugin for Git integration--might be worth looking into.

Cheers!

-Tom

zoffixznet commented 8 years ago

Perl 5 comes with with perlbug tool that you just run and it collects all the relevant info. Perhaps, something like that can ship with P6 too.

polettix commented 8 years ago

@AlexDaniel to be honest I didn't experience anything of what you wrote. It might depend on the fact that I have a bitcard account since a long time or just sheer luck, but I was able to log in correctly, look for bugs, open them, hit the dreaded "Comment" button and actually get a form for commenting back. I didn't go as far as commenting as I had nothing to say on the example bug I selected.

I'm not trying to push a "works on my PC" answer to your frustrations, but RT so far it provided a decent (although not really visually appealing IMHO) service to the Perl community at large and they deserve the right to a bug report before just tossing them away because "it does not work".

WRT the subject of the thread, I didn't follow the discussion in IRC but I would like to understand:

MadcapJake commented 8 years ago

Couldn't we create a form on rakudo.org or perl6.org that submits the email for you? The form could even search issues based on your subject line and then if you decide to continue as a comment, it would just plug the required RT# into the subject line of the email. It could either send it for you from the server or just open a mailto link with the details filled in from the form.

autarch commented 8 years ago

I agree that RT really doesn't provide a great user experience. Email isn't a great user experience for most folks either. They expect something pointy and clicky. GitHub isn't perfect, but it's pervasive, which is nice.

AlexDaniel commented 8 years ago

What do you mean by “handle them well”? What's the problem? And how much should we care about developers looking through these bug reports?

I say "amen" to most all Aleks says.

I'm not sure what you mean by that. To clarify, Perl 6 has always been about “torture the implementor on behalf of the user” kind of thing. So if we are looking for a balance between “the users are frustrated with the ticket system” and “the developers are frustrated with the ticket system” we should probably move as much pain as we can from the user side.

tbrowder commented 8 years ago

On Wed, Feb 3, 2016 at 8:31 AM, Flavio Poletti notifications@github.com wrote:

@AlexDaniel to be honest I didn't experience anything of what you wrote. It might depend on the fact that I have a bitcard account since a long time or just sheer luck, but I was able to log in correctly, look for bugs, open

Him Flavio. Good comments, but it may be the OS and browser. I meant to ask about that earlier.

I, too, have been a long-time RT account holder (with a bitcard account) but I don't remeber ever having any luck.

I have always tried to use RT on Linux (Debian for the past eight years or so) with either the Chrome or Firefox browser.

I still have problems with RT as others have had. When I login with my account I see NOTHING about Perl 5 or Perl 6 or help. I do notice words saying that this is a perlbug instance.

And, again, I have attempted to ask for help with the problem through the e-mail address shown there and have yet to receive a reply. I have gone to the RT site and haven't found any help with the initial user interface.

I vaguely remember the reason I joined years ago was for Perl 5 issues, but I think I had the same type problems then

Just my experience so far.

Best,

-Tom

P.S. With a new language, perhaps its time for a new bug tracker. And GCC seems to function very well with it and has for many years.

polettix commented 8 years ago

@tbrowder I've uses RT for some time as a CPAN Author and it mostly did its job but again, "works for me" is no excuse. My consideration was about providing feedback on issues to the devs of RT instead of just ditching it away for those very issues.

I'm anyway curious of the alternatives if we want to select something different, e.g. more in line with Perl 6 approach ("torture the implementor on behalf of the user") and what a casual user might expect ("... something pointy and clicky" as opposed to email). I have to admit I didn't understand the reference to GCC though.

tony-o commented 8 years ago

Does RT provide something that github’s issue tracking does not? The experience from a reporter/reporting perspective is kind of nice and you can attribute bugs/enhancements to milestones (or releases) and you get feedback on bugs you’ve filed or are interested in; RT seems to turn into a black hole but that could just be me not using it well.

To @moritz’s point, it doesn’t handle huge volumes of data well out of the box. I have private repos that have had 300-500 issues on them that are manageable but require administrative work to keep things organized, filed, and assigned to milestones/people. Essentially it requires someone to do the administrative side of a project manager.

Those things said, RT is pretty decent as far as bug tracking systems go. Github’s is nice from a “pointy clicky” perspective. Both require management and reporter/maintainer tooling knowledge or training.

uvtc commented 8 years ago

Some points:

zoffixznet commented 8 years ago

I'm not familiar with perlbug. From the first few paragraphs of its man page, I'm not sure what it's supposed to do. My guess is that folks won't use it unless there are some explicit and simple instructions for what they're supposed to do with it.

You run it and it asks you a few questions about the bug you're reporting, as well as prepares some info about your environment (OS, perl version, etc). Then, as I recall, it attempts to email the report and if that fails, it gives you what to mail and where to mail it and you just email it yourself.

I don't think that last part is too usable; maybe we could make it so if it fails to send email, it'd attempt to use the web API to create it, first.

ghost commented 8 years ago

I don't think "torture the implementors on behalf of the user" applies here – we've already got enough bugs in Perl 6 for doing that, without also having to fight with the (non-Perl 6) tool used to track them. Everyone's a user in this context (and the implementors insufficiently tortured, IMHO).

MadcapJake commented 8 years ago

Here's my first draft at an issue submitter design. I think the search results will accordion out between the subject field and version dropdowns. It will then fold back in if you click any of the textarea boxes. Each search result item will have a linked title of the issue on rt.perl.org and a button to "Change to Comment" or something that will prepend the RT# at the beginning of your subject. (Or should it replace the whole subject line?) I could also add a "miscellaneous" textarea but I feel that if you're really going to write it your way, then just use an email (agree?). All that these sections will do is provide some structure to the resultant email message.

moltar commented 8 years ago

perlbug: what if I have a problem installing Perl? How do I use perlbug then?

RT: It was good. But now we have better. Let's not be romantic about it. It's time to move on. The most widely used ticket system now is GitHub. It's also not perfect. But almost every developer has an account. If they don't, I'm sure GitHub made it VERY straightforward to onboard. After all, it's a business and they probably optimized the on boarding funnel to the max. There is code highlighting, you can link to images. Email notifications out of the box.

zoffixznet commented 8 years ago

@moltar

perlbug: what if I have a problem installing Perl? How do I use perlbug then?

You email to rakudobug@perl.org... assuming, of course, you don't have problems with your email :stuck_out_tongue_closed_eyes:

RT: It was good.

Was it? It seems ridiculously.. RIDICULOUSLY.. over-engineered to the point where you can't even link to the Perl 6 queue directly because of CSRF protection and you are presented with more than a dozen of search fields to do a search. I'm glad sanity prevailed and they stopped before adding such silly things as editing.

But almost every developer has an account

I know quite a few people without one, due to concerns over GitHub's terms of service.

I'm sure GitHub made it VERY straightforward to onboard

Yeah, many services offer a one-click subscription if you have, say, a Facebook account, but why the hell would I want to get "onboard" of some service I never intend to use? Multiple services get hacked yearly with all the emails and personal info leaked. Why would I want to increase my risk just so some group of people could fix their software? Each hoop your users have to jump through to report a bug filters out a portion of reporters unwilling to jump through it... which brings me back on topic:

@MadcapJake

Here's my first draft at an issue submitter design.

The one obvious thing I see missing is spam protection, like reCAPTCHA.

Also, to me, that form looks daunting. If a compiler implements a particular language version, why do I need to specify the langauge version as well? I've no idea what compiler version I'm using, how do I find it? Actually, all I'm reporting is a typo in the docs, do I need to fill all of that stuff out?

I'd go with the simplest form, like in the screenshot below, as the primary form and maybe have all of the fields you propose available in the "advanced" version of the form.

report

moritz commented 8 years ago

@AlexDaniel

What do you mean by “handle them well”? What's the problem? And how much should we care about developers looking through these bug reports?

What's the use of bug reports if developers don't look through them, find the most relevant, and fix them?

MadcapJake commented 8 years ago

@zoffixznet maybe the default should be the advanced form. Otherwise, no one will ever use it. "Path of least clicks" and all. The language drop down has no other available option, I could remove it but it's just there for future-proofing (and I think it's healthy to remind users of the distinction between language and compiler).

Also in regards to a simple form, perhaps what I could do is include a category dropdown and it would subsequently show different fields for each. For documentation it would just show "Is there a typo or missing information?", for articles it'd show "Is there an out-of-date article/blog that you're aware of?", and so on (some other categories could be "repl", "module usage", "module creation", "syntax error" "unexpected result", "crashes", "perl6.org", etc.). This could even be plugged into the subject line as an added way to filter bugs.

hoelzro commented 8 years ago

@tony-o

Does RT provide something that github’s issue tracking does not?

For me, the things that RT provides are dashboards for organizing issues and attachments. I have a dashboard for issues I've filed (which GH provides), but also for issues I've starred in RT, which I find interesting and have maybe considered fixing myself. As far as I know, the only way to "star" issues is to watch them for updates, assign yourself, or label it. Watching for updates gets you updates on the issue, but I can't ask the top level issues view to just "show me issues I'm watching". Assigning an issue to myself would communicate the wrong message (plus, can you have more than one assignee, if multiple people want to star an issue?), and labelling would just make labels noisy for everyone.

Regarding attachments, I know GH issues supports image attachments, but I thought that it doesn't allow arbitrary text files (although that may have changed). I make heavy use of RT's attachments to attach example code that demonstrates a bug. Granted, you can put that in the issue in fenced code blocks, but to me that's LTA.

@moltar

RT: It was good. But now we have better.

Which ways in particular make GH issues better than RT?

But almost every developer has an account. If they don't, I'm sure GitHub made it VERY straightforward to onboard. After all, it's a business and they probably optimized the on boarding funnel to the max.

True, but as @zoffixznet points out, not everyone wants to have a GH account.

here is code highlighting

I'll concede that.

you can link to images.

You can still attach images in RT. Granted, I don't think it will display them inline, which is definitely LTA.

Email notifications out of the box.

Doesn't RT do this too?

Other advantages others have pointed out:

I'm definitely coming at this from the perspective of a veteran RT user, but I don't know if we should uproot our entire issues system for a handful of advantages. I have no love for RT, but to me, GH issues just feels like a minimalistic issue system for projects to use until they get off the ground - until they need to migrate to a "serious" issue tracker. The recent "Open Letter to GitHub" seems to confirm that.

zaucker commented 8 years ago

On Thu, 4 Feb 2016, Jake Russo wrote:

@zoffixznet maybe the default should be the advanced form. Otherwise, no one will ever use it. "Path of least clicks" and all. The language drop down has no other available option, I could remove it but it's just there for future-proofing (and I think it's healthy to remind users of the distinction between language and compiler).

I think the distinction between language and compiler is better left to the developers (or queue maintainers) to decide. I think I finally got it by now, but especially new users will not always be able to tell if the want to report a language issue or compiler bug.

Also in regards to a simple form, perhaps what I could do is include a category dropdown and it would subsequently show different fields for each. For documentation it would just show "Is there a typo or missing information?", for articles it'd show "Is there an out-of-date article/blog that you're aware of?", and so on (some other categories could be "repl", "module usage", "module creation", "syntax error" "unexpected result", "crashes", "perl6.org", etc.). This could even be plugged into the subject line as an added way to filter bugs.

Oh my god ...

If I want to report a typo, I'd rather just have a text field to enter

s/string with bug/string without bug

I could even put that into the subject line of my eMail.

I certainly don't want to find my way through a myriade of selections in a drop down. And even less so, have initially selected the wrong one, entered my info into some field, and then correct my mistake with the entry field disappearing and other (empty ones) appearing.

I am all for submitting useful bug reports, but I'd suggest to rather err on the side of getting too little information (without asking back) than not getting any report because of asking to much.

Cheers, Fritz

Oetiker+Partner AG tel: +41 62 775 9903 (direct) Fritz Zaucker +41 62 775 9900 (switch board) Aarweg 15 +41 79 675 0630 (mobile) CH-4600 Olten fax: +41 62 775 9905 Schweiz web: www.oetiker.ch

uvtc commented 8 years ago

I certainly don't want to find my way through a myriade of selections in a drop down.

Right. I think what @MadcapJake is saying is that he can javascript it up so the form starts simple (just a drop-down "what type of problem are you reporting"), and then the form only gets more detailed if the user goes in that direction. For example, if the user says it's a doc bug, then they just type in the box, prove they're not a bot, and click submit. If the user says it's a something more technical, then the form can expand to ask which implementation, version, etc.

As for explanation of the form's fields, I've usually found hover tooltips to be excellent. For example, the tooltip over a "compiler version" field a might contain something like "Run perl6 --version and look for ...".

zoffixznet commented 8 years ago

can javascript it up so the form starts simple [...] then the form only gets more detailed

:+1: on a dynamic form that asks extra questions, providing the user can just short-circuit it and submit the form if they "had enough"

MadcapJake commented 8 years ago

@uvtc right! and that's a great idea for the tooltips!

I think I've come up with a plan that should alleviate some of the fears of complexity.

Instead of changing the html of the fields, the issue category would just change the placeholder text. This way the user can just skip the category and get a blank form and do their thang, but if they want to get some direction on how to fill it out, then selecting a category will add some general questions into the body to be filled out (or removed).

AlexDaniel commented 8 years ago

What @zoffixznet posted is perfect. The form should not force you into some strict structure, but it should give you some good ideas (just like it does on the image, great!). Yes, yes and yes! Just think about it, if that thing submits a bug to RT just like we do now, then we're not loosing anything but at the same time it is much, MUCH more easier to submit bugs (I don't even have to type that this bug should go to rakudobug@perl.org!). No javascript bullshit required, this is just awesome.

That being said, it should also link to other places to submit bug reports. E.g. documentation bugs should go to https://github.com/perl6/doc/issues/new (notice how it points directly to the submit form).

Also, it is missing a Title field.

@moritz

What do you mean by “handle them well”? What's the problem? And how much should we care about developers looking through these bug reports?

What's the use of bug reports if developers don't look through them, find the most relevant, and fix them?

I'm not saying that bug reports should go down the sink. I'm saying that it should be easier for users to submit bugs, but it's OK if it will be a bit harder for developers to view these bugs. I think that developers will do it anyway, but most users will just say “screw it” if it's too tedious.

@hoelzro

Email notifications out of the box.

Doesn't RT do this too?

Uhm, not really. Often I find activity on my bug reports that I haven't seen in my email. Non-comment changes are not emailed for sure, but I think that sometimes comments are left unnoticed too. Also, in the past I emailed comments to the wrong address (there are different email addresses!) so every time I do something on RT I just go and hit F5 until it actually appears, otherwise I have no idea if it actually succeeded and if everything is correct.

Unless someone can guide me through getting my bitcard account to work I don't think that I'll ever feel at home when using RT. Too much pain. Sure enough, a pretty form on perl6.org is not going to solve most of these problems (so I'm still voting to switch to something else), but at least it may hide some of them.

Now comes a good question: if we're going to have a form on perl6.org, is it possible to get a link to the bug report once it is submitted?

zaucker commented 8 years ago

On Thu, 4 Feb 2016, Aleks-Daniel Jakimenko-Aleksejev wrote:

Email notifications out of the box.

Doesn't RT do this too?

Uhm, not really. Often I find activity on my bug reports that I haven't seen in my email. Non-comment changes are not emailed for sure, but I think that sometimes comments are left unnoticed too.

I think this is just a matter of configuration

Oetiker+Partner AG tel: +41 62 775 9903 (direct) Fritz Zaucker +41 62 775 9900 (switch board) Aarweg 15 +41 79 675 0630 (mobile) CH-4600 Olten fax: +41 62 775 9905 Schweiz web: www.oetiker.ch

AlexDaniel commented 8 years ago

I think this is just a matter of configuration

Configuration of what? I can't get my bitcard account to work, so I cannot fix that myself.

zaucker commented 8 years ago

On Thu, 4 Feb 2016, Aleks-Daniel Jakimenko-Aleksejev wrote:

I think this is just a matter of configuration

Configuration of what? I can't get my bitcard account to work, so I cannot fix that myself.

Of what is sent out by eMail. But I don't know out of my head what and where. I just know that the RT instance we run here at our company sends me (more than) enough eMails. But we might just have a simpler use case, I didn't really look into the Perl RT setup(s).

Oetiker+Partner AG tel: +41 62 775 9903 (direct) Fritz Zaucker +41 62 775 9900 (switch board) Aarweg 15 +41 79 675 0630 (mobile) CH-4600 Olten fax: +41 62 775 9905 Schweiz web: www.oetiker.ch

tbrowder commented 8 years ago

On Wed, Feb 3, 2016 at 8:04 AM, Tom Browder tom.browder@gmail.com wrote:

On Wed, Feb 3, 2016 at 7:56 AM, Aleks-Daniel Jakimenko-Aleksejev notifications@github.com wrote:

but I haven't yet seen a github project with 800+ issues that was able to handle them well.

What do you mean by “handle them well”? What's the problem? And how much should we care about developers looking through these bug reports? ...

I say "amen" to most all Aleks says.

And I just looked, Bugzilla has a plugin for Git integration--might be worth looking into.

I would be willing to donate exclusive use of one of my leased server for a Perl (or just Perl 6) Bugzilla installation:

Intel Core 2 Quad - Pre-built Server - dedi3 Primary Hard Drive: 500GB Hard Drive (Default) Memory: 4GB DDR2 RAM (Default) IP Allocation: 1 Usable IPv4 (Default) Bandwidth: 100Mbit 95th Percentile Operating System: Debian 7 64-bit Control Panel: No Control Panel (Default) Server Management: None (Default)

It's not been touched yet.

-Tom

moltar commented 8 years ago

If we are going to go with self hosted form, then we can do the following:

Page 1: Name, Email, Textarea (bug report)

Once user submits that, the ticket is already saved, but on page 2, user has an option to add more info. There we can ask additional questions about versions and all that.

Worst case, we have the info already from page 1 and we can talk to the user to find out more, if necessary.

If we look at a bug report as a form of conversion, then it should ask the bare minimum of questions. Arguable it doesn't even need to ask for user's name. Just an email and description of the problem.

tbrowder commented 8 years ago

On Thu, Feb 4, 2016 at 10:26 AM, Aleks-Daniel Jakimenko-Aleksejev notifications@github.com wrote:

I think this is just a matter of configuration

Configuration of what? I can't get my bitcard account to work, so I cannot fix that myself.

Aleks, I finally got a response to my plea for rt help at perlbug-admin@perl.org and I can get a better display after logging in. I'm sure, if you can't get help with the bitcard people, you might be able to get help from the Perl RT people.

In spite of the better view on RT now, I'll put my money would on making a plan to move to Bugzilla.

Cheers to all!

-Tom

autarch commented 8 years ago

I'm not sure moving from one fairly hard to use piece of software (RT) to another (Bugzilla) would help much. Both of these tools are great for organizations that need a lot of complex logic around bug reporting. These tools provide a lot of power for people who spend a lot of time using them.

However, the most important use case here is "casual/first time Perl 6 user finds a bug and wants to report it". From that standpoint, GitHub is a huge win. A custom form on the perl6.org site is better than what we have now. Bugzilla is really not any better than RT for that use case.

tbrowder commented 8 years ago

On Sun, Feb 7, 2016 at 12:47 PM, Dave Rolsky notifications@github.com wrote:

I'm not sure moving from one fairly hard to use piece of software (RT) to another (Bugzilla) would help much. Both of these tools are great for ... However, the most important use case here is "casual/first time Perl 6 user finds a bug and wants to report it". From that standpoint, GitHub is a huge win. A custom form on the perl6.org site is better than what we have now. Bugzilla is really not any better than RT for that use case.

Dave, I do agree with you. What I failed to remember was the awkwardness of my initial Bugzilla uses (and some aspects still "bug" me).

Best,

-Tom

AlexDaniel commented 8 years ago

Aleks, I finally got a response to my plea for rt help at perlbug-admin@perl.org and I can get a better display after logging in.

Yeah, I also wrote an email to the same address and they fixed my account too (it took two days). Perhaps this should be mentioned somewhere.

AlexDaniel commented 8 years ago

Somewhat relevant: https://github.com/blog/2111-issue-and-pull-request-templates

MadcapJake commented 8 years ago

Ok, my issue submitter is making progress, you can check it out here. I still need input on issue templates, please submit a PR (place them in templates), issue, or message me on IRC/slack/email with your suggestions. I'm thinking markdown at first and then I'll translate into html when sending the email.

softmoth commented 6 years ago

I think this issue may be largely moot now given perl6/perl6.org#96 (Rakudo bugs are now using GitHub issues).

Also, see duplicate of this issue at perl6/perl6.org#61.

nxadm commented 6 years ago

This is indeed an old thread that can be closed now that Github Issues are also used.