Things that scare off potential contributors

[ldorigo]

Hi there,

I have been wanting for a while now to get involved in TW's development. I encountered quite a lot of barriers that are probably common for newcomers, and which you may not be fully aware of as long-term contributors (through habit and a deep knowledge of the code base). I thought I'd list some here, so that steps can maybe be taken to improve the experience for newcomers.

1. The issues section on GitHub is one big mess. There are over 600 open issues, which is a lot, but the real problem is they are totally unstructured. When getting into a new project, I like to pick an issue that looks easy to solve, and then dive in. Here I don't know:

• Which issues are already being worked on by someone?
• How easy/hard is an issue?
• Is it something that the maintainers want to solve/implement or not?
• Is it a bug, a feature request, a comment, ...

This could all be largely improved by using specific labels and assigning issues to people.

2. There is no (or very little) information on coding style, language level used, etc. I noticed that the JavaScript being used is quite ancient by web standards - no arrow functions, constructs such as !! (which I had never seen before), and so on. This is fine - I guess the fact that TW is so stable and portable is largely due to this. But it's very confusing for millennials like me who started out with ES6 ;-) Without proper guidelines I wouldn't know what I can and cannot use.

3. The architecture of the whole app is rather confusing. I understand that you built everything so that it could be executed in any environment, be it node or the browser. While I find this a bit strange (why not bundle common code together and have specific modules for the different environments?) I respect the design decision - but it would require a bit more inline documentation on what is geared towards what. In the kernel, which functions will be called in the browser and which will be used in node? It's possible to figure it out by using a step-by-step debugger or by reading the code carefully, which is what I did, but it would definitely be a more pleasant experience with some extra comments.

4. There is no live chat where I can interact with other developers. I already complained about this on the google group. Getting into a new code base is hard. If I have to wait 3-4 hours for each question I have, the learning process is extremely slow, especially if it's a "blocking" question: I have to ask it, stop what I was doing (or spend hours trying to figure it out on my own), then maybe someone will answer and I will get a bit further until I have a follow up question and I have to wait for hours again. With a group chat, I could just drop in, ask a few question, and ask for specific details that still confuse me afterwards.

Those are the main obstacles I encountered, I will edit if I encounter more.

Thanks for reading this and I hope being able to start pushing soon :-)

[Jermolene]

Hi @ldorigo thanks for the comments, very useful. And I'm interested in what others have to say too.

Some of the problem is that I (@Jermolene) am still a bottleneck for too many activities. Accordingly, we plan to migrate to an organization repository so that more people can help with the bug triage etc. (See #3114). Once we've done that then we can address CI with #3115 and that will enable us to offer a much better workflow for docs contributions etc.

[TiddlyTweeter]

@ldorigo, I am not a programmer but occasionally manage to raise a relevant issue.

I agree that having over 600 open issues is both disconcerting and a turn-off from wanting to add more.

From my point of view, as an ordinary TW user, GitHub requires more of me than usual--to be clear in what I write. That is good! But its a disincentive to make the effort if I feel its never going to be read or go anywhere.

[pmario]

@ldorigo .. Good points!

just as a reverence - related to:

• 682

  • Be explicit about browser support targets #2278

---

I'm adding related links here, to avoid spamming the thread.

[ldorigo]

@pmario Besides the related content, it illustrates my point precisely: That issue was opened 4 years ago, is still open, and I would never have seen it other than by you linking to it.

@Jermolene I am happy to hear that things are moving. Why is there a need to migrate it to another repo, isn't it possible to add contributors (as in, give "more people [the power to] help with the bug triage etc.") on the current one? CI is a great idea, as of now it's unclear how, when and where to run tests because of lack of documentation (you must be tired to hear this by now :-)).

And while I mentioned the issues section, the PR section on GitHub is just as bad - it can be pretty demotivating for a potential contributor to see there are 100+ unprocessed pull requests coming before his.

[Jermolene]

Why is there a need to migrate it to another repo, isn't it possible to add contributors (as in, give "more people [the power to] help with the bug triage etc.") on the current one?

Personal GitHub repos don't offer fine-grained delegation of permissions. Organizations like https://github.com/TiddlyWiki can setup named teams to handle different processes.

CI is a great idea, as of now it's unclear how, when and where to run tests because of lack of documentation (you must be tired to hear this by now :-)).

Have you found the /dev wiki? https://tiddlywiki.com/dev#TestingMechanism

[ldorigo]

Oh - I had made it to the Wiki, but didn't see that specific tiddler. Good to know.

Actually, it seems like that tiddler is not referenced anywhere (adding <<list-links "[all[current]backlinks[]]">> gives no result, and nothing seems to be pointing to the tag Mechanisms). That's probably why I didn't find it before.

[diego898]

@ldorigo thank you for this! I've long thought that if we as a community could just organize this enough, it would allow for better first-time contributor experiences. A great goal would be to allow TW to serve as a term-long project for an undergrad software engineering team. Many uni (at least in US) have undergrad SE teams pick an open source project and either contribute a major new feature or implement planned ones, etc. Right now, TW is in no place to enable that.

[ldorigo]

@diego898 Yes! I know Belgian universities also sometimes allow contributing to open source projects as an (unpaid) internship, which is obligatory in the last year. Would be a great opportunity to let the project grow - but again, it wouldn't be possible at the moment.

[TiddlyTweeter]

I want to add a point I think is important. At least from a contributor's point of view.

When I comment on an issue on Github that issue still remains in the same place in the listing when you view, online, the Issues.

The participants will get an email, but a public viewer would have no idea whether its a hot topic with lots of interest because it does not move up when a new comment is made.

The longer the Issue list gets the harder it gets to see what is of interest and topical. And age of original Issue posting kinda buries it. Even if its just as valid today as it was two years ago.

[Evolena]

That's the GitHub interface; issues are listed by their identifier. You can:

• sort the issues by "recently modified"
• go to the "Insight" tab for view on last month/week/3 days/24 hours activity

[TiddlyTweeter]

Thanks @Evolena. I'm a naive user so that info is really helpful. My question is this: how many users here know that?

[sukima]

I notice a lot of complaining about the GitHub interface. Are these concerns that should be brought to to GitHub, Inc.? Does GitLab also suffer the same experience? I'm curious to understand how the use of a very popular and well understood tool such as GitHub is a detrimental choice in the context of a single software project (TiddlyWiki). More specifically is there something completely unique to TiddlyWiki that no other software project hosted on a software platform like GitHub, GitLab, SourceForge, BitBucket, etc. suffers from the same difficulties?

What is it about TiddlyWiki and GitHub that are incompatible when several other Wiki projects are successful in using such tools.

[Jermolene]

@TiddlyTweeter @Evolena you may be interested in Octobox, an alternative front end for viewing GitHub tickets: https://octobox.io/

I'm curious to understand how the use of a very popular and well understood tool such as GitHub is a detrimental choice in the context of a single software project (TiddlyWiki). More specifically is there something completely unique to TiddlyWiki that no other software project hosted on a software platform like GitHub, GitLab, SourceForge, BitBucket, etc. suffers from the same difficulties?

I think the apparent antipathy towards GitHub in the TiddlyWiki community is because only a small proportion of TiddlyWiki users have any familiarity with it. The difference between TiddlyWiki and the majority of open source projects is that our audience is not just other software developers: it's an end-user application and not a library. The barriers to using TiddlyWiki are much lower than those for adopting a conventional app, so the community is very broad, and broadly non-technical.

[TiddlyTweeter]

@Jermolene

I think the apparent antipathy towards GitHub in the TiddlyWiki community is because only a small proportion of TiddlyWiki users have any familiarity with it.

Absolutely right. I have NO antipathy towards it. I simply don't know quite how to use it well.

[TiddlyTweeter]

@Jermolene the fact GitHub is a challenge for non-technical users who DO arrive at good (albeit messy) technical observations is really interesting. Innovation can get driven a different way. Its quite hard to differentiate that difference, but I think worthwhile.

What I would like is an "Intro to TW GitHub By Example."

I got as far as basic confidence on Issues so far. A Pull Request is still a foreign land.

[Marxsal]

The phrasing of this post suggests that users are intimidated by the GitHub process. For documentation, that's probably true. But most contributors are developers and everyone is expected to know it. If anything, it appears that contributions pour in faster than Jeremy can keep up.

During the past month there were 23 merges and two releases. There were 95 issues closed. It doesn't seem like sleepy-town to me. Most of the other projects I see (other than funded projects like mozilla) are nearly dead by comparison.

My concern is that simple documentation PR's are ignored so long that they become outdated. That sort of thing douses enthusiasm. Documentation doesn't break anything. It should be quickly approved so that someone else can work on it if they have an addition to make.

[Marxsal]

@TiddlyTweeter What about @pmario 's github videos?

https://www.youtube.com/watch?v=L4zTkMYcri8&list=PLuiC_HFhI4OyfLdEqdK4lkFpFJLI4NrKo

[TiddlyTweeter]

@Marxsal that is an EXCELLENT example of the issue facing folk like me.

Who knows those videos exist?

How would they be in the right place at the right time? They would help at start.

I have no recollection whatsoever of having come across them ever.

Whilst knowing of those now helps ME, does it help anyone else? This is NOT a criticism. Its not your job to be an advertiser. But helpful entry needs be there when you need it. Where are they such that coming to GitHub that help is available?

[pmario]

...What about @pmario 's github videos?

tiddler: https://tiddlywiki.com/#Improving%20TiddlyWiki%20Documentation which also links to the videos.

The basic workflow is still valid, but GitHub did change the UI a bit. ...

I just watched the first 2 videos. For me, it's fun, to see the differences ... For new users it may be confusing. I'll have to update them.

[pmario]

How would they be in the right place at the right time? They would help at start.

I have no recollection whatsoever of having come across them ever.

This shows me, that we definitely need to make the link more prominent.

• The first link "Find out how" ... will open the tiddler I linked in the other post.

[ldorigo]

I feel like two things are being mixed here, on one hand the difficulties for contributors (as in, developers) to get involved, and on the other hand the difficulties for end users to be heard/participate in the improvement of TiddlyWiki (which is not what I was talking about in my issue, but is definitely important as well).

While mangling the two things (general user feedback and technical issue reporting) can work for some projects like https://github.com/Microsoft/vscode where the end users are also technical people and thus have at least some familiarity with github and issues, I don't think it's optimal for a more general app like TiddlyWiki.

An approach like the one used by Discord (who is reputed for how they process/follow user feedback) might be much better: https://feedback.discordapp.com/forums/326712-discord-dream-land . Here end users can freely submit/vote/comment/discuss feature requests and bugs, and the technical team can then translate them to more detailed issues on github.

[TiddlyTweeter]

@ldorigo I think you are right that its not such a clean discussion.

FYI, I am NOT a developer, and never want to be one.

I only use GitHub to try get other people (you :-) to solve problems for me so I can write & make film :-).

HOWEVER, I think @Jermolene pointed accurately that in TW actual usage there is NO ultimate separation between enthusiasts and developers. Its seems to me a LOT of TW users on the Google Group are "budding developers".

Frankly I been amazed at people's interest there in how to STRUCTURE TW. Rather than how to FILL content.

I think I said all I am capable of on this thread :-)

Buongiorno, J.

[twMat]

If we get a federated TW network going, then we could build our own organic contribution system. I even don't think it would be very difficult but it would, of course, take some time for it to mature. The thing is, people could tweak their own interfaces for it.

[sukima]

The phrasing of this post suggests that users are intimidated by the GitHub process. For documentation, that's probably true. But most contributors are developers and everyone is expected to know it.

I'm curious how other end-user applications accomplish the same thing. I've never seen an open source project that accepted contributions to not use a version control system (like Git) or have an issue tracker or project management tools like GH. If it is really GH that is the issue there are many alternatives. We could look into JIRA and BitBucket. Or SourceForge. or GitLab. I am just not understanding how this differs. Even other static site generators use tools very similar if not GH.

What I'm getting at is instead of just saying GH sucks or is intimidating why not articulate the actual challenges. I know that GH is used in government for non-technical people. It is used bny writers who never seen a line of code ever. Really these problems we face need to go to GitHub themselves so they can fix the problems for everyone.

I also fail to see how a project written in source code could avoid the use of source code. This isn't like we are running Wikipedia or anything were maintaining a build tool and a compiled application.

Apologies if this reads poorly my tone is likely lost in conversion. I am genuinely interested at how user experience seems to drop on just this project compared to countless others in the same genre.

[twMat]

I am genuinely interested at how user experience seems to drop on just this project compared to countless others in the same genre.

@sukima - I think Jeremys observation explains this:

I think the apparent antipathy towards GitHub in the TiddlyWiki community is because only a small proportion of TiddlyWiki users have any familiarity with it. The difference between TiddlyWiki and the majority of open source projects is that our audience is not just other software developers: it's an end-user application and not a library. The barriers to using TiddlyWiki are much lower than those for adopting a conventional app, so the community is very broad, and broadly non-technical.

The following is a "side-reply" to @sukima to clarify a non-developers perspective. @ldorigo 's OP was from a more advanced perspective so this likely does not represent his issues. So no need for everyone to read this unless you wonder how mere mortals feel:

I have honestly tried to understand the GitHub workflow several times. I can fairly easily contribute documentation PRs but even that is somewhat confusing ("Post it again? But I just posted it! Even retype the heading?"). For code, maybe there is some conceptual linchpin that I'm missing so that nothing else seems to make sense.

For one thing, there is an underlying frustration that if I were to post a "suggestion" in real life, I would just, say, e-mail it. That's it. For code, it must be ensured that it works in its environment... which my assumed PR does because I built it in TW. But to make a PR I must enter some totally different environment where I must even have my own copy of the whole project.

Just now, for this discussion, I decided to quickly check how far I could get before hitting weirdness. I've kind of promised to make a PR for an issue I posted some days ago so I know I'll be wrestling with this in some time anyway. As noted, I already have a gh account, I have made doc PRs and I believe I actually have made some real PR years ago - so I'm already way ahead of most TW community members. So, anyway, now (just a minute ago), I clicked "create new PR"... and - bam! - immediate weirdness:

Imagine I have made some improvement in my local TW. I have the wikitext code. WTF is this "compare changes across branches, commits, tags, and more below." And "If I need to, I can compare across forks". Dafuq? I want to make a PR, that's it. The only thing that seems relevant there for me is the green button "Create pull request" - I, of course, just clicked such a button but OK, appearently I have to click it again... "click" - blocked!!! (In-your-face!)

OK... I'll play along. I'm going to: "base: master <--- compare: master" (huh?). Well, weirdness strikes again: There's some dropdown with.. I have no idea what that is.

I don't get paid for this, I was going to share out of generosity... but let's face it, even the lingo is too advanced for me so how the hey am I going to be able to use github?

I give up.

Now, this is probably a very realistic scenario. As noted, I think I have passed this nonsense previously and successfully posted a code PR (I'll have to dig up this). I've also previously watched instruction videos but atm I don't remember those, likely because of all the arbitrary magic. I probably will come past this when I hopefully make a PR in a few days/weeks but I'm pretty sure I'll have to spend an unreasonable amount of time on this and I'm pretty sure I'll forget it again because I don't do this as part of my daily activity.

In addition... it is not sure the PR will be dealt with by an (understandably busy) Jeremy or other admins and it is, of course, not sure the PR is good enough. For a non-coder it is even less likely to be good enough.

Maybe this clarifies some of the confusion that innocent tiddlywikians get hit by. It is just too much.

[ldorigo]

I think @Jermolene pointed accurately that in TW actual usage there is NO ultimate separation between enthusiasts and developers.

That's probably the reason why adoption of TW is so slow: for now it mainly appeals to, if not developers, at least technically inclined people.

Thinking more about this, I think there are actually three things being discussed here:

1. The difficulty for developers to get involved in the development of TW
2. The difficulty for end users to report bugs/ask for features
3. The difficulties for advanced but non-technical users to contribute to the documentation, translations etc.

I think those three things should be handled separately as they are entirely different problems.

I will make two new issues to specifically address those other two problems, so that this one can be reserved for discussion on how to improve the experience for developers.

[TiddlyTweeter]

@sukima

What I'm getting at is instead of just saying GH sucks or is intimidating why not articulate the actual challenges.

I agree. Detail of a few cases highlight it. @twMat did it above.

Without in any way wanting to take away from this thread I did post a note about it on the main Google Group as I think its worth seeking responses to how GitHub looks from there. I thought @telmiger's reply to it was very interesting for its detailed articulation of his problems.

https://groups.google.com/d/msg/tiddlywiki/m0RI5wqUDag/8PhqBG58AgAJ

He needed your help, as you know, to try move a PR on for his SVG cats, and still the PR is hanging.

I find his post enlightening. I mean, if people like @telmiger and @twMat are struggling, someone like me has no chance whatsoever getting beyond raising an issue.

Not that I need to! But its just good to be a bit clearer. I AM getting a bit clearer--because of this discussion.

[TiddlyTweeter]

@ldorigo

The difficulty for developers to get involved in the development of TW
The difficulty for end users to report bugs/ask for features
The difficulties for advanced but non-technical users to contribute to the documentation, translations etc.

Looks good to me. I mean, it makes great sense.

One reflection I have is simply ... is there a way not to have such a backlog of both Issues & PRs? I think as is it adds to "the fog" about TW to have large numbers of things that are either in process or waiting.

And THEN, because I'm terribly anthropological, in the back of my mind the social scientist says: "There are just not enough developers to assist the PR end."

The question of "staffing" is so muted in these conversations I find it bizarre.

IS it really about more efficient systems, or just more numbers of skilled folk at the end-point?

[ldorigo]

is there a way not to have such a backlog of both Issues & PRs? I think as is it adds to "the fog" about TW to have large numbers of things that are either in process or waiting.

Well, that's what this discussion is all about :-) See #3319 for a better way to collect user feedback, and about the amount of backlog....@pmario has started cleaning up the issues a bit (judging by the amount of notifications I'm receiving), and once we implement a good user feedback system, we can "migrate" a lot the existent issues that wouldn't belong here anymore (and thus clean up the issue section on Github)

IS it really about more efficient systems, or just more numbers of skilled folk at the end-point?

Hopefully, improving user and developer experience will also increase the amount of people wanting to get implicated :-)

[TiddlyTweeter]

@ldorigo

@pmario has started cleaning up the issues a bit (judging by the amount of notifications I'm receiving

I recently commented on Google Groups that posts are only made by 3 people there and 30 of them are by PMario :-)

More seriously, are there really enough people competent at his level active to sustain this? Its only a few months ago that @tobibeer went over the Issues already.

That is what I am pointing to: the match between activity needed and the actual available resource base. If they don't match it won't work. ONE person? Is it enough?

Just probing the issue Josiah

[pmario]

As I wrote at the other issue, I did setup an account at GitLab, that could be used by tiddlywiki to manage the whole community. ... At the moment I'm creating a proof of concept, and test the possibilitys, up to CI/CD.

I'm not finished yet, but I hope to start a new thread a the list on sunday. Including some simple videos.

[pmario]

@ldorigo ... If you PM me, I can send you an invite. ... Do you use Discord?

[ldorigo]

"manage the whole community" as in, leave GitHub completely? Or are you only talking about the user feedback?

And sure, my discord is Gloomy#3417 :)

[ldorigo]

But let's rather use Gitter ? I'm not so often on Discord. I sent you a PM !

[pmario]

"manage the whole community" as in, leave GitHub completely?

If the community likes, what I have to show. ... may be!

I'm searching for simple to use workflows, for years now. GitHub was the open source "go-to" platform and it made sense to use it. Now that changed.

I'm watching GitLab since Jan. 2013. ... They made great progress. Especially the group (namespace-feature), the integrated CI/CD process, Issue voting and chat (Gitter is owned by GitLab)

I think it is time to have a very close look!

All of this functionality is really hard to setup using the free version of GitHub. I tried for 2 years and I'm constantly struggling. ... and I'm not the only one.

The last 4 days, testing GitLab with a real project in mind (TiddlyWiki.org) I think, I did make good progress, with a system that is fun to work with.

The "end-user" side of things need improvement, but I hope I can show you something more on sunday.

There will be a lot of work left for the community. .. But I think it would be possible, that the community can actually do it, independent from -- but including "the core"

[pmario]

Hi folks, you may be interested in a new Video series I did upload today. ... It' about improving Community Communication.

The whole thing is an experiement, so we can heavily test it, without damaging anything.

It would be nice, if you guyes would request access to the different groups, that are interesting for you If you are interested in everything .... But just watch the videos and have fun!

Starting from video 6/7 it gets more developer centric.

https://www.youtube.com/playlist?list=PLuiC_HFhI4OyUiDGqvzB64mTQTURABjlW

[dubiouscript]

The participants will get an email, but a public viewer would have no idea whether its a hot topic with lots of interest because it does not move up when a new comment is made

in desktop browser it is possible to sort the view by "Recently updated"

https://github.com/Jermolene/TiddlyWiki5/issues?utf8=%E2%9C%93&q=is%3Aissue++sort%3Aupdated-desc+

but apparently this is not possible on droid