Organizing documentation

[jgbarah]

This issue was triggered by the insightful comments by @filmaj in chaoss/grimoirelab-sirmordred#167

To summarize, now we have the following documentation:

• Some README.md files in source code repos, including some in subdirectories of them.
• Some automatically built documentation from source code comments. In some cases (well, in one case) it is available in readthedocs: Perceval documentation
• The Grimoirelab Tutorial, produced automatically from md files in docs dir in this repo.
• Some interesting information here and there in some issues.

I agree with @filmaj that we should better organize it. In principle, the idea (I think) was:

• README files for stuff like how to configure and run, how to contribute (maybe in separate CONTRIBUTING files), how to run tests and do other development-related stuff, and links to further documentation. In short, this would be information mainly for developers, and for letting users know the very basics, and where to find more. I would say this would be a kind of a reference manual for users, and in part (except for everything in the next item) for developers.
• Automatically generated documentation from comments in the source code. This should be for the rest of developer-related information, including definition of the APIs, for example.
• Tutorial. This was intended not as a reference manual, but as a tutorial of how to get stuff done. This should be the second thing a user would consult, after the README files that would explain the very basics, and after the detailed user guides that would be also a part of those README.

What do you (anyone reading this issue) think about this organization? Is still a sensible schema? (I'm not saying it is implemented this way now, just checking whether you find it convenient to set it as a goal).

[filmaj]

All of that sounds great! I would support that.

The tutorial would arguably be one of the first things a new (potential) user would read. A reasonable new-user journey would be: land on the grimoirelab page and then very quickly click on "Docs" to land on the tutorial. From that perspective, perhaps it could use a review to make it gentler and aimed at users with little understanding of the architecture.

The READMEs in each repo could therefore be a bit more developer-centric, and more detailed with respect to the particular project. I think reviewing each repo's README (some have multiple, embedded in subdirectories!) from that angle would be helpful as well. I fully support adding a CONTRIBUTING.md file. Worth noting that how GitHub displays this file is different from READMEs: GitHub highlights this file when people file issues or are creating pull requests (and calls it out even more for a person filing issues/PRs for the first time). With that in mind, I think a CONTRIBUTING.md could be written up to include instructions directed at the contributor: what do they have to do or consider before filing an issue or sending a pull request? Do they need to sign a CLA? Hint to search the issue tracker first for an existing issue? Does the pull request have a related issue filed? Worth considering that each repo can have its own README / CONTRIBUTING and even issue and pull request templates, and each one could contain information that is more relevant to that particular repo.

Finally, the automatically-generated API references, I think, would be very helpful, especially if those are generated from comments within the code, or code itself, directly.

I am happy to help with any of these efforts! I am, however, still learning the project, so could use some direction :)

[MalloZup]

i will create some notes for setting up grimoireLab https://github.com/MalloZup/luce-on-grimoirelab

@jgbarah If you find any utility feel free to take/copy them, atm there will be only notes for me to remember how the setup and mechanism etc works. :sunflower: I just post them here so you can have a feedback if you are interested otherwise for me are usefull :dango:

[MalloZup]

small feedback https://github.com/chaoss/grimoirelab/issues/144 on elastic-search.

---

Globally and in answer with this issue, i have the same opinion as @jgbarah but only one concern as i am a newbie now the grimoirelab experience :smile:

to me seems more comfortable to have a webpage (tutorial or official webpage) that have a global installation. ( just to really minimalist and easy like create your dashboard with 10 commands)

pip install grimoirelab is really nice shot to install quite everything. ( or also docker is ok)

Having a global webpage ( or central readme) that say the basic about grimoireab to get the dashboard is more helpfull and less painfull to go through every github repository ( which should happen later when user want more advanced infos). This can help afaik.

In github repository to me should be more documented the tool as single detailed one , so having the scope of the tool + developers infos for contributing + api

[Polaris000]

As I have previously mentioned on the grimoirelab mailing list (and opened a pull request), the grimoirelab tutorial website also lacks a favicon image which should appear in a browser's tab when visiting the website.

[MonikaEve]

Hi, I found Bitergia through LinkedIn, as I'm currently on the job hunt. I thought I could break the ice, by contributing to your open source project. Do you still need support with this issue?

[valeriocos]

Hi @MonikaEve, thank you for your interest in GrimoireLab, and indeed you are more than welcome to contribute to it. If you want to get a glimpse of GrimoireLab, you can start with the tutorial. Feel free to have a look also to the GSoC ideas for this year [1] or to the GrimoireLab components [2]. If you enjoying reading, we have also a nice blog and some research papers [3] waiting for you. WRT to job hunt, probably @jgbarah , @sduenas and @jsmanrique can help you with this.

[1] GSoC ideas

• Implementing CHAOSS Metrics in Kibana Dashboards
• Implementing CHAOSS metrics with Perceval
• Support of Source Code Related Metrics

[2] GrimoireLab components

• Perceval - data collector
• Graal - source code data collector
• KingArthur - scheduler
• SortingHat - identity manager
• ELK - data processor and enricher
• SirMordred - orchestrator
• Sigils - visualizations
• Kibiter - visualizations mananger
• Anything that starts with https://github.com/chaoss/grimoirelab-*

[3] Research papers

• SortingHat: Wizardry on Software Project Members
• Graal: The Quest for Source Code Knowledge
• Perceval: Software Project Data at Your Will

[jsmanrique]

Hi @MonikaEve! Thank you very much for your interest. We always need help with this ticket. Another project that might be interesting is GrimoireLab Hatstall, the web app that connects and manages SortingHat. Since we are adding creating a GraphQL API for SortingHat, there is a work in progress to create a React based UI for Hatstall ... Sadly, there is not much doc about that (CC @sduenas @dlumbrer )

[dlumbrer]

Hi @MonikaEve! Thank you very much for your interest. We always need help with this ticket. Another project that might be interesting is GrimoireLab Hatstall, the web app that connects and manages SortingHat. Since we are adding creating a GraphQL API for SortingHat, there is a work in progress to create a React based UI for Hatstall ... Sadly, there is not much doc about that (CC @sduenas @dlumbrer )

Hi @MonikaEve!!, @jsmanrique is right. As he said, we are working on the development of a new React based web app, this app will have the same aim than the GrimoireLab Hatstall based on Django: provide a user interface in order to manage SortingHat, but now, using the new version of SorthingHat that is has a GraphQL API.

This is the repository where the project is being developed. There is no doc yet, the project is still "new", but as I said before, the aim of this new web app is the same that the old Django GrimoireLab Hatstall has, but using the new GraphQL API of Sortginhat, so we are including all the functionality that the old app has as the new GraphQL API is having more "queries/mutations" (the way that a GraphQL API is). Therefore, we need more improvements and development in this app, in terms of general layout and in terms of complete the functionality. We are using React because is a powerful library in order to build user interfaces and we are used to working with Kibana, also based (most of it) on React.js.

[MonikaEve]

Hi All,

Thanks a lot for your replies :) I started going through the tutorial last night and I'll continue tonight and I'll definitely check out GrimoireLab Hatshall and SortingHat. The project looks really interesting, so I'm excited to get started :) I will let you know once I've read through the documentation and probably ask more questions.

On Wed, Mar 27, 2019, 11:31 David Moreno Lumbreras notifications@github.com wrote:

Hi @MonikaEve https://github.com/MonikaEve! Thank you very much for your interest. We always need help with this ticket. Another project that might be interesting is GrimoireLab Hatstall https://github.com/chaoss/grimoirelab-hatstall, the web app that connects and manages SortingHat https://github.com/chaoss/grimoirelab-sortinghat. Since we are adding creating a GraphQL API for SortingHat, there is a work in progress to create a React based UI for Hatstall ... Sadly, there is not much doc about that (CC @sduenas https://github.com/sduenas @dlumbrer https://github.com/dlumbrer )

Hi @MonikaEve https://github.com/MonikaEve!!, @jsmanrique https://github.com/jsmanrique is right. As he said, we are working on the development of a new React based web app, this app will have the same aim than the GrimoireLab Hatstall https://github.com/chaoss/grimoirelab-hatstall based on Django: provide a user interface in order to manage SortingHat https://github.com/chaoss/grimoirelab-sortinghat, but now, using the new version of SorthingHat that is has a GraphQL API.

This is the repository https://github.com/dlumbrer/Hatstall-React where the project is being developed. There is no doc yet, the project is still "new", but as I said before, the aim of this new web app is the same that the old Django GrimoireLab Hatstall https://github.com/chaoss/grimoirelab-hatstall has, but using the new GraphQL API of Sortginhat, so we are including all the functionality that the old app has as the new GraphQL API is having more "queries/mutations" (the way that a GraphQL API is). Therefore, we need more improvements and development in this app, in terms of general layout and in terms of complete the functionality. We are using React because is a powerful library in order to build user interfaces and we are used to working with Kibana, also based (most of it) on React.js.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chaoss/grimoirelab/issues/135#issuecomment-477085840, or mute the thread https://github.com/notifications/unsubscribe-auth/AYGLpt4HnfspDHUJu-j-PdgehTm7qCj_ks5va0h_gaJpZM4VaHN3 .

[MonikaEve]

Hi,

I finally went through the documentation and installed the new version of SortingHat :) I'm currently working with react and I think it would be the easiest to start with the documentation of the new SortingHat. So maybe I could add some general info (taken from the old SortingHat and tutorial) to the readme file. I could also add a "How to install" chapter. How does that sound?

Regards, Monika

On Wed, Mar 27, 2019, 12:13 Moni Eversmann moni.eversmann@gmail.com wrote:

Hi All,

Thanks a lot for your replies :) I started going through the tutorial last night and I'll continue tonight and I'll definitely check out GrimoireLab Hatshall and SortingHat. The project looks really interesting, so I'm excited to get started :) I will let you know once I've read through the documentation and probably ask more questions.

On Wed, Mar 27, 2019, 11:31 David Moreno Lumbreras < notifications@github.com> wrote:

Hi @MonikaEve https://github.com/MonikaEve! Thank you very much for your interest. We always need help with this ticket. Another project that might be interesting is GrimoireLab Hatstall https://github.com/chaoss/grimoirelab-hatstall, the web app that connects and manages SortingHat https://github.com/chaoss/grimoirelab-sortinghat. Since we are adding creating a GraphQL API for SortingHat, there is a work in progress to create a React based UI for Hatstall ... Sadly, there is not much doc about that (CC @sduenas https://github.com/sduenas @dlumbrer https://github.com/dlumbrer )

Hi @MonikaEve https://github.com/MonikaEve!!, @jsmanrique https://github.com/jsmanrique is right. As he said, we are working on the development of a new React based web app, this app will have the same aim than the GrimoireLab Hatstall https://github.com/chaoss/grimoirelab-hatstall based on Django: provide a user interface in order to manage SortingHat https://github.com/chaoss/grimoirelab-sortinghat, but now, using the new version of SorthingHat that is has a GraphQL API.

This is the repository https://github.com/dlumbrer/Hatstall-React where the project is being developed. There is no doc yet, the project is still "new", but as I said before, the aim of this new web app is the same that the old Django GrimoireLab Hatstall https://github.com/chaoss/grimoirelab-hatstall has, but using the new GraphQL API of Sortginhat, so we are including all the functionality that the old app has as the new GraphQL API is having more "queries/mutations" (the way that a GraphQL API is). Therefore, we need more improvements and development in this app, in terms of general layout and in terms of complete the functionality. We are using React because is a powerful library in order to build user interfaces and we are used to working with Kibana, also based (most of it) on React.js.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chaoss/grimoirelab/issues/135#issuecomment-477085840, or mute the thread https://github.com/notifications/unsubscribe-auth/AYGLpt4HnfspDHUJu-j-PdgehTm7qCj_ks5va0h_gaJpZM4VaHN3 .

[sduenas]

Hi @MonikaEve, first of all thanks for your help!

Although, SortingHat is one of our more stable pieces, the new version is not, so I'm not sure how helpful would be to start working there. I suggest to start with Perceval. It's the most stable one and there are many people out there using it. It would be great to have really good documentation for newbies. There's some stuff written in the grimoirelab-tutorial but documentation talking only about Perceval would be idea.

Perceval is partially well documented in terms of API but we need something better for the different kind of users the tool has.

Anyway, I can help you with both if you need to understand how things work or something is not clear. I'll happy to do it.

[MonikaEve]

Hi Valerio,

that sounds good. I just briefly read about Perceval, so I have an idea, but I didn't try to use it. So I will go through the tutorial one more time and start working with it and write down where I had problems and so on. I will then let you know what I came up with and maybe we can set up a call?

Regards, Monika

Am Do., 4. Apr. 2019 um 16:56 Uhr schrieb Santiago Dueñas < notifications@github.com>:

Hi @MonikaEve https://github.com/MonikaEve, first of all thanks for your help!

Although, SortingHat is one of our more stable pieces, the new version is not, so I'm not sure how helpful would be to start working there. I suggest to start with Perceval. It's the most stable one and there are many people out there using it. It would be great to have really good documentation for newbies. There's some stuff written in the grimoirelab-tutorial but documentation talking only about Perceval would be idea.

Perceval is partially well documented in terms of API but we need something better for the different kind of users the tool has.

Anyway, I can help you with both if you need to understand how things work or something is not clear. I'll happy to do it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chaoss/grimoirelab/issues/135#issuecomment-479957372, or mute the thread https://github.com/notifications/unsubscribe-auth/AYGLpjpefsSe21hjSPK8f5goVgrOUDdKks5vdiDCgaJpZM4VaHN3 .

[valeriocos]

Sure @MonikaEve , @sduenas and myself will be happy to set up a call, just ping us when you are ready :)

[MonikaEve]

Ok, I forgot to tell you, that I am travelling this week and I will be in Sevilla and Barcelona. So I won't have time this week, but I'll start working on it next week.

Regards, Monika

On Sun, Apr 7, 2019, 12:09 valerio notifications@github.com wrote:

Sure @MonikaEve https://github.com/MonikaEve , @sduenas https://github.com/sduenas and myself will be happy to set up a call, just ping us when you are ready :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chaoss/grimoirelab/issues/135#issuecomment-480580480, or mute the thread https://github.com/notifications/unsubscribe-auth/AYGLpvy4J5pJkzFJiHb4r4Ybg9EOXEM1ks5vedHagaJpZM4VaHN3 .

[valeriocos]

Thank you for letting us know, enjoy!

[GeorgLink]

Possibly related issue: https://github.com/chaoss/grimoirelab-tutorial/issues/93