Feature Request : Documentation Automation: Bridging talawa-api and talawa-docs repositories

[tasneemkoushar]

The current approach to generating and synchronizing documentation between the talawa-api and talawa-docs repositories faces challenges related to branch protection rules and the specification of source and destination branches.

Current Workflow: talawa-api generates markdown files from inline tsdoc comments. These files are stored in a dedicated folder within the talawa-api repository. GitHub Actions encounters difficulties transferring these files to the talawa-docs repository, particularly due to some rules.

Challenges:

1. Branch Protection Rules: The develop branch in talawa-api is safeguarded, impeding direct commits.
2. Source and Destination Branch Specification: Complications arise in specifying source and destination branches for the documentation transfer.

Proposed Solution: Trigger the workflow on a schedule or in response to push events on the develop branch of talawa-api. Automate the generation of documentation files from inline comments (Already Implemented). Commit files directly to the specified branch and folder location in the talawa-docs repository.

Implementation Steps: Explore existing workflows for innovative insights and references. Fork the talawa-api and talawa-docs repository to kickstart the documentation automation. Conduct thorough local testing to validate the efficacy of the automation.

[tasneemkoushar]

Kindly share your strategy or solution to this problem along with the request that the issue be assigned. You may ask your doubts here in the chat. Take help from @anwersayeed

[adi790uu]

hey I would like to work on this

[AdityaRaimec22]

Can I get assigned to the issue.

[Olatade]

@adi790uu @AdityaRaimec22

Kindly share your strategy or solution to this problem along with the request that the issue be assigned. You may ask your doubts here in the chat. Take help from @anwersayeed

[adi790uu]

Trigger the workflow on a schedule or in response to push events on the develop branch of talawa-api.

for this we can set up the workflow to be activated either by push events on the develop branch of talawa-api and we can schedule a recurring cron job.

[palisadoes]

cron only works on the default branch. We need this to work with any branch type and a filter for develop

[adi790uu]

correct me If I am wrong we need to trigger workflow only when there is a push or merge to the develop branch in talawa-api and after the workflows the docs are updated in automated-docs branch from there it has to be copied to talawa-docs repo? @anwersayeed

[adi790uu]

@Olatade @tasneemkoushar I have tried the whole workflow so I have a question that when we merge the code from any other branch, is workflow triggered? If not we need to implement that and somehow Copy-docs-to-talawa-docs workflow is getting success but the changes are not reflected in my forked talawa-docs repo.

[anwersayeed]

Hi @adi790uu @AdityaRaimec22 @adi790uu Yes, the workflow needs to be triggered either by a push event or on scheduled basis, but we do not necessarily need the automated-docs branch, the task is to send the generated docs to the talawa-docs repository. Hence, it would be better if we generate the documents and copy it directly to the talawa-docs location.

[anwersayeed]

@Olatade @tasneemkoushar I have tried the whole workflow so I have a question that when we merge the code from any other branch, is workflow triggered? If not we need to implement that and somehow Copy-docs-to-talawa-docs workflow is getting success but the changes are not reflected in my forked talawa-docs repo.

When copy-docs-to-talawa-docs workflow runs, you will be able to see some logs. There could be several reasons if the changes are not reflected.

1. No changes detected (nothing to commit)
2. You may be running the workflow in the automated-docs branch, try running the workflow in the develop or from the default branch by manually doing some changes to the documentation.

[anwersayeed]

We plan to not use this workflow "Copy-docs-to-talawa-docs". Either we find some alternative or write a script of our own.

[anwersayeed]

The workflow for the generation of the documentation files is already written, what we are looking for is a workflow or simply a script that runs with the push/ scheduled event, that copies these files from directly after generating them to the talawa-docs without committing them in between to any repositories or other branches.

[palisadoes]

Would this approach be viable?

1. Generate the docs in the respective repos as we do now.
2. Use a cron job in Docs to merge on top of the Docs default branch.
   1. We can change the default in Docs to develop and/or not use develop at all

[adi790uu]

@palisadoes so basically generating the docs in respective repos and then at a specific schedule get those changes from those repos?

[palisadoes]

1. Possibly, especially if it can be proven to be reliable
2. The current methodology is temperamental, however we don't know why.

[adi790uu]

Okay @palisadoes let me test this locally,I feel it's a good idea.

[adi790uu]

@Olatade can you assign me this issue.

[anwersayeed]

Hi @adi790uu, did you make any progress?

[adi790uu]

@anwersayeed I am currently working on the cron job part on talawa-docs repository, it would fetch all the latest changes in docs from respective repositories at particular schedule.

[adi790uu]

What's your opinion on this approach?

[anwersayeed]

@adi790uu Yeah, a centralised point to fetch all docs seems fine to me. Can you prepare a demo with dummy files and repositories?

[adi790uu]

@anwersayeed do you want me create an entirely new project and test as I was doing the testing and integration in talawa-docs repo.

[adi790uu]

@anwersayeed here is the script : #!/bin/bash

git clone --depth=1 --branch automated-docs https://github.com/adi790uu/talawa-api.git

mkdir -p docs/talawa-api-docs

cp -r talawa-api/talawa-api-docs/* docs/talawa-api-docs/

rm -rf talawa-api

ls

git config --global user.email "adi37068@gmail.com" git config --global user.name "adi790uu"

git add .

git commit -m "Pull changes from Talawa API automated-docs branch"

git push origin develop

[adi790uu]

@anwersayeed here is the script : #!/bin/bash

git clone --depth=1 --branch automated-docs https://github.com/adi790uu/talawa-api.git

mkdir -p docs/talawa-api-docs

cp -r talawa-api/talawa-api-docs/* docs/talawa-api-docs/

rm -rf talawa-api

ls

git config --global user.email "adi37068@gmail.com" git config --global user.name "adi790uu"

git add .

git commit -m "Pull changes from Talawa API automated-docs branch"

git push origin develop

basically will use this in a workflow and run a cron job each time the talawa-api-docs folder in talawa-docs will be replaced by latest available docs from automated-docs branch. Is this the right approach for solving this issue?

[adi790uu]

we can extend this for talawa-admin as well.

[anwersayeed]

@adi790uu Not necessarily a new project, this is fine. Complete your script, test it, and ping me.

[adi790uu]

@anwersayeed

the workflow is working in case there are changes in talawa-api-docs and pushing it to develop branch in talawa-docs

[adi790uu]

this is the version of script used :

!/bin/bash

git clone --depth=1 --branch automated-docs https://github.com/adi790uu/talawa-api.git

mkdir -p docs/talawa-api-docs

cp -r talawa-api/talawa-api-docs/* docs/talawa-api-docs/

rm -rf talawa-api

git config --global user.email "adi37068@gmail.com" git config --global user.name "adi790uu"

git add .

git commit -m "Pull changes from Talawa API automated-docs branch"

git push origin develop

[adi790uu]

what change need to made here as I am using my credentials:

git config --global user.email "adi37068@gmail.com" git config --global user.name "adi790uu"

[adi790uu]

similarly we can clone mobile and admin repos add the doc folders to talawa-doc and push them to develop.

[adi790uu]

@anwersayeed I need your review on above so that I can proceed further with this.

[adi790uu]

@anwersayeed sorry for bothering you but can you please review above, I have to proceed with the complete implementation and testing.

[anwersayeed]

@adi790uu Please find me on the Slack channel. We may need to connect tomorrow for above changes.

[adi790uu]

@adi790uu Please find me on the Slack channel. We may need to connect tomorrow for above changes.

Okay @anwersayeed will ping you on slack

[palisadoes]

How's progress on this?

[adi790uu]

@palisadoes this will be completed soon, after @anwersayeed approve the latest changes I will create a PR.

[adi790uu]

@palisadoes I have created a PR in docs repo this is for api-docs only mobile-docs and admin-docs will have different Pr's as suggested by @anwersayeed

[palisadoes]

Please work with @anwersayeed and @tasneemkoushar to review and merge them

[adi790uu]

Okay @palisadoes

[anwersayeed]

I have provided some review comments, please address them. @adi790uu

[adi790uu]

@anwersayeed I had to create a new PR as somehow the previous PR got closed, I have implemented the suggested changes in new PR.

[palisadoes]

The documentation workflow isn't functioning. We really need it activated. Any news?

[adi790uu]

@palisadoes yeah I am aware of the issue and in contact with @anwersayeed we are discussing about this.

[github-actions[bot]]

This issue did not get any activity in the past 10 days and will be closed in 180 days if no update occurs. Please check if the develop branch has fixed it and report again or close the issue.

[github-actions[bot]]

This issue did not get any activity in the past 10 days and will be closed in 180 days if no update occurs. Please check if the develop branch has fixed it and report again or close the issue.

[github-actions[bot]]

This issue did not get any activity in the past 10 days and will be closed in 180 days if no update occurs. Please check if the develop branch has fixed it and report again or close the issue.

[palisadoes]

unassigning, inactive

[gautam-divyanshu]

@anwersayeed Please help

[anwersayeed]

@gautam-divyanshu Hi, kindly text @tasneemkoushar on slack.

[gautam-divyanshu]

@palisadoes @anwersayeed @tasneemkoushar

Progress-

1. Push workflow runs when a PR is merged to develop branch
2. when PR is merged to develop then it creates a empty commit to automated-docs branch
3. then after empty commit copy-docs-to-talawa-docs workflow runs

it's working locally on my forked repo.