Document general usage of Gitpod

[mickmister]

My workflow of doing general Git operations in Gitpod has been done using the VSCode command palette. Opening the command palette can be done by pressing Cmd Shift P. Many of these steps can be done in the terminal instead of the command palette, but these shortcuts provided by VSCode are useful when working with a multi-project workspace.

Here's my workflow:

• Make changes to one of the projects
• Open the command palette and type checkout, which autocompletes to Git: Checkout to...
• Run that command and select the project folder I'm working on
• Select Create new branch...
• Type a new branch name, such as fix-channel-header-styling. Then press enter to create this branch.
• Stage files to be committed. This can be done in the Source Control tool in the left sidebar of VSCode.
• Open the command palette and type commit, which autocompletes to Git: Commit
• Run that command and select the project folder I'm working on
• Type my commit message in the editor, save the tab with Cmd S, and close the tab. If you are running Gitpod as an installed shortcut, you can type Cmd W to close the tab. The commit is now finished locally in your workspace. We now need to push the branch to your repository on GitHub. VSCode will help us create a fork in the next step.
• Open the command palette and type publish, which autocompletes to Git: Publish Branch...
• Run that command and select the project folder I'm working on
• Select origin as the remote to push to
• VSCode will ask if you want to create a fork, even if you already have one made. Click the Create Fork button in the bottom right corner of the screen. This will create a fork in your GitHub account, and set the origin remote to your fork.
• Your new branch with your changes is now on your GitHub account!
• If you would like to create a pull request from within the VSCode interface, you can click Create PR in the bottom right corner.
• I prefer to use GitHub's website to create pull requests, so I instead click Open on GitHub. Then you can see "(branch) had recent pushes less than a minute ago", with a button that says Compare and pull request
• You can now create a pull request with your changes!
• If you would like to add more changes on your branch/PR, you can repeat the staging and commit steps above, then run the Git: Push command to push your new commits onto your remote branch.

As noted in one of the steps above, I recommend installing Gitpod as a shortcut, so you can benefit from certain keyboard shortcuts, and have the opened workspaces live in their own dedicated windows on your computer.

Something I may need to fix is it seems when opening a workspace from a plugin project (or project not listed in the additionalRepositories list), it automatically set the upstream remote of the mattermost-gitpod-config repo to the plugin project's remote. Probably not a big deal because it only seems to affect the remotes for mattermost-gitpod-config, which someone will likely not be making edits to unless they are more familiar with this sort of workflow or workarounds anyway.

[mickmister]

@sinansonmez I'm curious how your workflow comapares to this, and if you have any shortcuts or improvements to add/edit here. I'd eventually like this to be in a walkthrough guide or even a tutorial video.

Though a video can get outdated pretty quickly. Maybe the guide can point to the video, and mention anything that may be out of date about the video. I think a video will be the most approachable medium to someone completely new to Gitpod or development/collaboration in general. cc @azigler for thoughts on this idea

[sinansonmez]

@mickmister My workflow is more or less the same. However, I use the user interface of VSCode. I documented my workflow below, together with highlighting the same steps as you with SAME notation. I also attached screenshots to support documentation efforts.

• I usually create new workspaces for each ticket / issue using mattermost/mattermost-gitpod-config repo as a starting point.

• SAME Make changes to one of the projects. I will follow an example using the webapp.

• Click the "Source Control" on left sidebar to create a branch.

  

• SAME Type a new branch name, such as fix-channel-header-styling. Then press enter to create this branch.

• Click publish branch to publish your branch on your fork.

• SAME However, current upstream is mattermost/mattermost-webapp. As I don't have access to mattermost repos, VSCode will ask if you want to create a fork, even if you already have one made. Click the Create Fork button in the bottom right corner of the screen. This will create a fork in your GitHub account. This action will also set your
  • origin remote to your fork (i.e. sinansonmez/mattermost-webapp in my case)
  • upstream remote to mattermost/mattermost-webapp

• SAME Stage files to be committed. This can be done in the Source Control tool in the left sidebar of VSCode (Action 1 in the picture). Push your commits (Action 2 in the picture)

• SAME If you would like to create a pull request from within the VSCode interface, you can click Create PR in the bottom right corner.
• SAME I prefer to use GitHub's website to create pull requests, so I instead click Open on GitHub. Then you can see "(branch) had recent pushes less than a minute ago", with a button that says Compare and pull request

• SAMEYou can now create a pull request with your changes!
• SAME If you would like to add more changes on your branch/PR, you can repeat the staging and commit steps above, then run the Git: Push command to push your new commits onto your remote branch.

[mickmister]

@sinansonmez Thanks for all of this awesome curated information and screenshots! I also like the idea of using a GitHub comment as an artifact of documentation efforts. Awesome job!!

[mickmister]

@cwarnermm @azigler The information in this issue thus far covers a chunk of the Gitpod development guide we're wanting to make. Specifically the "What to do once your in Gitpod" piece.

I see this as a "developer story" with a few parts:

• Choosing an issue
• Creating a Gitpod workspace from the appropriate repository. We will explain how to do this. I personally use the browser extension made by Gitpod, but we can also recommend our Gitpod links if we have them in our project READMEs.
• Programming in Gitpod and commiting work. This is partially covered in this issue, but it does not go in detail with how Mattermost is running inside of Gitpod. We should go in some detail about how this works.
• Submitting a pull request, discussing feedback, implementing any requested feedback, pushing new work onto the pull request.

We should also mention the role of the community server in our contribution process. We should encourage contributors to ask questions on the community server.

I think having this guide in place for Hacktoberfest would be a huge win

[cwarnermm]

Thank you, @mickmister! You've captured a wealth of great information in this issue! I agree that having this available for Hacktoberfest would be amazing.

@azigler - Do you have bandwidth to develop an initial draft of this excellent developer story in September?

[azigler]

Hello everyone -- @cwarnermm @mickmister @sinansonmez !

All of this information is really helpful, as someone who has not really used the Gitpod platform to try and develop for MM. I would love to make this into a video and/or guide, taking advantage of this collected knowledge and @mickmister's excellent organizing/enabling of the repos to be Gitpod-ready.

I made a card for this but have to push it out a few sprints -- currently getting developer experiences created for our fall conferences + Mattercon demo prep. Let's continue these discussions and start working on a step-by-step guide to follow? I'm going to approach it very naively, like any new developer would.

[simcard0000]

Hello everyone! Just wanted to add my two cents for the workflow on Gitpod that is most intuitive for me personally. I've been tasked with creating 2 videos for Hacktoberfest: getting started with Gitpod, and walking through how to create your first E2E test (which encompasses aspects like how to pick issues and submitting PRs).

Like @sinansonmez, I use the UI of Gitpod for all operations (including Git operations) - so I don't really use the command palette. I will also preface some of my steps with SAME if they match what others have posted in this issue. The steps will be in the vein of what you would do in the second video.

Choosing an issue/ticket

1. For finding an existing issue to work on, checking out help wanted tickets on GitHub is a good place to start. The list here is generated from these search terms: is:open archived:false org:mattermost label:"Help Wanted" label:"Up For Grabs".

2. You can also see a lot of issues that need help specifically in the issues tab of the mattermost-server repository and filtering with these search terms: is:issue is:open label:"Help Wanted" label:"Up For Grabs". At the top of this page are three pinned issues: Mattermost Plugins: Help Wanted tickets, 🎯 Focalboard: Help wanted tickets, and 📋 Playbooks: Help-wanted Tickets, which are portals to finding issues that relate to those projects respectively.

3. Once you find an issue you would like to work on (for example, Write Webapp E2E with Cypress: "MM-T642 Attachment does not collapse" #18184 that's been claimed by me), comment on the issue to claim it. For finding E2E issues specifically, try appending on another search term to filtering, like e2e.

4. An E2E ticket will have the following format:

   • Steps: what the code in the test should do/emulate
   • Expected: what the results of the test should be
   • Test Folder: where the file that holds the test code should be
   • Test code arrangement: starter code for the test
   • Notes: comments on what not to add to the test file, plus resources for contributing, asking questions, and writing code

Starting up Gitpod

5. SAME I create a new workspace for a ticket/issue by going to the mattermost-config-repo and clicking the "Open in Gitpod" badge.

   • I also have the Gitpod extension in my Firefox browser (because that's simpler than manually prefixing GitHub repository URLs with gitpod.io/#), if the repository that needs to be worked on is not mattermost-server, mattermost-webapp, or mattermost-plugin-apps. What the extension does is add a green Gitpod button to repository pages on GitHub, and clicking it spins up a new environment for the repository on Gitpod.

6. You may need to sign in (through GitHub) to access the workspace on GitPod. Once Gitpod has done loading, the user interface presented is that of VSCode. [In my first video, I will be going over the main parts of the IDE, and what has started up, but I won't explain that here.]

Working on an issue/ticket

7. SAME Make changes to one of the projects. In the context of the issue that I picked:

   • End-to-end (E2E) integration tests is where software gets checked from the perspective of the user (behavioral flow), whereas with unit testing software components are tested individually. E2E tests in Mattermost are written using Cypress and Playwright; for this issue, we will only need to use Cypress. Cypress documentation can be accessed here, as well as cheatsheets for how to use functions that are common in other Cypress tests in Mattermost repositories.

   • Navigate to the folder where the test file should be, as specified in the Test Folder section of the issue. This would be: mattermost-webapp > e2e > cypress > tests > integration > integrations > incoming_webhook. All tests live inside the ... > e2e > cypress > tests > integration folder.

• Notice that the test files here end with spec.js or spec.ts. If you need to create a new file to hold your test code (some issues might ask you to append to an existing test file instead), use a name that refers back to the issue you claimed. For example, something suitable might be attachment_does_not_collapse_spec.ts/'attachment_does_not_collapse_spec.js'.

  • Also notice that other test files have a comment header that includes information such as copyright and license information, a section to explain how to tag comments in your code appropriately, and two lines specifying the stage and group the test belongs to (which will help with running tests).

    • If you have created a new test file, you can add this to the top of your file:

      // Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
      // See LICENSE.txt for license information.
      
      // ***************************************************************
      // - [#] indicates a test step (e.g. # Go to a page)
      // - [*] indicates an assertion (e.g. * Check the title)
      // - Use element ID when selecting an element. Create one if none.
      
      // ***************************************************************
      // Group: @incoming_webhook

    • Otherwise, remove the @prod label from the comment header. This will also be mentioned in the Notes section of the E2E issue.

  • Underneath the comment header, we can add the starter code as defined from the Test code arrangement part of the issue. Each test has a corresponding test case listed in Zephyr, a test management program. The text of the describe block also corresponds to the folder name in Zephyr. The text of the it block has the Zephyr test case number as "Test Key" and then the test title. Since we've claimed an E2E issue with the Help Wanted and Area/E2E Tests label, the Test Key is already included in the starter code.

     describe('Integrations/Incoming Webhook', () => {
         it('MM-T642 Attachment does not collapse', () => {
         // code
         });
     });

  • Before we write the main body of the test in the it block, it can help to write some setup code before for test isolation. For more information, check out this page on basic test code structure in the Mattermost developer documentation.

    • In attachment_does_not_collapse_spec.js:

       let incomingWebhook;
       let testChannel;
      
       before(() => {
           // # Create and visit new channel and create incoming webhook
           cy.apiInitSetup().then(({team, channel}) => {
               testChannel = channel;
      
               const newIncomingHook = {
                   channel_id: channel.id,
                   channel_locked: true,
                   description: 'Incoming webhook - attachment does not collapse',
                   display_name: 'attachment-does-not-collapse',
               };
      
               cy.apiCreateWebhook(newIncomingHook).then((hook) => {
                   incomingWebhook = hook;
               });
      
               cy.visit(`/${team.name}/channels/${channel.name}`);
           });
       });

  • Now, inside the body of the it block , we will write in code the Steps part of the issue. More information on the functions used here can be found at: Which query to use? and the Cypress cheatsheet section of the Mattermost developer documentation.

    1. Create an incoming webhook and send POST with attachment

       // # Post the incoming webhook with a text attachment
       const content = '[very long lorem ipsum test text]';
       const payload = {
         channel: testChannel.name,
         attachments: [{fallback: 'testing attachment does not collapse', pretext: 'testing attachment does not collapse', text: content}],
       };
       cy.postIncomingWebhook({url: incomingWebhook.url, data: payload, waitFor: 'attachment-pretext'});

    2. View the webhook post that has the attachment: already in the channel that has the attachment post, as specified by the line cy.visit(`/${team.name}/channels/${channel.name}`); from the setup section of the code

    3. Type /collapse and press Enter

       // * Check "show more" button is visible and click
       cy.getLastPostId().then((postId) => {
         const postMessageId = `#${postId}_message`;
         cy.get(postMessageId).within(() => {
             cy.get('#showMoreButton').scrollIntoView().should('be.visible').and('have.text', 'Show more').click();
         });
       });
       
       // # Type /collapse and press Enter
       const collapseCommand = 'collapse';
       cy.uiGetPostTextBox().type(`/${collapseCommand} {enter}`);

    4. Observe the integration post with the Message Attachment: where we ascertain what is Expected (as noted in the issue) of the test

       cy.getNthPostId(-2).then((postId) => {
       const postMessageId = `#${postId}_message`;
       cy.get(postMessageId).within(() => {
        // * Verify "show more" button says "Show less"
        cy.get('#showMoreButton').scrollIntoView().should('be.visible').and('have.text', 'Show less');
       
        // * Verify gradient
        cy.get('#collapseGradient').should('not.be.visible');
       });

8. We can check the test that's been written as follows:

   • In Gitpod, the server and the webapp should still be running. However, you can also run make test-data in the terminal for the mattermost-server repository to preload your server instance with initial seed data.

   • Next, in the terminal for the mattermost-webapp repository, enter the directory mattermost-webapp/e2e/cypress using the cd command, and then run npm i, which will install more dependencies needed for testing.

   • To run all E2E tests, run npm run cypress:run in the terminal for the mattermost-webapp repository. This does not include the spec files in the /cypress/tests/integration/enterprise folder because they need an Enterprise license to run successfully.

• You can also run tests selectively based on the metadata in the test code (located in the top header comment). For example, if you want to run all the tests in a specific group, such as those in "incoming_webhook", the command would be: node run_tests.js --group='@incoming_webhook'.

9. Finally, we need to make sure that our code is formatted properly. We can check this by running the command npm run check in the terminal for the mattermost-webapp repository. If there are any problems, we can either fix them manually in the file itself, or use the npm run fix command. This is an important step because if any code is not formatted properly, one of the checks that run when you create a pull request for your branch will fail, and may block merging.

Creating a new branch on your own fork

10. SAME You most likely won't have direct write access to the repository you are working on. Thus, you will need to bring the changes you've made over from the master or main branch to a new branch on your own fork of the repository.

    • SAME Click the "Source Control" icon on the left sidebar. On the panel next to the side bar, you will see a list of the repositories in the workspace, and under each repository a list of the files that have changed (if any). Next to the label for mattermost-webapp, click the button with the branch name and the source control icon.

    • SAME A dropdown will appear via the command palette. Click the option that says: "+ Create new branch..."

• SAME In the box that appears, name your new branch. A good name idea is to name it after the code that refers to your issue - in this case, this is the "Test Key", so a suitable name for the branch is MM-T642. Then, hit Enter on your keyboard.

• SAME Now that the branch has changed, we can commit our changes to it. Back in the source control panel, we can make a commit message in the text box above the "commit button" in the section for the mattermost-webapp repository. Then, press the "commit button". A modal may appear asking you to first stage your changes.

• SAME We can now publish our branch to the remote. However, as we will not have access to the main repository itself, we will be prompted to first create a fork. Click on the "publish branch button" on the source control panel. A popup will appear, asking if you would like to make a fork and push to that instead. On the popup, select the "create fork" button.

• SAME During the process of fork creation, you may be prompted to grant GitHub access to Gitpod's GitHub extension (you should allow this).

   - You may also be asked to update the permissions you give Gitpod to access GitHub through a popup. If this happens, you will have to restart the fork creation process (publish branch -> say yes to creating a fork). Click the "open access control" button on the popup.

   - You will get taken to the Integration section of Gitpod's settings. In the list of Git Providers on the page, find GitHub, click the three dots next to the listing, and select "Edit Permissions" from the dropdown menu.

   - A popup will appear with a list of permission checkboxes. It's a good idea to check all of them off so you don't need to go through this process again. When you're done checking off the permissions, click the "Update Permissions" button.

   - Another tab might also appear where on GitHub's end you accept the additional permissions Gitpod is requesting. Click on the "authorize gitpod-io" button.

• SAME Once the forking process is done, there will be a couple of popups that appear on Gitpod: one asking if you'd like to periodically run git fetch (which will periodically download content from the remote), another one asking if you'd like to create a pull request for the branch you're on right inside Gitpod/the VSCode editor, and finally one informing that the fork was successfully created.

  • SAME On the successful fork creation popup, click the open on GitHub option.

Creating a Pull Request

11. You'll get taken to Github, to the fork of the repository you've worked on in your account instead of the main one in the Mattermost organization. Navigate to the branch you've made on your fork if you're not there already.

    • Near the top of the page will be a bar mentioning how many commits ahead/behind your branch is from the master/main branch of the main repository. There will also be two buttons: "contribute" and "sync fork". Click on the "contribute" button, and in the dropdown that appears, click the "open pull request" button.

12. You'll be taken to another page on GitHub called "Open a pull request".

• The first section of the page compares whether the branches (your branch on your fork - the "head repository" vs. the master/main branch on the main repository - the "base repository") can be merged automatically.

• The second section of the page will show other PRs that are based on your same branch, if any.

• The third section is where you create a write-up for your pull request - giving it a title, and filling out the template. At the bottom of this section is a "Create pull request" button. This button will be faded out until you make a title for the PR.

  • If you haven't already, give Mattermost's Contribution Checklist a read. An important takeaway is that you will need to sign the Contributor License Agreement - this will be another check on the pull request and if you haven't signed it it will also block merging.

  • Also check out this blog post about Submitting Great PRs, and other repository specific information here.

  • Parts of a PR body:

    • Title: a good title will refer back to the issue; and it should begin with the related Jira or Github ticket ID (e.g. [MM-394] or [GH-394]). In the context of the E2E issue example: MM-T642: Attachment does not collapse - Cypress Webapp E2E Test.
    • Summary: description of what the PR does, as well as QA test steps (if applicable and if not already added to the Jira ticket). For example: Verifies that attachments on posts do not collapse after entering the slash command collapse.
    • Ticket Link: Either link the relevant Jira ticket or if you picked up an issue/ticket with a Helped Wanted label, link to the GitHub issue. For example: Write Webapp E2E with Cypress: "MM-T642 Attachment does not collapse" #18184.
    • Related Pull Requests: Link other PRs here if they are related to this PR.
    • Screenshots: Illustrate what your changes have done.

    • Release Note: There are certain conditions that require release notes:

      • Config changes (additions, deletions, updates).

      • API additions—new endpoints, new response fields, or newly accepted request parameters.

      • Database changes (any).

      • Schema migration changes. Use the Schema Migration Template as a starting point to capture these details as release notes.

      • Websocket additions or changes.

      • Anything noteworthy to a Mattermost instance administrator (err on the side of over-communicating).

      • New features and improvements, including behavioral changes, UI changes, and CLI changes.

      • Bug fixes and fixes of previous known issues.

      • Deprecation warnings, breaking changes, or compatibility notes.

        If no release notes are required, write NONE. Use past-tense. For E2E tests, having NONE as a release note suffices. If you do not end up writing a release note at all; you'll get a warning on your PR like this: Adding the "do-not-merge/release-note-label-needed" label [to the PR] because no release-note block was detected, please follow our release note process to remove it. If this happens, you can just edit the body of the PR, and add it back in.

• The last section details the code changes on your branch, including information on the commits on the branch, the files changed, and the contributors.

13. Once you've created your pull request, you'll get taken to it's page, like this one: MM-T642: Attachment does not collapse - Cypress Webapp E2E Test #11231. Below your initial body text of the PR will be a list of commits and other comments. At the end of this list is a checklist which notes the status of reviews required for the pull request, and the checks that the pull request must pass, plus a place to write your own additional comments.

• SAME if you need to make any changes to your PR, you can return to Gitpod and stage and commit your changes from there onto your branch; and this will be reflected in GitHub.

Code Review

Information from this section comes from: Code review at Mattermost

14. Wait for a reviewer to be assigned - normally this is handled automatically, but if you need help feel free to ask for help in the Developers channel of the Mattermost community server.

15. Wait for a review - if a reviewer requests changes, your PR will disappear from their queue of reviews. Once you've addressed the concerns, re-request a review from any person requesting changes. Avoid force pushing, which is the act of overwriting the commit history on the remote with what you have on local.

16. Once all reviewers approve your pull request, they will handle the merging of your code. ✔️

This is a pretty long-winded post, but can hopefully serve as the basis of a blog post or an article for reference. If there is anything wrong with this post (spelling, grammar, or content-wise), please let me know.

[simcard0000]

Edit 1 for this comment: looks like based on this issue (https://github.com/mattermost/mattermost-gitpod-config/issues/20) working on focalboard is not supported yet.

[simcard0000]

Also tagging @cwarnermm @mickmister and @azigler for visibility 📣

[sinansonmez]

thanks @simcard0000 super detailed guide.

Currently, I am having trouble when I do npm i in e2e/cypress folder. It got stuck at some point and no progress.

I already to tried to delete node-modules folder and package-lock.json but no luck. Any idea why this is happening and how I can overcome this issue?

edit: it was fixed after waiting long time

[simcard0000]

@sinansonmez Good to know! Let me know if anything else comes up :)

[sinansonmez]

One more thing to be added to guide. Please see @mickmister detailed explanation https://github.com/mattermost/mattermost-gitpod-config/pull/10#issuecomment-1223523082

[mickmister]

@simcard0000 Absolutely amazing work on this!!!! I'm wondering how this can be laid out in our docs in a digestible format as there is a lot of information to take in here. Maybe spread the content out into multiple pages? Or one large page with table of contents? It would be much easier to review the content if it were put into a PR, but I think we should determine the format we want it to exist in the docs first. @cwarnermm What are your thoughts on the format we can approach here?

@simcard0000 @sinansonmez Since this guide incorporates e2e testing, I'm wondering if we should add some more automation in the workspace creation process for e2e tests specifically. This would include instantiating a terminal for the e2e folder, and fetching dependencies during workspace creation. Possibly even running the cypress process and outputting in the console how to access the cypress window (or in a welcome page https://github.com/mattermost/mattermost-gitpod-config/issues/22). Though auto-running e2e processes could put unnecessary load on the workspace if the dev is not going to be interacting with e2e tests.

Something I want to include in the guide is the default login that is made when the workspace runs make test-data. User: sysadmin Password: Sys@dmin-sample1. I've been wondering how new devs have been figuring out this login info.

[sinansonmez]

I've been wondering how new devs have been figuring out this login info.

@mickmister After make test-data command is completed, username and password information is provided to the developer like below

[mickmister]

In the Gitpod workspace creation process, this command is run automatically, and more commands are run afterwards, so this output is sort of buried in the terminal's logs. I'm thinking it would be beneficial to have these credentials more visible for Gitpod docs specifically for this reason

[saturninoabril]

@simcard0000 This is so great! For https://github.com/mattermost/mattermost-gitpod-config/pull/10#issuecomment-1223523082, you may look at https://community.mattermost.com/core/pl/ju6npwippjbhbpduz96mj8i8ah. I was planning to do it at one point but if you'd like to add in the docs, it would be so nice. Otherwise, I'll help add in the future once the docs/format has been decided/merged.

[simcard0000]

@mickmister Thanks for the feedback!

• Definitely think it's a good idea to put this somewhere in the docs, I'll loop Carrie in on Mattermost (in the Community Onboarding Refresh 2022 channel).
• Speaking of automating some parts of the e2e process, I think at the very least instantiating a terminal for the e2e folder, and fetching dependencies during workspace creation would be nice.
• +1 for a welcome page, that would be a great place for linking further resources
• +1 for also making the make test-data information more visible (at least in the command line)

@saturninoabril made note of that thread! I think it would be good to add it to what I wrote

[cwarnermm]

Huge +1 to this great information being incorporated into the developer documentation! @simcard0000 Really great work on this! The content and the way you've chunked up the process into bite-sized steps is perfect for inclusion in the docs as individual pages:

Develop using GitPod (open to ideas on this title!)

• Choose an issue/ticket
• Start up GitPod
• Work on an issue/ticket
• Create a new branch on your own fork
• Create a pull request
• Code reviews

"issue" versus "ticket" - let's pick one term to use consistently. Which one resonates more with our target audiences?

@simcard0000 - Let either myself or @neflyte know how best we can assist you with this effort.

[simcard0000]

Check out: https://github.com/mattermost/mattermost-developer-documentation/pull/1170 (first pass at it) 🎊