Acceptance Testing - covering all user journeys in form of user stories

[Shivkant-Chauhan]

Describe the bug

In order to verify that key user journeys do not break, the Oppia development team is implementing Acceptance Testing for the develop branch. This means that, on every commit, all key requirements are verified through a set of acceptance tests that check that each type of end user can do the things they expect to do on the site.

The aim of this issue is to build up a set of such tests. Please see the instructions below for how to claim a task in this issue. (Feel free to contact @oppia/acceptance-test-reviewers if you have any doubts!)

# Instructions

1. Carefully read the “How to write Acceptance Tests” wiki page.
2. Select 1 or more “package” (of 1-2 user stories each) from the list below, and get yourself assigned to them from @oppia/acceptance-test-reviewers
3. After the user stories are assigned to you, go to the user file for the tests inside core/tests/puppeteer-acceptance-tests/spec/<user-type>tests.spec.js
4. Start writing the assigned user stories based on the Testing Spreadsheet. Each user story in the spreadsheet should be covered within 1 it block, and all the test steps and expectations should match the expected behavior. You might also need to write utility functions, similar to the existing examples below.
5. After you have written the tests, make sure they run fine on your local machine before submitting your PR. If this is your first time writing acceptance tests, we suggest writing the tests for one user story and verifying that it runs fine before adding the rest of the tests to your PR.
6. In case of any confusion or for PR reviews, feel free to ask @oppia/acceptance-test-reviewers
7. After getting assigned to a package(containing 2-3 user stories), your inactivity/no communication for more than 7 days may lead to getting unassigned from the task assigned.

For reference: You can see an example set of acceptance tests for “blog admin”/”blog editor” here. Please follow a similar format and file naming convention for the tests that you write:

• Top-level tests:
  • core/tests/puppeteer-acceptance-tests/spec/blog-admin-tests/
  • core/tests/puppeteer-acceptance-tests/spec/blog-editor-tests/
• User-utilities: core/tests/puppeteer-acceptance-tests/userutilities/blog-post-admin-utils.js
• User factory: core/tests/puppeteer-acceptance-tests/spec/blog-admin-tests.spec.js

List of user stories (arranged by user type)

(serial numbers same as testing spreadsheet row number for each user type)

Exploration Creator

• [ ] 3- Users can record voiceovers in different languages, upload voiceovers for their question explorations in different languages, and monitor progress…………. 4- Users can create questions from the exploration, add questions from the exploration

• [x] 5- Users can see history in the exploration editor, can check the latest version with notes, user, and date, and compare any two versions from ………… -- @Akhilesh-max (PR: https://github.com/oppia/oppia/pull/19828) 6- User can preview the exploration, and can start the questions from the beginning -- @Akhilesh-max

• [ ] 7- User can navigate to the Feedback Tab: The user can access the Feedback Tab in the Exploration Editor and view all the feedback …………. 8- User can ask for "Help" for the editor's exploration, take a tour of the editor's exploration, visit the help centerTake the translator tour

• [x] 9- User can do the Basic setting: Title, Goal, Add a category, Language, Name of the first card, Tags, Advance features, Roles, Voice artist, Permissions …….. -- @rahat2134 (PR: https://github.com/oppia/oppia/pull/19780) 10- User can Publish the latest changes, User can draft the latest changes -- @rahat2134

• [ ] 11- User can use all the interactions.

• [ ] 13- User can use all RTE components.

• [ ] 12- User can publish an exploration with an interaction. 14- User can view and make changes to a previously-published exploration 15- User can see statistics about number of views, ratings, etc. in the creator dashboard

Contributor Dashboard

• [ ] Translation submitter Tracking bug: https://github.com/oppia/oppia/issues/18892 1.1 User can select a topic and a language, and then successfully submit a translation for that lesson card in that language. Translation Reviewer 2.1 User can view the translated lessons in a topic, review the cards in the lesson and accept the translation in each card

• [x] Translation Admins Tracking bug: https://github.com/oppia/oppia/issues/18893 3.1 Admin should be able to provide Translation rights in a language to a user 3.2 Admin should be able to remove translation rights in a language to a user assigned to @agallop

• [ ] Practice Question Submitter Tracking bug: https://github.com/oppia/oppia/issues/18894 4.1 Users can suggest questions by selecting the difficulty level of a lesson in a Topic 4.2 Users can suggest questions by selecting difficulty level as Hard to a lesson in a Topic

• [ ] Practice Question Reviewer Tracking bug: https://github.com/oppia/oppia/issues/18896 5.1 Users can review a question on a topic and Accept it 5.2 Users can review a question in a topic and Reject it

• [ ] Practice Question Admins Tracking bug: https://github.com/oppia/oppia/issues/18897 6.1 Admin can provide Contribution Rights to submit and review Questions 6.2 Admin should be able to remove Review and Submit Question permissions to a user

Logged-in Learner @AFZL210

• [ ] 3- User can add an exploration to 'play later' from the /community-library page, remove that exploration from 'play later' from the /learner-dashboard page………. (draft PR: https://github.com/oppia/oppia/pull/20043)

• [ ] 4- User can subscribe to a creator and should be able to view all explorations authored by that creator. (draft PR: https://github.com/oppia/oppia/pull/20195)

• [ ] 5- User can check current progress from the lesson info modal, can return to the most recent checkpoint visited, and resume the exploration after the previously visited checkpoint 6- User can change his profile, export account (upload thumbnail image, add a bio, add subject interest , add site language add audio language) ……………

Anonymous Learner

• [ ] 3- Users can play an exploration as an anonymous user, search for exploration on the community library and play it 4- Users can leave feedback on an exploration (particular state), eave feedback on an exploration (whole exploration)

Logged-in user (browsing static pages)

• [x] 3- Users can open the About, The Oppia Foundation and Blog pages, Schools and Organizations, Volunteer, Donate, Contact us and Donate in the Nav bar -- @imchristie 11- Users can open About, The Oppia Foundation, Blog, Forum, Get Started, Creator Guidelines, For Parents/teachers, Browse the Library, Contact Us,......... -- @agallop

Topic Manager

• [ ] 3- As a topic manager user should be able to filter topics and skills, can merge skills, assign, unassign skills, and delete topic/skill 4- As a topic manager user should be able to create questions using image interaction and modify images

• [ ] Voiceover Artist 3- Voiceover artist can add audio recordings to any exploration Voiceover Admin @hrshkshri ( PR: #19994 ) 5- Voiceover admin can add voiceover artists to an exploration

• [ ] Curriculum Admin @Vir-8 ( PR: #19914 ) 1- As a curriculum admin, the user should be able to create a Topic, sub-topic, skill, story, chapter and publish it

• [ ] Curriculum Admin 2- As a curriculum admin user should be able to create a Classroom, add topics to it

• [ ] #19781 3- As a learner I should be able to play an embedded lesson

[github-actions[bot]]

Hi @Shivkant-Chauhan, thanks for proposing this as a good first issue. I am removing the label for now and looping in @DubeySandeep to approve the label. It will be added back if approved. Thanks!

[github-actions[bot]]

Hi @Shivkant-Chauhan, thanks for proposing this as a good first issue. I am removing the label for now and looping in @DubeySandeep to approve the label. It will be added back if approved. Thanks!

[caramelmelmel]

Hi @ashish-patwal and @Shivkant-Chauhan can I get 7 and 8 of exploration creator assigned to me? thank you! since it's my first time writing such acceptance testing, i might need some time or guidance on this aspect.

1. As a pre-requisite, does my browser need to redirect to the /creator-dashboard route?
2. If there are bugs encountered in the code that is to be tested, do I flag or fix the developed code bugs?
3. Also, having done some user testing, can I suggest some UX changes? One example is the placing of the icons that may be unintuitive Thank you!

[Shivkant-Chauhan]

Hi @ashish-patwal and @Shivkant-Chauhan can I get 7 and 8 of exploration creator assigned to me? thank you! since it's my first time writing such acceptance testing, i might need some time or guidance on this aspect.

1. As a pre-requisite, does my browser need to redirect to the /creator-dashboard route?
2. If there are bugs encountered in the code that is to be tested, do I flag or fix the developed code bugs?
3. Also, having done some user testing, can I suggest some UX changes? One example is the placing of the icons that may be unintuitive Thank you!

Hey @caramelmelmel, assigned you to the 7,8 user stories of exploration creator and thanks for showing your interest in the Acceptance tests. I and @ashish-patwal will be there for you for clearing your blockers and helping you out when stuck. But get a fair idea by reading the wiki pages for Acceptance tests and go through the reference PR #16394 (links are shared within the issue desc.).

1 --> yes, you need to have the local server setup on your system and you must be accessible to the /creator-dashboard page. also, you need to have a proper idea of all the components present on the page and what must be the expected behavior of the components (if there is a bug anywhere). 2 --> if you are able to fix the bug within the PR itself, that would be very appreciable. otherwise, you need to open an issue for the bug you encountered with proper desc. 3 --> it depends if your idea seems good to the product reviewers, then we can implement your suggested changes.

[marcella-stefan]

Hello @ashish-patwal and @Shivkant-Chauhan, it's my first time contributing to an open source software and I would like to get assigned to (5,6) Exploration Creator or/and Anonymous Learner(3,4)?

[Shivkant-Chauhan]

Hello @ashish-patwal and @Shivkant-Chauhan, it's my first time contributing to an open source software and I would like to get assigned to (5,6) Exploration Creator or/and Anonymous Learner(3,4)?

hey @marcella-stefan , since you are new to the codebase, so I am assigning you only 1 package from Exploration Creator. Please ask here in case of any blocker/confusion.

[Shivkant-Chauhan]

@caramelmelmel please share the updates on the package you are assigned.

[SahilB1]

Hi @Shivkant-Chauhan and @ashish-patwal, this is my first time contributing to an open source project as well, would I be able to get assigned to Logged-in user browsing static pages (3, 11)?

[Shivkant-Chauhan]

Hi @Shivkant-Chauhan and @ashish-patwal, this is my first time contributing to an open source project as well, would I be able to get assigned to Logged-in user browsing static pages (3, 11)?

sure @SahilB1 ! assigning you to the requested package.

[SahilB1]

@Shivkant-Chauhan Thank you! Will the acceptance tests wiki page have the information needed to get started on understanding how to go about writing these tests?

[SahilB1]

@Shivkant-Chauhan @ashish-patwal I tried to run the acceptance tests as they are using the command python -m scripts.run_acceptance_tests from my root directory of oppia, but I'm getting a No module named scripts.run_acceptance_test error when I try to run the command.

Additionally, running jasmine --config="core/tests/puppeteer-acceptance-tests/jasmine.json" core/tests/puppeteer-acceptance-tests/spec/blog-editor-tests/check-blog-editor-unable-to-publish-duplicate-blog-post.spec.js as the example test in the acceptance test page yields the response

Running suite with 0 specs. No specs found Finished in 0.002 seconds Incomplete: No specs found

Looking into the actual scripts directory in my cloned repo, it looks like run_acceptance_test.py does not exist in this directory. Could it have moved elsewhere or have been renamed?

[Shivkant-Chauhan]

@Shivkant-Chauhan @ashish-patwal I tried to run the acceptance tests as they are using the command python -m scripts.run_acceptance_tests from my root directory of oppia, but I'm getting a No module named scripts.run_acceptance_test error when I try to run the command.

Additionally, running jasmine --config="core/tests/puppeteer-acceptance-tests/jasmine.json" core/tests/puppeteer-acceptance-tests/spec/blog-editor-tests/check-blog-editor-unable-to-publish-duplicate-blog-post.spec.js as the example test in the acceptance test page yields the response

Running suite with 0 specs. No specs found Finished in 0.002 seconds Incomplete: No specs found

Looking into the actual scripts directory in my cloned repo, it looks like run_acceptance_test.py does not exist in this directory. Could it have moved elsewhere or have been renamed?

we are still having a PR opened on that (#17364 ) that will fix both the things you asked for. for now, run tests with: jasmine --config=core/tests/puppeteer-acceptance-tests/jasmine.json --filter="Blog Editor" As soon as the above PR is merged, all the things will be running smoothly as stated in the wiki pages.

[SahilB1]

@Shivkant-Chauhan sounds good, thank you!

[caramelmelmel]

Thanks for the response! I'm also experiencing the same issue! In the meantime, could we copy the PR that fixes #17364 to run the acceptance tests?

[Shivkant-Chauhan]

@caramelmelmel @SahilB1 until the PR #17364 is not merged, please run the test using: jasmine --config=core/tests/puppeteer-acceptance-tests/jasmine.json --filter="Blog Admin" command in your root /oppia directory

[caramelmelmel]

sure! thank you so much! right now, I'm asking for signed in users wise, must I mock the sign in phase?

[SahilB1]

@Shivkant-Chauhan Thank you! Would you happen to have an idea or estimate as to when PR #17364 might be merged? No worries if not, just curious.

[Shivkant-Chauhan]

@SahilB1 We are making it asap. but it might not be a blocker for you!.. you can start writing tests by referring to the wiki and the already written tests. also just to run your test, use jasmine --config=core/tests/puppeteer-acceptance-tests/jasmine.json --filter="{{your suite name}}"

[SahilB1]

@Shivkant-Chauhan Perfect, I'll be sure to do so. Thanks!

[jjxiong23]

Hi @Shivkant-Chauhan and @ashish-patwal, can I get the Practice Question Reviewer (5.1 and 5.2) of Contributor Dashboard assigned to me? I'm a college student and looking to contribute to an open source project as part of a class I'm taking right now. Thank you!

[noramark]

Hi @Shivkant-Chauhan and @ashish-patwal, Could I possibly be assigned to Practice Question Admin 6.1 and 6.2? I'm also a college student and a first time contributor to an open source project. Thank you!

[Shivkant-Chauhan]

@jjxiong23 @noramark done!

[maxliu2001]

Hello @Shivkant-Chauhan and @ashish-patwal, I'm wondering if I can be assigned to Logged-in learner (3,4)? I'm a first time contributor to an open source project and I'm taking a class in my college that asks me to contribute to an open source project. Thank you!

[SACHARYAS]

Hi @Shivkant-Chauhan and @ashish-patwal, I'm a cs student and have a open source project as a class final project. can I get the Practice Question submitter (4.1 and 4.2) of Contributor Dashboard assigned to me?

[Shivkant-Chauhan]

@maxliu2001 @SACHARYAS done!

[katehrkim]

Hi @ashish-patwal and @Shivkant-Chauhan can I get 5 and 6 of logged in learned assigned to me? thank you!

[michbyiri]

Hi, @ashish-patwal and @Shivkant-Chauhan can I get 4 or 9 of Exploration Creator assigned to me? Thank you!

[aashanp01]

Hi @ashish-patwal and @Shivkant-Chauhan, just following up to see if we can get these tasks assigned to us, thanks!

Hi, @ashish-patwal and @Shivkant-Chauhan can I get 4 or 9 of Exploration Creator assigned to me? Thank you!

[suraj-mandal]

HI @ashish-patwal and @Shivkant-Chauhan . I am new to the codebase and intereseted in contributing to the project. Can I get some of the user stories assigned to me? Thanks a lot!

[Shivkant-Chauhan]

@katehrkim @michbyiri @suraj-mandal I have assigned each of you with a package. PTAL!

[Shivkant-Chauhan]

@SahilB1 @caramelmelmel @noramark @jjxiong23 can you please update me with the user stories you are working on.

[noramark]

Hi @Shivkant-Chauhan currently working on the practice question admin story - ensuring the admin can provide a user with contribution rights (being able to review and submit a question) - For updates generally, is there specific information I should be providing or updating with?

[jjxiong23]

Hi @Shivkant-Chauhan, I'm also currently working on writing the acceptance tests for practice question reviewers. Starting with the first user story (accepting a question on a topic), and then will move on to the second user story (rejecting a question on a topic). Seconding @noramark, how should we keep you updated as we work on these tests?

[SahilB1]

Hi @Shivkant-Chauhan, I'm currently working on writing the acceptance tests for logged-in users navigating to static pages via the navbar and footer sitemap to ensure that the users are navigating to the correct pages when clicking on the links to these static pages.

[suraj-mandal]

Hi! @Shivkant-Chauhan, I was trying to run the acceptance test using the command

python -m scripts.run_acceptance_tests

However, I am getting the given error:

/home/suraj-mandal/Desktop/suraj-work/OSPs/oppia/.direnv/python-3.8.15/bin/python: No module named scripts.run_acceptance_tests

When I checked thescripts directory I found that there is no such script with the given name. I have installed and configured Oppia as per the instructions presented in the installation page.

Can you help me figure out how exactly the acceptance tests can be run?

My OS is Ubuntu 22.04.

[suraj-mandal]

Here, I tried running the server first at localhost:8181. Then I am executing the jasmine script. But getting the error.

Since I am new to tests, I need to know if I am proceeding correctly or not.

Getting this error:

Here is the full stack trace

[suraj-mandal]

Can someone help me with configuring Jasmine for e2e testing... I have been trying since yesterday, unable to make any progress

[michbyiri]

Hi, @ashish-patwal and @Shivkant-Chauhan, we finished writing the acceptance test for exploration creator 4, and we are just wondering where do we submit our code for review? Or do we just push it to the repo?

[Shivkant-Chauhan]

Hi, @ashish-patwal and @Shivkant-Chauhan, we finished writing the acceptance test for exploration creator 4, and we are just wondering where do we submit our code for review? Or do we just push it to the repo?

you need to open a PR for the same! also please try to check that your tests are running fine in your local machine

[Shivkant-Chauhan]

Can someone help me with configuring Jasmine for e2e testing... I have been trying since yesterday, unable to make any progress

Hi, would it be fine for you to ping me on the Google Chat? --shivkantchauhan2002@gmail.com

[SACHARYAS]

Resolved, missed something while rereading

[SACHARYAS]

Hi @ashish-patwal and @Shivkant-Chauhan, To use the type function do we find the CSS selector of the input field by inspecting elements on the website or is there another way?

[suraj-mandal]

Hi, @Shivkant-Chauhan pinged you

[agallop]

Hello @Shivkant-Chauhan. I am a first-time contributor and would like to tackle the Translation admins user journeys (3.1, 3.2). Would you mind assigning me?

Thank you

[SACHARYAS]

Hi @ashish-patwal and @Shivkant-Chauhan, following up on my question from 2 weeks ago. I'm almost done but need this clarified before I can pull request

[Shivkant-Chauhan]

Hi @ashish-patwal and @Shivkant-Chauhan, To use the type function do we find the CSS selector of the input field by inspecting elements on the website or is there another way?

Hi @ashish-patwal and @Shivkant-Chauhan, To use the type function do we find the CSS selector of the input field by inspecting elements on the website or is there another way?

Yes, you need to have the CSS selector for that field. Please go through the already written tests of Blog Admin and Blog Editor, you will get a fair idea of how to proceed

[Shivkant-Chauhan]

Hello @Shivkant-Chauhan. I am a first-time contributor and would like to tackle the Translation admins user journeys (3.1, 3.2). Would you mind assigning me?

Thank you

done..

[Shivkant-Chauhan]

Hello everyone, I would like to inform that the PR for the acceptance tests workflow and the script for executing the acceptance tests has been submitted to the develop branch. From now on, it is possible to run acceptance tests for any suite by using the command python -m script.run_acceptance_tests --suite="{{suiteName}}". It is important to ensure that your branches are updated with the latest changes from the develop branch.

(tagging all for notifying) @maxliu2001 @SACHARYAS @noramark @agallop @suraj-mandal @michbyiri @SahilB1 @jjxiong23 @aashanp01 @katehrkim @caramelmelmel @marcella-stefan

[suraj-mandal]

Thanks a lot.. it will be easier now

[caramelmelmel]

thanks! will note this! pulling branch now!