search

overhangio / tutor-discovery

Course Discovery plugin for Tutor
GNU Affero General Public License v3.0
12 stars 42 forks source link

Program MFE page 404 ouccred and error detected in Discovery #77

Closed Yagnesh1998 closed 4 months ago

Yagnesh1998 commented 4 months ago

I am currently working on quince release in my local system, I have already followed this step

tutor plugins install discovery
tutor plugins enable discovery
tutor dev launch
tutor dev run discovery ./manage.py createsuperuser

Then I followed this document to make the program

when i hit this comand tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=openedx

WARNINGS:
course_metadata.Course.collaborators: (fields.W340) null has no effect on ManyToManyField.
course_metadata.Degree.specializations: (fields.W340) null has no effect on ManyToManyField.
course_metadata.SearchDefaultResultsConfiguration.courses: (fields.W340) null has no effect on ManyToManyField.
course_metadata.SearchDefaultResultsConfiguration.programs: (fields.W340) null has no effect on ManyToManyField.
taxonomy.JobPostings.job: (fields.W342) Setting unique=True on a ForeignKey has the same effect as using a OneToOneField.
    HINT: ForeignKey(unique=True) is usually better served by a OneToOneField.
taxonomy.SkillsQuiz.skills: (fields.W340) null has no effect on ManyToManyField.
2024-06-27 12:03:56,433 INFO 1 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:134 - Command is not using threads to write data.
2024-06-27 12:03:56,434 INFO 1 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:178 - Executing Loader CoursesApiDataLoader, url: http://local.edly.io/api/courses/v1/
2024-06-27 12:03:56,436 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:57 - Refreshing Courses and CourseRuns from http://local.edly.io/api/courses/v1/...
2024-06-27 12:03:56,436 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:103 - Requesting course run page 1...
2024-06-27 12:03:56,439 INFO 1 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:105 - Backing off _make_request(...) for 1.8s (requests.exceptions.ConnectionError: HTTPConnectionPool(host='local.edly.io', port=80): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery&active_only=True (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x71d310b59520>: Failed to establish a new connection: [Errno 111] Connection refused')))
2024-06-27 12:03:58,225 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:103 - Requesting course run page 1...
2024-06-27 12:03:58,227 INFO 1 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:105 - Backing off _make_request(...) for 68.4s (requests.exceptions.ConnectionError: HTTPConnectionPool(host='local.edly.io', port=80): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery&active_only=True (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x71d310b59be0>: Failed to establish a new connection: [Errno 111] Connection refused')))

tried this command also

tutor dev run discovery ./manage.py update_index --disable-change-limit

tutor dev run lms ./manage.py lms create_catalog_integrations --enabled \
    --internal_api_url="" \
    --service_username=lms_catalog_service_user

tutor dev run lms ./manage.py lms cache_programs --domain="local.edly.io:8000"

my program is not showing on programs tab and also giving me 404

tutor dev run lms ./manage.py lms cache_programs --domain="local.edly.io:8000"
docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms cache_programs --domain=local.edly.io:8000
WARN[0000] /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml: `version` is obsolete 
WARN[0000] /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml: `version` is obsolete 
[+] Creating 6/0
 ✔ Container tutor_dev-mysql-1          Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-smtp-1           Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-permissions-1    Created                                                                                                                                    0.0s 
 ✔ Container tutor_dev-elasticsearch-1  Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-mongodb-1        Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-redis-1          Running                                                                                                                                    0.0s 
[+] Running 1/1
 ✔ Container tutor_dev-permissions-1  Started                                                                                                                                      0.3s 
2024-06-27 12:11:09,474 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:60 - populate-multitenant-programs switch is ON
2024-06-27 12:11:09,505 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:161 - Requesting program UUIDs for local.edly.io:8000.
2024-06-27 12:11:09,531 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:169 - Received 0 UUIDs for site local.edly.io:8000
2024-06-27 12:11:09,531 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:201 - Requesting pathways for local.edly.io:8000.
2024-06-27 12:11:09,588 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:212 - Failed to retrieve pathways for site: local.edly.io:8000.
Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py", line 207, in get_pathways
    response.raise_for_status()
  File "/openedx/venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://discovery.local.edly.io:8381/api/v1/pathways/?exclude_utm=1&page=1
2024-06-27 12:11:09,588 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:217 - Received 0 pathways for site local.edly.io:8000
2024-06-27 12:11:09,588 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:114 - Caching UUIDs for 0 programs for site local.edly.io:8000.
2024-06-27 12:11:09,589 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:121 - Caching ids for 0 pathways for site local.edly.io:8000.
2024-06-27 12:11:09,590 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:127 - Caching details for 0 programs.
2024-06-27 12:11:09,590 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:130 - Caching details for 0 pathways.
2024-06-27 12:11:09,590 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:133 - Caching programs uuids for 0 courses.
2024-06-27 12:11:09,591 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:136 - Caching programs uuids for 0 catalog courses.
2024-06-27 12:11:09,591 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:139 - Caching program UUIDs by 0 program types.
2024-06-27 12:11:09,591 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:142 - Caching program UUIDs by 0 program type slugs.
2024-06-27 12:11:09,591 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:145 - Caching programs uuids for 0 organizations
Error: Command failed with status 1: docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms cache_programs --domain=local.edly.io:8000

Screenshot from 2024-06-27 17-45-50 Screenshot from 2024-06-27 17-52-57

I think this is a bug or something, a few months ago I added the program, so it won't show up in the MFE programming tab, so I guess I need to post here thank you :)

Yagnesh1998 commented 4 months ago

when i hit this command tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=openedx The following problem is occurring and does not stop

WARNINGS:
course_metadata.Course.collaborators: (fields.W340) null has no effect on ManyToManyField.
course_metadata.Degree.specializations: (fields.W340) null has no effect on ManyToManyField.
course_metadata.SearchDefaultResultsConfiguration.courses: (fields.W340) null has no effect on ManyToManyField.
course_metadata.SearchDefaultResultsConfiguration.programs: (fields.W340) null has no effect on ManyToManyField.
taxonomy.JobPostings.job: (fields.W342) Setting unique=True on a ForeignKey has the same effect as using a OneToOneField.
    HINT: ForeignKey(unique=True) is usually better served by a OneToOneField.
taxonomy.SkillsQuiz.skills: (fields.W340) null has no effect on ManyToManyField.
2024-06-27 12:03:56,433 INFO 1 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:134 - Command is not using threads to write data.
2024-06-27 12:03:56,434 INFO 1 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:178 - Executing Loader CoursesApiDataLoader, url: http://local.edly.io/api/courses/v1/
2024-06-27 12:03:56,436 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:57 - Refreshing Courses and CourseRuns from http://local.edly.io/api/courses/v1/...
2024-06-27 12:03:56,436 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:103 - Requesting course run page 1...
2024-06-27 12:03:56,439 INFO 1 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:105 - Backing off _make_request(...) for 1.8s (requests.exceptions.ConnectionError: HTTPConnectionPool(host='local.edly.io', port=80): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery&active_only=True (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x71d310b59520>: Failed to establish a new connection: [Errno 111] Connection refused')))
2024-06-27 12:03:58,225 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:103 - Requesting course run page 1...
2024-06-27 12:03:58,227 INFO 1 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:105 - Bac
Faraz32123 commented 4 months ago

Hello @Yagnesh1998, Did you tried adding marketing path equal to /programs in ProgramsApiConfig model and enabling it, in LMS admin panel as mentioned here.

Yagnesh1998 commented 4 months ago

@Faraz32123 I receive a 'Page Not Found' error when I navigate to Programs and click on the 'Explore Program' button and encountering connection errors as well on Terminal.

Screenshot 2024-07-01 at 8 54 04 PM
Faraz32123 commented 4 months ago

Hello @Yagnesh1998,

when i hit this command tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=openedx The following problem is occurring and does not stop

The problem is that you are running tutor in development mode and you are using partner_code=openedx. Please use partner_code=dev like tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=dev.

@Faraz32123 I receive a 'Page Not Found' error when I navigate to Programs and click on the 'Explore Program' button and encountering connection errors as well on Terminal.

Actually this explore programs link doesn't exists, you have to add your custom link here for your all programs catalog as mentioned here. And for enrolled programs to show in programs tab, if you are enrolled in any of the course in a program, that program will show here in the program tab.

Yagnesh1998 commented 4 months ago

@Faraz32123 please check this

tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=dev

tutor dev run lms ./manage.py lms create_catalog_integrations --enabled \
    --internal_api_url=http://discovery:8000/api/v1 \
    --service_username=lms_catalog_service_user

tutor dev run lms ./manage.py lms cache_programs

(demo) yagnesh@yagnesh:~/demo-tutor$ tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=dev
docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm discovery ./manage.py refresh_course_metadata --partner_code=dev
WARN[0000] /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml: `version` is obsolete 
WARN[0000] /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml: `version` is obsolete 
[+] Creating 7/0
 ✔ Container tutor_dev-mysql-1          Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-smtp-1           Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-permissions-1    Created                                                                                                                                    0.0s 
 ✔ Container tutor_dev-mongodb-1        Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-elasticsearch-1  Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-redis-1          Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-lms-1            Running                                                                                                                                    0.0s 
[+] Running 1/1
 ✔ Container tutor_dev-permissions-1  Started                                                                                                                                      0.3s 
System check identified some issues:

WARNINGS:
course_metadata.Course.collaborators: (fields.W340) null has no effect on ManyToManyField.
course_metadata.Degree.specializations: (fields.W340) null has no effect on ManyToManyField.
course_metadata.SearchDefaultResultsConfiguration.courses: (fields.W340) null has no effect on ManyToManyField.
course_metadata.SearchDefaultResultsConfiguration.programs: (fields.W340) null has no effect on ManyToManyField.
taxonomy.JobPostings.job: (fields.W342) Setting unique=True on a ForeignKey has the same effect as using a OneToOneField.
    HINT: ForeignKey(unique=True) is usually better served by a OneToOneField.
taxonomy.SkillsQuiz.skills: (fields.W340) null has no effect on ManyToManyField.
2024-07-02 10:22:01,900 INFO 1 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:134 - Command is not using threads to write data.
2024-07-02 10:22:01,901 INFO 1 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:178 - Executing Loader CoursesApiDataLoader, url: http://local.edly.io:8000/api/courses/v1/
2024-07-02 10:22:01,905 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:57 - Refreshing Courses and CourseRuns from http://local.edly.io:8000/api/courses/v1/...
2024-07-02 10:22:01,905 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:103 - Requesting course run page 1...
2024-07-02 10:22:02,121 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:111 - Retrieved 4 course runs...
2024-07-02 10:22:02,121 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:119 - Starting course processing for id course-v1:abm+demo89+2024_y7
2024-07-02 10:22:02,129 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:180 - Processed course run with UUID [2ed5b158-acb0-42a4-92c3-480c9984fb5e] and key [course-v1:abm+demo89+2024_y7].
2024-07-02 10:22:02,131 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:119 - Starting course processing for id course-v1:DEMO+EDX2023+EDX2023
2024-07-02 10:22:02,134 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:180 - Processed course run with UUID [46504da3-8a82-49dc-a5a2-0d93bf0e06aa] and key [course-v1:DEMO+EDX2023+EDX2023].
2024-07-02 10:22:02,136 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:119 - Starting course processing for id course-v1:edX+DemoX+Demo_Course
2024-07-02 10:22:02,138 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:180 - Processed course run with UUID [83719e47-6e35-4e93-a97a-09955d9b3cbb] and key [course-v1:edX+DemoX+Demo_Course].
2024-07-02 10:22:02,140 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:119 - Starting course processing for id course-v1:openedx+demo12+dem1
2024-07-02 10:22:02,142 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:180 - Processed course run with UUID [70220d79-5c5f-42df-b500-930078944058] and key [course-v1:openedx+demo12+dem1].
2024-07-02 10:22:02,143 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:66 - Looping to request all 1 pages...
2024-07-02 10:22:02,143 INFO 1 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:83 - Retrieved 4 course runs from http://local.edly.io:8000/api/courses/v1/.
(demo) yagnesh@yagnesh:~/demo-tutor$ tutor dev run lms ./manage.py lms create_catalog_integrations --enabled \
    --internal_api_url=http://discovery:8000/api/v1 \
    --service_username=lms_catalog_service_user
docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms create_catalog_integrations --enabled --internal_api_url=http://discovery:8000/api/v1 --service_username=lms_catalog_service_user
WARN[0000] /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml: `version` is obsolete 
WARN[0000] /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml: `version` is obsolete 
[+] Creating 6/0
 ✔ Container tutor_dev-smtp-1           Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-permissions-1    Created                                                                                                                                    0.0s 
 ✔ Container tutor_dev-elasticsearch-1  Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-redis-1          Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-mongodb-1        Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-mysql-1          Running                                                                                                                                    0.0s 
[+] Running 1/1
 ✔ Container tutor_dev-permissions-1  Started                                                                                                                                      0.3s 
Successfully created CatalogIntegration enabled=True url=http://discovery:8000/api/v1 service_username=lms_catalog_service_user
(demo) yagnesh@yagnesh:~/demo-tutor$ tutor dev run lms ./manage.py lms cache_programs
docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms cache_programs
WARN[0000] /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml: `version` is obsolete 
WARN[0000] /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml: `version` is obsolete 
[+] Creating 6/0
 ✔ Container tutor_dev-permissions-1    Created                                                                                                                                    0.0s 
 ✔ Container tutor_dev-elasticsearch-1  Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-mysql-1          Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-smtp-1           Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-redis-1          Running                                                                                                                                    0.0s 
 ✔ Container tutor_dev-mongodb-1        Running                                                                                                                                    0.0s 
[+] Running 1/1
 ✔ Container tutor_dev-permissions-1  Started                                                                                                                                      0.3s 
2024-07-02 10:22:33,656 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:60 - populate-multitenant-programs switch is ON
2024-07-02 10:22:33,669 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:85 - Skipping site example.com. No configuration.
2024-07-02 10:22:33,676 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:161 - Requesting program UUIDs for local.edly.io.
2024-07-02 10:22:33,678 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:166 - Failed to retrieve program UUIDs for site: local.edly.io.
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 415, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7985df78c670>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 798, in urlopen
    retries = retries.increment(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='discovery.local.edly.io', port=80): Max retries exceeded with url: /api/v1/programs/?exclude_utm=1&status=active&status=retired&uuids_only=1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7985df78c670>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py", line 162, in get_site_program_uuids
    response = client.get(api_url, params=querystring)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='discovery.local.edly.io', port=80): Max retries exceeded with url: /api/v1/programs/?exclude_utm=1&status=active&status=retired&uuids_only=1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7985df78c670>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-07-02 10:22:33,682 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:169 - Received 0 UUIDs for site local.edly.io
2024-07-02 10:22:33,682 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:201 - Requesting pathways for local.edly.io.
2024-07-02 10:22:33,684 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:212 - Failed to retrieve pathways for site: local.edly.io.
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 415, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/opt/pyenv/versions/3.8.18/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7985df78cb50>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 798, in urlopen
    retries = retries.increment(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='discovery.local.edly.io', port=80): Max retries exceeded with url: /api/v1/pathways/?exclude_utm=1&page=1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7985df78cb50>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py", line 206, in get_pathways
    response = client.get(api_url, params=dict(exclude_utm=1, page=next_page))
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='discovery.local.edly.io', port=80): Max retries exceeded with url: /api/v1/pathways/?exclude_utm=1&page=1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7985df78cb50>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-07-02 10:22:33,684 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:217 - Received 0 pathways for site local.edly.io
2024-07-02 10:22:33,684 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:114 - Caching UUIDs for 0 programs for site local.edly.io.
2024-07-02 10:22:33,685 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:121 - Caching ids for 0 pathways for site local.edly.io.
2024-07-02 10:22:33,690 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:161 - Requesting program UUIDs for local.edly.io:8000.
2024-07-02 10:22:33,728 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:169 - Received 0 UUIDs for site local.edly.io:8000
2024-07-02 10:22:33,728 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:201 - Requesting pathways for local.edly.io:8000.
2024-07-02 10:22:33,783 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:212 - Failed to retrieve pathways for site: local.edly.io:8000.
Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py", line 207, in get_pathways
    response.raise_for_status()
  File "/openedx/venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://discovery.local.edly.io:8381/api/v1/pathways/?exclude_utm=1&page=1
2024-07-02 10:22:33,784 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:217 - Received 0 pathways for site local.edly.io:8000
2024-07-02 10:22:33,784 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:114 - Caching UUIDs for 0 programs for site local.edly.io:8000.
2024-07-02 10:22:33,785 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:121 - Caching ids for 0 pathways for site local.edly.io:8000.
2024-07-02 10:22:33,788 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:85 - Skipping site preview.local.edly.io. No configuration.
2024-07-02 10:22:33,791 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:85 - Skipping site preview.local.edly.io:8000. No configuration.
2024-07-02 10:22:33,795 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:85 - Skipping site studio.local.edly.io. No configuration.
2024-07-02 10:22:33,799 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:85 - Skipping site studio.local.edly.io:8001. No configuration.
2024-07-02 10:22:33,800 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:127 - Caching details for 0 programs.
2024-07-02 10:22:33,801 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:130 - Caching details for 0 pathways.
2024-07-02 10:22:33,801 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:133 - Caching programs uuids for 0 courses.
2024-07-02 10:22:33,801 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:136 - Caching programs uuids for 0 catalog courses.
2024-07-02 10:22:33,801 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:139 - Caching program UUIDs by 0 program types.
2024-07-02 10:22:33,801 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:142 - Caching program UUIDs by 0 program type slugs.
2024-07-02 10:22:33,802 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:145 - Caching programs uuids for 0 organizations
Error: Command failed with status 1: docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms cache_programs
(demo) yagnesh@yagnesh:~/demo-tutor$ tutor dev run lms ./manage.py lms cache_programs
Faraz32123 commented 4 months ago

Please try tutor dev run lms ./manage.py lms cache_programs --domain="local.edly.io:8000" as mentioned here. You are trying to cache programs from all sites.

Yagnesh1998 commented 4 months ago

@Faraz32123 

tutor dev run discovery ./manage.py refresh_course_metadata --partner_code=dev

tutor dev run lms ./manage.py lms create_catalog_integrations --enabled \
    --internal_api_url=http://discovery:8000/api/v1 \
    --service_username=lms_catalog_service_user

tutor dev run lms ./manage.py lms cache_programs --domain="local.edly.io:8000"
2024-07-02 10:48:34,185 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:212 - Failed to retrieve pathways for site: local.edly.io:8000.
Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py", line 207, in get_pathways
    response.raise_for_status()
  File "/openedx/venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://discovery.local.edly.io:8381/api/v1/pathways/?exclude_utm=1&page=1
2024-07-02 10:48:34,187 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:217 - Received 0 pathways for site local.edly.io:8000
2024-07-02 10:48:34,187 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:114 - Caching UUIDs for 0 programs for site local.edly.io:8000.
2024-07-02 10:48:34,188 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:121 - Caching ids for 0 pathways for site local.edly.io:8000.
2024-07-02 10:48:34,189 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:127 - Caching details for 0 programs.
2024-07-02 10:48:34,189 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:130 - Caching details for 0 pathways.
2024-07-02 10:48:34,190 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:133 - Caching programs uuids for 0 courses.
2024-07-02 10:48:34,190 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:136 - Caching programs uuids for 0 catalog courses.
2024-07-02 10:48:34,190 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:139 - Caching program UUIDs by 0 program types.
2024-07-02 10:48:34,191 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:142 - Caching program UUIDs by 0 program type slugs.
2024-07-02 10:48:34,191 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:145 - Caching programs uuids for 0 organizations
Error: Command failed with status 1: docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms cache_programs --domain=local.edly.io:8000
Faraz32123 commented 4 months ago

Caching UUIDs for 0 programs for site local.edly.io:8000.

please check the program that you had created in discovery and also verify the courses in that program if those are not archived or expired. Courses should be active.

a few months ago I added the program

If that's the same program, may be you can try creating a new program with new courses.

Yagnesh1998 commented 4 months ago

is it ok? @Faraz32123 Screenshot from 2024-07-02 16-50-51

Faraz32123 commented 4 months ago

Also verify the courses in this program.

Yagnesh1998 commented 4 months ago

Same error as above but the program is showing. thank you @Faraz32123 

2024-07-02 10:48:34,185 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:212 - Failed to retrieve pathways for site: local.edly.io:8000.
Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py", line 207, in get_pathways
    response.raise_for_status()
  File "/openedx/venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://discovery.local.edly.io:8381/api/v1/pathways/?exclude_utm=1&page=1
2024-07-02 10:48:34,187 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:217 - Received 0 pathways for site local.edly.io:8000
2024-07-02 10:48:34,187 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:114 - Caching UUIDs for 0 programs for site local.edly.io:8000.
2024-07-02 10:48:34,188 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:121 - Caching ids for 0 pathways for site local.edly.io:8000.
2024-07-02 10:48:34,189 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:127 - Caching details for 0 programs.
2024-07-02 10:48:34,189 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:130 - Caching details for 0 pathways.
2024-07-02 10:48:34,190 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:133 - Caching programs uuids for 0 courses.
2024-07-02 10:48:34,190 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:136 - Caching programs uuids for 0 catalog courses.
2024-07-02 10:48:34,190 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:139 - Caching program UUIDs by 0 program types.
2024-07-02 10:48:34,191 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:142 - Caching program UUIDs by 0 program type slugs.
2024-07-02 10:48:34,191 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] [ip None] cache_programs.py:145 - Caching programs uuids for 0 organizations
Error: Command failed with status 1: docker compose -f /home/yagnesh/.local/share/tutor/env/local/docker-compose.yml -f /home/yagnesh/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev run --rm lms ./manage.py lms cache_programs --domain=local.edly.io:8000

Screenshot from 2024-07-02 17-24-12

Faraz32123 commented 4 months ago

Closing this issue, as the programs are showing now.

Please don't use Github issues for feature requests or for support. Instead, you should create a new topic in the official Open edX forum: https://discuss.openedx.org/

The process for troubleshooting an existing installation and for contributing to the platform is described in the docs: https://docs.tutor.overhang.io/troubleshooting.html https://docs.tutor.overhang.io/tutor.html#contributing