42atomys / stud42

Stud42 official repository since major 3 update (https://s42.app)
https://s42.app
MIT License
862 stars 27 forks source link

fix: mismatch between number of campuses in database and API resulting to an error #516

Open 42atomys opened 1 year ago

42atomys commented 1 year ago

Describe the bug

An error generated: campus not found is raised during the execution of the webhooks-processor. This error is undesirable as we crawl all campuses each week. Upon investigation, it was discovered that there is a mismatch between the number of campuses in our production database and the number returned by the API. The production database contains 51 campuses, while the API returns 69. The cause of this mismatch is currently unknown.

Investigating and resolving this mismatch is crucial to ensure the accuracy and reliability of the data in our system. The mismatch leads to the campus not found error during the webhooks-processor, which could potentially impact other functionalities.

To Reproduce

  1. Run the command go run main.go jobs crawler campus
  2. Observe the discrepancy between the number of campuses returned by the API and the number inserted into the database.

Expected behavior

The data between the API and our database should match.

Relevant log output

10:25PM INF Start the crawling of all campus of 42 network version=v0.30
10:25PM DBG Requesting https://api.intra.42.fr/v2/campus?per_page=100 version=v0.30

... TRUNC DATA DUE TO PRIVATE INFORMATION ...

10:37PM INF Successfully imported 51 campus version=v0.30

Version of software

v0.30

Environment

Live (https://s42.app)

Additional context

Error Message: generated: campus not found Link to Discord message from monitoring

Code of Conduct

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 28 days with no activity.

42atomys commented 1 year ago

Relative slack message https://42born2code.slack.com/archives/C0FEKP3P1/p1698092483815449