Integrate logging system into Django

[mehalyna]

Task Description:

We need to integrate a logging system into the Django project to track and monitor events, errors, and general runtime behavior. The logging system will help identify and troubleshoot issues by capturing important events, such as database queries, view processing, security issues, and application errors.

---

Objectives:

1. Configure Logging in settings.py:

   • Add a logging configuration that will output logs to both the console and a file.
   • Configure different log levels (e.g., DEBUG, INFO, WARNING, ERROR, CRITICAL).
   • Ensure logs are generated for database queries, request-response cycles, errors, and security events.
   • Ensure the logging captures issues across the entire project (including all apps: users, profiles, projects, communications, dashboard).

2. Log Errors in Views:

   • Add logging to the views across different apps (e.g., users/views.py, profiles/views.py, projects/views.py, etc.).
   • Ensure that critical errors are logged appropriately in case of exceptions (e.g., try/except blocks).

3. Log Security Events:

   • Add logging in permissions.py (in users/) to capture security-related events, such as permission denials.

4. Capture Database Events:

   • Ensure that the logging system captures significant database operations for monitoring queries.

---

Steps to Implement:

1. Step 1: Modify settings.py to include the logging configuration:

   • Add a LOGGING dictionary in projectname/settings.py that configures loggers, handlers, and formatters.
   • Set up log rotation for file logging (e.g., using TimedRotatingFileHandler).

   Example:

   # settings.py
   
   LOGGING = {
      'version': 1,
      'disable_existing_loggers': False,
      'formatters': {
          'verbose': {
              'format': '{asctime} {levelname} {name} {message}',
              'style': '{',
          },
          'simple': {
              'format': '{levelname} {message}',
              'style': '{',
          },
      },
      'handlers': {
          'console': {
              'level': 'DEBUG',
              'class': 'logging.StreamHandler',
              'formatter': 'simple',
          },
          'file': {
              'level': 'WARNING',
              'class': 'logging.handlers.TimedRotatingFileHandler',
              'filename': 'logs/projectname.log',
              'when': 'midnight',
              'backupCount': 7,
              'formatter': 'verbose',
          },
      },
      'loggers': {
          'django': {
              'handlers': ['console', 'file'],
              'level': 'DEBUG',
          },
          'django.db.backends': {
              'handlers': ['console', 'file'],
              'level': 'INFO',
              'propagate': False,
          },
          'myapp': {
              'handlers': ['console', 'file'],
              'level': 'DEBUG',
              'propagate': True,
          },
      },
   }

2. Step 2: Add logging to views in each app:

   • In each app’s views.py, import Python’s logging module and add logging calls.

   • For example:

     import logging
     
     logger = logging.getLogger(__name__)
     
     def example_view(request):
      try:
          # Some processing
          logger.info("Processing the request.")
      except Exception as e:
          logger.error(f"Error occurred: {e}")

3. Step 3: Add logging for permission issues in permissions.py:

   • Add logging where permissions are denied or other security-related events occur.

   Example:

   import logging
   from rest_framework.permissions import BasePermission
   
   logger = logging.getLogger(__name__)
   
   class CustomPermission(BasePermission):
      def has_permission(self, request, view):
          if some_condition_fails:
              logger.warning(f"Permission denied for user {request.user}")
              return False
          return True

4. Step 4: Ensure logs are captured for database operations:

   • The database queries can be logged by enabling logging for the django.db.backends logger in the LOGGING configuration.

---

Deliverables:

1. Logging Configuration:

   • A fully working logging configuration in settings.py that outputs logs to both console and file with appropriate log rotation.

2. Log Implementation in Views:

   • Ensure that views in users/views.py, profiles/views.py, projects/views.py, etc. have logging for critical operations and exceptions.

3. Log Implementation for Permissions:

   • Add logging in users/permissions.py to track permission denials or security issues.

4. Database Logging:

   • Ensure logs are generated for database operations and are monitored.

---

Testing:

• Test the logging by triggering different events in the project (e.g., successful requests, errors, database queries, permission issues).
• Verify that logs are being written to the console and the log file (logs/projectname.log).
• Ensure that logs contain useful information, including timestamps, log level, and relevant details about the events.

---

Notes:

• Make sure to exclude sensitive data from logs (e.g., user passwords, tokens).
• Ensure the logging setup is scalable and won’t cause performance bottlenecks.