search

dotCMS / core

Headless/Hybrid Content Management System for Enterprises
http://dotcms.com
Other
864 stars 467 forks source link

New REST Endpoints for Content Export Job Management #30559

Open jgambarios opened 3 weeks ago

jgambarios commented 3 weeks ago

Parent Issue

https://github.com/dotCMS/core/issues/29575

Task

We need to create REST endpoints to interact with the new ExportContentletProcessor to manage content export operations through the job queue system.

Objectives

REST Endpoints Specification

  1. Create Export Job

    • Endpoint: POST /content/export
    • Purpose: Creates and enqueues a new content export job
    • Returns: Job identifier

  2. List Export Jobs

    • Endpoint: GET /content/export
    • Parameters:
    • page: Page number
    • size: Items per page
    • status: Filter by job status
    • Purpose: Lists all enqueued export jobs with pagination

  3. Get Job Status

    • Endpoint: GET /content/export/{jobId}
    • Parameters: jobId
    • Returns:
    • Job state
    • Progress percentage
    • Executing node
    • Download URL (when completed)

  4. Cancel Job

    • Endpoint: POST /content/export/{jobId}/cancel
    • Parameters: jobId
    • Purpose: Cancels a running export job

  5. Monitor Job

    • Endpoint: GET /content/export/{jobId}/monitor
    • Parameters: jobId
    • Type: Server-Sent Events (SSE)
    • Purpose: Real-time job status monitoring

  6. Get Export File

    • Endpoint: GET /content/export/{jobId}/download
    • Parameters: jobId
    • Purpose: Downloads the generated export file
    • Security: Requires valid authentication and authorization

Technical Requirements

Export Features:

Job Queue Implementation:

Error Handling:

Acceptance Criteria

Proposed Objective

Core Features

Proposed Priority

Priority 2 - Important

fmontes commented 2 weeks ago

Review Points for Endpoint Implementation

  1. Error Handling: Since we aim for a fail-continue option, what is our plan for error handling?

  2. Authorization: Can we confirm this alignment with our current permissions model?

  3. Job Cleanup Policies: Cleanup was noted, but can we specify how this will be managed?

Additional Clarifications Needed

  1. Notifications Integration: How do we plan to integrate system notifications for job completions or status updates?

  2. Download Link Expiry: Will the generated download links for completed jobs have an expiration policy? If so, could we detail how this will be managed?

jgambarios commented 2 weeks ago

@fmontes

  1. Error Handling: The error handling including the use of the "fail-continue" option will be handled in https://github.com/dotCMS/core/issues/30557, this new rest endpoint will just pass the options to the job and respond with the information generated by the new ExportContentletProcessor.

  2. Authorization: The user will be able to export only contents he has access to, the user requesting the export will be passed to the job in order to execute all the required logic with that user allowing us to use the permissions infrastructure.

  3. Job Cleanup Policies: As we are generating files for download, we should have a process, based on some configurable retention policy, where we delete "old" exports.

Additional Clarifications Needed

  1. Notifications Integration: The new job infrastructure already handles status updates, we can monitor any status change, and, on completion, we should use dotCMS system notifications to notify the user, in the admin UI, the export finished and if finished successfully, including the download link.

  2. Download Link Expiry: We should have a configurable retention policy (time) that will be use, by a cleanup process, to delete "expired" exports.