search

dotCMS / core

Headless/Hybrid Content Management System for Enterprises
http://dotcms.com
Other
860 stars 467 forks source link

CLI: 404 Errors When Pulling Content After Deleting Files with --removeAssets Flag #27985

Closed jgambarios closed 6 months ago

jgambarios commented 7 months ago

Problem Statement

Deleting content using the CLI with the --removeAssets flag, and then pulling content, results in 404 errors. It appears that we are not excluding the removed assets (which are now archived on the server), leading to the 404 errors.

Found [9] errors during the pull process:

[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/custom-fields/og-preview.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/custom-fields/og-preview.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/url-title.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/url-title.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/compare_auth_prod.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/compare_auth_prod.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/documentation_preview_prod.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/documentation_preview_pro...
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/whatsnew.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/whatsnew.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/older-releases.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/older-releases.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/eol-chart.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/eol-chart.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/code_share_detail.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/code_share_detail.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
[ERROR] ❗  Error pulling file [//dotcms.dev/application/vtl/docs/documentation_toc_grid.vtl] to [/Users/jonathan/Downloads/CLI/files/working/en-us/dotcms.dev/application/vtl/docs/documentation_toc_grid.vtl]
 HTTP 404 Not Found: The requested resource was not found or is restricted.
run with -e or --errors for full details on the exception.

Steps to Reproduce

  1. Delete files using the CLI using the push command and the --removeAssets flag.
  2. Pull

Acceptance Criteria

Archived contents should be ignored, and no error should be thrown.

Proposed Objective

Core Features

Proposed Priority

Priority 2 - Important

jgambarios commented 7 months ago

On this ticket, the getContentUnderParentFromDB and selectQuery methods in BrowserAPIImpl class have been refactored to remove the unnecessary dependency with Elasticsearch, leaving only the DB lookup.

For reference, the removed ES changes were introduced based on https://github.com/dotCMS/core/issues/23449#issuecomment-1456592701 and in this commit https://github.com/dotCMS/core/commit/a3030c6a7da9d72c46122c08c3c6b0e0853dea67

nollymar commented 7 months ago

Note to QA: Please test again this card #23449

fabrizzio-dotCMS commented 6 months ago

I tested this against a local copy of our authoring server (where the issue was originally found). Before testing a full reindex was run. My test was:

  1. removing the assets locally (not the folder containing the items)
  2. then running a pull operation passing the --removeAssets option
  3. then followed by another pull operation.

I did this always trying to pull/push several files at once I did it for live and working items.

Also tried this using the --removeFolder option

in none of the attempts I got a 404 error like the one described above. I also verified this card as requested

bryanboza commented 6 months ago

Fixed, tested following the provided steps and I'm unable to reproduce