GhostManager / Ghostwriter

The SpecterOps project management and reporting engine
https://ghostwriter.wiki
BSD 3-Clause "New" or "Revised" License
1.23k stars 174 forks source link

Release domains task not working #451

Closed domwhewell-sage closed 2 weeks ago

domwhewell-sage commented 3 weeks ago

Describe the bug I have the release domains task scheduled to run daily, but a checked out domain for a project that has completed is not being released. Looking at the task status it says it was successful and this is the result {'errors': {}, 16: {'domain': '<redacted>', 'release_date': datetime.date(2024, 5, 17), 'change': 'released'}} However this is it in ghostwriter image These are the logs from the latest run in django-queue

10:30:13 [Q] INFO Process-1 created a task from schedule [Release Checked out Domains]
INFO 2024-06-09 10:30:13,078 tasks 19 139795110296392 Starting domain release task at 2024-06-09 10:30:13.078433
WARNING 2024-06-09 10:30:13,088 tasks 19 139795110296392 The domain <redacted> is marked for release
INFO 2024-06-09 10:30:13,088 tasks 19 139795110296392 Releasing <redacted> back into the pool.
10:30:13 [Q] INFO Processed [pennsylvania-river-helium-autumn]
INFO 2024-06-09 10:30:14,326 tasks 19 139795110296392 Domain release task completed at 2024-06-09 10:30:14.326298

To Reproduce Steps to reproduce the behavior:

  1. Schedule a release domains task to run daily
  2. Check out a domain for a project
  3. Complete the project and let the end date pass
  4. See the release domains task run with no errors yet the domain status does not change to Available

Expected Behavior Domain is changed to available

chrismaddalena commented 3 weeks ago

Hey there, I'm unable to reproduce this. The domains we have checked out have been releasing as expected. I also tried it with our latest release in my development environment and the task flagged the checked out domains for release and their status flipped to "Available."

Have you made any custom changes to the task or the domain status?

The status change happens between the log message "Releasing back into the pool" and updating the JSON results with 'change': 'released' for the affected domain. You're seeing the log message and the changed key set with no errors in-between, so the domain status should be set correctly.

Jut in case, I have to ask to rule it out, did you refresh the page after seeing the task complete? The status won't update on the domain dashboard or in the list of domains until you refresh after the task completes.

domwhewell-sage commented 2 weeks ago

Hi @chrismaddalena thank you for your reply. I have not made any modifications to the tasks apart from modifications to the container for the nmap fix as described here https://github.com/GhostManager/Ghostwriter/pull/362.

I have been looking at the logs the past couple of days and the same message appears each day at the scheduled time. I have been refreshing the page when the task runs however I have noticed today that it appears available once the task has run image

And then appears checked out again moments later image

So I think what is happening is the release task it successfully making the domain available and another task is potentially changing it back to checked-out again image

chrismaddalena commented 2 weeks ago

That could explain it. Since all the tasks in the screenshot ran at the same time, it may be that Domain Monitor causes the status to "reset" after Release Checked out Domains flips it.

When the task pulls a queryset or record, those don't update automatically if the record(s) change between the task pulling the record and the end. The sequence might be something like:

  1. Both tasks start and pull a domain marked as Unavailable
  2. The Release Checked out Domains tasks releases the domain by marking it as Available and saving the record
  3. The Domain Monitor task takes longer so it completes after the release task and updates the domain record
  4. The Domain Monitor task's copy of the domain record had the status as Unavailable so the status flips back when the task saves the record

The easiest solution is running the tasks at different times, but we can probably do some things to avoid this sort of collision. If you expect a record may have changed, you can call refresh_from_db() to make sure the record is up-to-date. We can also update on the changed fields instead of calling save(). We'll look into adjusting the task for that.