search

NOAA-OWP / wres

Code and scripts for the Water Resources Evaluation Service
Other
2 stars 1 forks source link

As a developer, I want the workers to handle preparing a declaration that uses posted data #159

Open epag opened 2 months ago

epag commented 2 months ago

Author Name: Hank (Hank) Original Redmine Issue: 125870, https://vlab.noaa.gov/redmine/issues/125870 Original Date: 2024-01-26

See #125836-4 and following comments. The recommendation is for the tasker to hand off the declaration and paths to the worker, and for the worker-shim/worker to prepare the final declaration.

Leaving it in the backlog with normal priority. I don't think anyone is clamoring for this soon.

Hank

epag commented 2 months ago

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2024-01-26T14:16:48Z

Clarified description,

Hank

epag commented 2 months ago

Original Redmine Comment Author Name: Hank (Hank) Original Date: 2024-01-26T14:19:10Z

Evan mentioned this:

I think to James point, I would much rather support spinning up new Validator containers that just deal with the validation of an evaluation if we want that to be done really quickly and not have to wait for a worker but that is much more work. I think there should be clear separation of duties, especially if we want to continue to progress towards a distributed microservice architecture

That might be a reasonable approach. It would make the service architecture more complex, but would at least allow for quick feedback if the declaration is bad.

Hank

epag commented 2 months ago

Original Redmine Comment Author Name: James (James) Original Date: 2024-01-26T14:38:54Z

Again, I just don't think performance is the issue right now. Let's address design/correctness/accuracy first, performance if/when really needed, which is probably never. If fast-failing evaluations are consistently waiting for workers, that is a good problem to have because it means we have some actual users to help.