microsoft / onefuzz

A self-hosted Fuzzing-As-A-Service platform
MIT License
2.82k stars 199 forks source link

Improve slow Task queries #3594

Closed Porges closed 1 year ago

Porges commented 1 year ago

Querying Tasks only by TaskID is slow as it requires querying many partitions. Querying Task by JobID and TaskID is fast as it is a point lookup.

  1. rename existing GetTaskById to GetTaskByIdSlow
  2. include JobID in event data from agent (new properties are optional for back-compat)
  3. include JobID in NodeTask entity (if this is missing upon lookup we will fall back to GetTaskByIdSlow)
codecov-commenter commented 1 year ago

Codecov Report

Merging #3594 (71cb0c3) into main (ca3c503) will increase coverage by 0.01%. The diff coverage is 44.64%.

@@            Coverage Diff             @@
##             main    #3594      +/-   ##
==========================================
+ Coverage   39.96%   39.97%   +0.01%     
==========================================
  Files         303      303              
  Lines       37635    37698      +63     
  Branches     1721     1725       +4     
==========================================
+ Hits        15040    15070      +30     
- Misses      22276    22307      +31     
- Partials      319      321       +2     
Files Coverage Δ
src/ApiService/ApiService/OneFuzzTypes/Model.cs 69.06% <100.00%> (+0.03%) :arrow_up:
src/agent/onefuzz-agent/src/agent.rs 58.05% <100.00%> (+1.66%) :arrow_up:
src/agent/onefuzz-agent/src/agent/tests.rs 100.00% <100.00%> (ø)
src/agent/onefuzz-agent/src/work.rs 16.66% <ø> (-1.86%) :arrow_down:
src/agent/onefuzz-agent/src/worker.rs 48.25% <100.00%> (+0.27%) :arrow_up:
src/agent/onefuzz-agent/src/worker/tests.rs 99.39% <100.00%> (+<0.01%) :arrow_up:
.../ApiService/ApiService/Functions/QueueJobResult.cs 0.00% <0.00%> (ø)
.../ApiService/ApiService/Functions/TimerRetention.cs 0.00% <0.00%> (ø)
src/ApiService/ApiService/OneFuzzTypes/Requests.cs 59.62% <87.50%> (+0.70%) :arrow_up:
...ApiService/ApiService/onefuzzlib/TaskOperations.cs 28.40% <50.00%> (+0.16%) :arrow_up:
... and 9 more
Porges commented 1 year ago

Passed check-pr.