search

harlam357 / hfm-net

Client Monitoring Application for Folding@Home
34 stars 6 forks source link

Few Issues When Creating Custome Data View In Work Unit History #337

Open PantherX opened 3 years ago

PantherX commented 3 years ago

Your Environment

Expected Behavior

When using the Work Unit History, I am making custom Data view where:

  1. I would like to see historical data for 2 or more Projects based on the Project numbers.
  2. See all WUs that didn't successfully fold.

Current Behavior

When I am attempting to create the first view (two or more projects), I don't get any results. My initial guess is that the SQL query is doing an AND between the two rows which would make sense as that's an impossible condition to meet hence, no results are being displayed. image

When I am attempting to create the second view (failed WUs), I get all WUs returned and am not sure why that is. image

Possible Solution (Optional)

For the first view, the ability to define AND/OR between the rows would be nice. However, the ability to support nested conditions might be a consideration too.

For the second view, I have no idea but the workaround for me is to use the Frames Completed is less than 100 to get the results.

Steps To Reproduce

  1. Have a WU database that has different Projects and some failures
  2. Create the query as shown in the screenshots above
  3. View the issues

Context

For view one, it is sometimes good to compare a bunch of Projects as they are similar in nature thus, should have similar PPD. It would be easy to spot any Project within that group that doesn't behave nicely.

For the second view, it is good to know what WUs have failed and when so you can see a pattern and troubleshoot or report the issue.

I am aware that you have some plans around WU History so it is something that you can keep in mind when you cross that bridge 😄

harlam357 commented 3 years ago

Yes, you are correct that the default WHERE behavior is AND. When I first created this (many years ago) I thought that getting too complex with this query configuration would lead to more confusion. However, in the ProjectID query you're constructed, I can see where it would logically make sense to OR conditions on the same field automatically. So that would accomplish what you're looking for without needing to worry the user with specifying the AND or OR detail.

The query on Unit Result should work that way. In fact, I have a query here setup for UNSTABLE_MACHINE result, but it is failing to return results as well. I'm guessing there's a regression here that probably snuck in some time ago. The reason this isn't working for you is because those values aren't stored verbatim, they're assigned to an enumeration. Here are the values you need to make this work in the mean time.

"FINISHED_UNIT" = 1 "EARLY_UNIT_END" = 2 "UNSTABLE_MACHINE" = 3 "INTERRUPTED" = 4 "BAD_WORK_UNIT" = 5 "CORE_OUTDATED" = 6 "GPU_MEMTEST_ERROR" = 8 "UNKNOWN_ENUM" = 9

The following query will get you what you want to see. Good observation. Thank you! I'll put this on the bug list.

image

PantherX commented 3 years ago

That's a really good idea of OR when Name is same. It doesn't need any additional UI changes and is intuitive!

Ah, that's fine. I did discover the bug in 0.9.17.1040 but before reporting it, updated to the latest release just in case it was fixed. Unfortunately, I don't know when that bug crept in 😢

Thanks for providing the list of values, I have three observations:

  1. Is there a reason "7" is missing?
  2. I noticed that WU_STALL is missing.
  3. Can there be more than 10 values stored in case there are more exit codes in future FahCores since the enumeration stops at 9?
harlam357 commented 3 years ago

Gotcha... it's been a while since I've been in that section of that app so I'm not surprised it's also an issue with v0.9.17.

  1. 7 is for the old Client core communications error from the v5 / v6 days. It's not valid anymore for v7 but I leave it in the code base because there are obviously things tied to these numeric values. I can't just remove it and allow the value of "GPU_MEMTEST_ERROR" to become 7. And some of these results, like "GPU_MEMTEST_ERROR", are also v6 related.

But I have 86000 WUs worth of history, a lot from those days and those are still valid for someone like me who has WU history dating back that far. I've got just about 2/3 of all my returned WUs cataloged in my own database. Lots of opportunity for data mining (and graphing) information from all that history. However, not going to approach that until I get the persistence of WU history and benchmarks updated to a modern approach. It's just too cumbersome right now with the tech in place.

  1. Can you share some log.txt with WU_STALL? IIRC, that is a return value that indicates expected progress isn't being made but isn't actually a result of the work unit. IOW, FAHClient will pick back up and restart the WU. So in terms of WU completion or failure, that result is neither.

  2. Yes, its just an integer and doesn't stop at 9.

PantherX commented 3 years ago

Nice! I too had that idea but unfortunately, lost several thousand WUs when my system died in a flood 😭 Since then, I didn't keep track but recently, got back into it. Related question, backing up WuHistory.db3 & WuHistoryQuery.dat is needed if I were to retain history and Data views, right?

I look forward to all the data mining/graphic options in future!

The correct error is WU_STALLED which is rather rare. Here are some examples that I have found on the Forum: https://foldingforum.org/viewtopic.php?f=74&t=35348 https://foldingforum.org/viewtopic.php?f=19&t=30905 https://foldingforum.org/viewtopic.php?f=19&t=34405

harlam357 commented 3 years ago

Backups, yes. If you want to be super safe... take a copy of the entire folder where those live once in a while. When this all gets reworked, everything will be in one, likely, db3 file (which is SQLite format). I plan to continue to use SQLite at this point, but that could change if I find another compelling option.

PantherX commented 3 years ago

Thanks for that!