search

superphy / spfy

Spfy: an integrated graph database for real-time prediction of Escherichia coli phenotypes and downstream comparative analyses
https://lfz.corefacility.ca/superphy/grouch/
Apache License 2.0
4 stars 2 forks source link

Job fingerprinting #249

Open kevinkle opened 7 years ago

kevinkle commented 7 years ago

This will likely be done with to move to a custom Job class. The idea would be to:

  1. fingerprint (read: hash) the job submission
  2. lookup this hash in blazegraph
  3. a. If found, rebuild the results from the graph
  4. b. If not, run the job and tag the hash onto the results.
kevinkle commented 7 years ago

This is effectively the solution to https://github.com/superphy/backend/issues/204 . But will keep both open in case this approach doesn't work as intended.

kevinkle commented 6 years ago

Fingerprints in https://github.com/superphy/spfy/releases/tag/v6.2.0 are deterministic. We're going to do it as described above, but also use redis as an buffer.