wildlife-dynamics / ecoscope-workflows

An extensible task specification and compiler for local and distributed workflows.
BSD 3-Clause "New" or "Revised" License
0 stars 1 forks source link

Visualize compilation specs with `pydot` #26

Open cisaacstern opened 1 month ago

cisaacstern commented 1 month ago

Allows visualization of graphs using pydot (and graphviz).

This is not implemented in the CLI yet, but from a Python interpreter we can do:

import yaml
from ecoscope_workflows.visualize import write_png

with open("examples/compilation-specs/time-density.yaml") as f:
    s = yaml.safe_load(f)
write_png(s, "output.png")

which outputs: image

cisaacstern commented 1 month ago

More complex dags can also be visualized, e.g. if we apply the following diff to the time density dag spec

diff --git a/examples/compilation-specs/time-density.yaml b/examples/compilation-specs/time-density.yaml
index e806b87..ba3cbec 100644
--- a/examples/compilation-specs/time-density.yaml
+++ b/examples/compilation-specs/time-density.yaml
@@ -6,7 +6,13 @@ tasks:
     observations: get_subjectgroup_observations
   relocations_to_trajectory:
     relocations: process_relocations
+  get_earth_engine_asset: {}
   calculate_time_density:
     trajectory_gdf: relocations_to_trajectory
+    image: get_earth_engine_asset
   draw_ecomap:
     geodataframe: calculate_time_density
+  summarize_table_stats: {}
+  serialize_aggregate_result:
+    ecomap: draw_ecomap
+    stats: summarize_table_stats
~

write_png will then generate

image

(This would be for a hypothetical dag in which calculate_time_density also required an Earth Engine asset as part of its calculation, which we wanted to fetch from a separate task. The part about the results is probably roughly representative of what we eventually do want to round out the time density workflow output.)

cisaacstern commented 1 month ago

cc @walljcg @Yun-Wu for viz. this isn't done yet, but thought you'd fine it interesting.

Hopefully this is helpful for communicating our vision of workflows this week!