SAME-Project / same-project

https://sameproject.ml/
Apache License 2.0
20 stars 8 forks source link

Azure Functions backend for SAME #17

Closed gohar94 closed 3 years ago

gohar94 commented 3 years ago

Basic structure layed out for using Azure Functions and Durable Functions as a backend.

github-actions[bot] commented 3 years ago

Unit Test Results

  1 files    1 suites   29s :stopwatch: 47 tests 45 :heavy_check_mark: 2 :zzz: 0 :x:

Results for commit 75de460e.

:recycle: This comment has been updated with latest results.

aronchick commented 3 years ago

This is terrific - i can't believe you've already gotten it to work with my code! Just outstanding :)

But I wonder if there's another way to architect this that we should talk about - the way this is architected, it implies the CLI is the orchestrator, handing off jobs one at a time to AzF and coordinating. However, in the old model (https://github.com/azure-octo/same-cli/tree/main/static), we built templates to create and execute on alternate backends (kubeflow and AML to start). I think this latter model is the way we want to go - we don't want to rewrite yet another workflow engine.

So, in this case, imagine we took your code and, instead, built a way to describe a workflow on Azure Logic Apps (or something similar), and, inside that, we executed functions serially.

Does that make sense?

cc @rynowak @SAME-Project/same-team-core

gohar94 commented 3 years ago

@aronchick How do you generate the root requirements.txt with the versions etc.?

gohar94 commented 3 years ago

cc: @goiri

github-actions[bot] commented 3 years ago

Coverage

Coverage Report
FileStmtsMissCover
cli/same
   init.py00100%
   helpers.py00100%
   main.py11110%
   mapping.py10100%
   same_config.py541376%
   stdlib.py10100%
cli/same/program
   init.py00100%
   commands.py15193%
cli/same/program/compile
   init.py00100%
   context.py40100%
   notebook_processing.py98991%
cli/same/version
   init.py00100%
   commands.py40100%
sdk
   setup.py12120%
sdk/same
   init.py10100%
   _version.py110%
   conda_env.py471177%
   core.py110%
   helpers.py00100%
   importer.py651478%
   parallel.py00100%
TOTAL3157377% 

Tests Skipped Failures Errors Time
47 2 :zzz: 2 :x: 0 :fire: 19.236s :stopwatch:
github-actions[bot] commented 3 years ago

Coverage

Coverage Report
FileStmtsMissCover
backends
   init.py00100%
backends/common
   init.py00100%
   code_executor.py24240%
   exception_utils.py880%
   http_utils.py16160%
   serialization_utils.py12120%
cli/same
   init.py00100%
   helpers.py00100%
   main.py11110%
   mapping.py10100%
   same_config.py541376%
   stdlib.py10100%
cli/same/program
   init.py00100%
   commands.py15193%
cli/same/program/compile
   init.py00100%
   context.py40100%
   notebook_processing.py103991%
cli/same/version
   init.py00100%
   commands.py40100%
objects
   init.py00100%
   json_serializable_object.py281064%
   step.py26581%
sdk
   setup.py12120%
sdk/same
   init.py10100%
   _version.py110%
   conda_env.py471177%
   core.py110%
   helpers.py00100%
   importer.py65691%
   parallel.py00100%
TOTAL43414068% 

Tests Skipped Failures Errors Time
47 2 :zzz: 0 :x: 0 :fire: 29.980s :stopwatch: