raptor-ml / raptor

Transform your pythonic research to an artifact that engineers can deploy easily.
https://raptor.ml
Apache License 2.0
148 stars 11 forks source link

Refactor to match the new design #176

Closed AlmogBaku closed 1 year ago

AlmogBaku commented 1 year ago

This PR introduces the business-logic aspects of the new design. Although it's a very-large PR, which is usually not a good practice, it was essential to lay out fundamental changes to the Core structure, which are hard to do incrementally.

This Epic includes:

  1. Introducing keys instead of entity_id, including key-name awareness
  2. Python runtime replaces the pyexp runtime
    1. remove the go-python-binding in the labsdk
    2. loading pip packages
    3. (loosely) protection against restricted packages(including dataset, i/o and modeling packages)
    4. resolving references to fqns
    5. unix domain socket interface
  3. many bug fixes
  4. data source refactoring
    1. data transformation happens in the data source
    2. rest dsrc
  5. new python handler api

feat: python runtime #157 feat: install pip packages #157 feat: support multiple keys #171 bug: runtime supports multiple returns #157 feat: add runtime dockerfile to make #157 refactor: change unix-socket location #157 refactor: remove schema from the runtime rpc feat: runtime-manager: discover runtimes, execute, load, attach to runner #157 fix: change status of dsrc when ready feat: introduce boolean and []boolean primitives fix: headless is not a primitive #156 fix: REST configuration should be inside the dsrc #150 refactor: remove pyexp-runtime #157 refactor: remove programregistry #157 fix: #166 fix: #165 fix #163 fix #155 fix #118

github-actions[bot] commented 1 year ago

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

AlmogBaku commented 1 year ago

without merging, it's gonna be very challenging to test. thankfully, we haven't launched it, so I'm gonna do it :P