search

logicalclocks / maggy

Distribution transparent Machine Learning experiments on Apache Spark
https://maggy.ai
Apache License 2.0
89 stars 14 forks source link

Environment #86

Closed RiccardoGrigoletto closed 3 years ago

RiccardoGrigoletto commented 3 years ago

The following modifications have been performed:

  1. Created an abstract environment class that defines the functions to be developed to work within an environment (hopsworks, databricks etc.)
  2. Developed the environment for hopsworks.
  3. Modified all the source code to be "hopsworks indipendent". This means that, instead of calling directly hopsworks, maggy now calls the functions of the available environment. These methods will be responsible to manage the calls with the available libraries, file systems etc.