meta-soul / MetaSpore

A unified end-to-end machine intelligence platform
Apache License 2.0
642 stars 118 forks source link

add metaspore flow init/create scene command #235

Closed javyxu closed 11 months ago

javyxu commented 1 year ago
  1. add metaspore flow init scene command Use metaspore flow init --scene_name scene_01 --scheduler_mode SagaMaker command can generate an initialized yaml as follows, you can change the configuration, such as scheduling mode, data source, intermediate result store, recommendation policy:
    name: scene_01
    scheduler_mode:
      name: SageMaker
      config:
        roleArn: ''
        securityGroups: []
        subnets: []
        s3Endpoint: ''
        s3WorkDir: ''
    datasource:
      connect_info:
        type: mysql
        host: 127.0.0.1
        port: 3306
        database: ''
        user: ''
        password: ''
      user_tab_info:
        name: user
        user_id_column_name: user_id
        age_col_name: ''
        gender_col_name: ''
        city_col_name: ''
      item_tab_info:
        name: item
        item_id_column_name: item_id
        price_column_name: ''
        title_column_name: ''
        brand_column_name: ''
        category_column_name: ''
        url_column_name: ''
        image_column_name: ''
        description_column_name: ''
      trans_tab_info:
        name: interaction
        user_id_column_name: user_id
        item_id_column_name: item_id
        timestamp_column_name: timestamp
    middle_result_storage:
      type: Mongo
      host: 127.0.0.1
      port: 27017
      database: ''
      user: ''
      password: ''
      authsource: admin
    recommend_strategy:
    recommend_result_count: 200
  2. add metaspore flow create scene command Use metaspore flow create --values ./scene_01.yml command can generate scene yamls, and then go into the scene folder to execute metaspore flow up/down/status command can start scene、stop scene and get scene status.