Authoring tool API + CLI Rewrite

taylortom commented 3 years ago

Have outlined below the kind of API we'd need from the AAT side :smile:

Function names/param naming & order are completely up for grabs, but should give an idea of what we need.

Framework

The following API endpoints should only handle framework specific stuff (i.e. no plugin installs/updates).

/**
 * Installs a clean copy of the framework
 * @param {String} version Specific version of the framework to install
 * @param {String} dir Directory to install into
 * @return {Promise}
 */
installFramework(version = "", dir)
/**
 * Updates the framework
 * @param {String} version Specific version of the framework to install
 * @param {String} dir Root path of the framework installation
 * @return {Promise}
 */
updateFramework(version = "", dir)

It would be great to be able to access the build tools from an API wrapper, with some nice errors etc. returned.

/**
 * Runs the build for a current course
 * @param {String} dir Root path of the framework installation
 * @param {Object} options
 * @param {Boolean} options.devMode Whether to run the build in developer mode
 * @return {Promise}
 */
buildCourse(dir, options)

Framework plugins

/**
 * Installs a single framework plugin
 * @param {String} plugin Name of plugin (with optional version, e.g. @4.0.0 - defaults to the latest supported)
 * @param {String} dir Root path of the framework installation
 * @return {Promise} Resolves with the plugin's parsed bower JSON
 */
installPlugin(plugin = "", dir = process.cwd())
/**
 * Removes a single framework plugin
 * @param {String} plugin Name of plugin
 * @param {String} dir Root path of the framework installation
 * @return {Promise}
 */
uninstallPlugin(plugin, dir = process.cwd())
/**
 * Updates a single framework plugin
 * @param {String} plugin Name of plugin (with optional version, e.g. @4.0.0 - defaults to the latest supported)
 * @param {String} dir Root path of the framework installation
 * @return {Promise} Resolves with the plugin's parsed bower JSON
 */
updatePlugin(plugin = "", dir = process.cwd())

Generic utilities

For clarity, the below could also be split into separate framework/plugin variants.

/**
 * Retrieves all schema defined in the framework
 * @return {Promise} Resolves with array of JSON schema contents
 */
getSchemas()
/**
 * Loads a single JSON schema file by name
 * @param {String} name Name of the schema to load
 * @return {Promise} Resolves with JSON schema contents
 */
getSchema(name)
/**
 * Gets the update information for installed framework/plugins
 * @param {String} plugin Name of plugin (if not specified, all plugins are checked), should also accept 'adapt_framework'
 * @param {String} dir Root path of the framework installation
 * @return {Promise} Resolves with array/object with plugin update info (see below)
 */
getUpdateInfo(plugin = "", dir = process.cwd())

/** 
 * @example return data for getUpdateInfo 
 */
[
  {
    "name": "adapt-contrib-accordion",
    "isManaged": false,
    "installedVersion": "4.0.0",
    "nextCompatibleVersion": "5.0.0",
    "latestVersion": "5.0.0"
  },
  {
    "name": "adapt-contrib-spoor",
    "isManaged": false,
    "installedVersion": "3.3.2",
    "nextCompatibleVersion": "3.3.2",
    "latestVersion": "3.3.2"
  },
  {
    "name": "adapt-custom",
    "isManaged": true,
    "installedVersion": "4.0.0",
    "nextCompatibleVersion": "5.0.0",
    "latestVersion": "5.0.0"
  }
]

oliverfoster commented 3 years ago

Awesome ta. Schemas?

taylortom commented 3 years ago

Regarding schemas: currently we have a glob that checks the current framework dir for /**/schema/*.schema.json, and register the matches in the json validator. We only store the schema name + path (altho we have to do a load upfront to make sure schema name + any extensions to other schemas are reigstered). We then load each schema fresh at the point of validation.

Not sure of the best way to represent this in an API (i.e. how much of this you want to handle on the CLI side)

Something else to discuss is how we share the validation tasks. Atm all validation is done in an AAT-specific json schema module (which includes various extra API stuff), but might be nice to make this a bit more abstract and use it on the framework side too?

https://github.com/adapt-security/adapt-authoring-jsonschema

oliverfoster commented 3 years ago

I'd like it to at least load the schemas for you, rather than by blob.

Secondarily and when the need arises to externalise validation (possibly never), we can then get the validation stuff out of the aat and make it a module. I don't think we have need on this side at the moment, but worth earmarking.

taylortom commented 3 years ago

Have updated the above. Happy to leave validation as is until (or indeed, if) we have a need for anything different.

oliverfoster commented 3 years ago

How does the new AAT install custom plugins currently?

taylortom commented 3 years ago

How does the new AAT install custom plugins currently?

There's nothing in the CLI that deals with this kind of thing, so we just do the following:

Preliminary compatibility checks
Database entry added
Zip contents is put in the framework, with a copy in a local 'data' folder
- Copy is used to restore the plugin if the framework folder is removed
- Copy will also be likely used for situations where rolling back a borked update/install is needed

taylortom commented 3 years ago

@oliverfoster I've just realised I haven't planned for installing custom versions of the framework. We allow this in the current AAT by pulling in a git URL from the config file and cloning from that. I don't think we've ever made use of it at Kineo, but I know of others that have.

Not sure of the best way to do this, possibly just passing the URL as a third param to the installFramework function.

It hopefully shouldn't impact any of the other functions, and I think we can be very strict with dealing with errors related to anything custom, so there shouldn't be any special behaviour there.

oliverfoster commented 3 years ago

I have the backend functions for that already made, so it should be quite straightforward. 👍

https://github.com/adaptlearning/adapt-cli/blob/issue/148/lib/integration/AdaptFramework/downloadFrameworkBranchTo.js

https://github.com/adaptlearning/adapt-cli/blob/issue/148/lib/integration/AdaptFramework/cloneFrameworkBranchTo.js

taylortom commented 3 years ago

Wahoo, that's good news :smile: There's no rush to get this in immediately, but wanted to throw it out there

oliverfoster commented 2 years ago

getSchema perhaps redundant if getSchemas returns all schema contents, do you need file paths?

Update: Actually, you were quite verbose in your original summary. I'll do my best here and we'll talk about it later.

Regarding schemas: currently we have a glob that checks the current framework dir for /*/schema/.schema.json, and register the matches in the json validator. We only store the schema name + path (altho we have to do a load upfront to make sure schema name + any extensions to other schemas are reigstered). We then load each schema fresh at the point of validation.

Not sure of the best way to represent this in an API (i.e. how much of this you want to handle on the CLI side)

oliverfoster commented 2 years ago

Initial API attempt

Framework

/**
   * Installs a clean copy of the framework
   * @param {Object} options
   * @param {string} [options.version=null] Specific version of the framework to install
   * @param {string} [options.repository] URL to github repo
   * @param {string} [options.cwd=process.cwd()] Directory to install into
   * @return {Promise}
   */
  async installFramework ({
    version = null,
    repository = ADAPT_FRAMEWORK,
    cwd = process.cwd()
  } = {})

  /**
   * @param {Object} options
   * @param {Object} [options.repository=ADAPT_FRAMEWORK] The github repository url
   * @returns {string}
   */
  async getLatestFrameworkVersion ({
    repository = ADAPT_FRAMEWORK
  } = {})

  /**
   * Runs build for a current course
   * @param {Object} options
   * @param {boolean} [options.sourceMaps=false] Whether to run the build with sourcemaps
   * @param {boolean} [options.checkJSON=false] Whether to run without checking the json
   * @param {boolean} [options.cache=true] Whether to clear build caches
   * @param {string} [options.outputDir="build/"] Root path of the framework installation
   * @param {string} [options.cachePath="build/.cache"] Path of compilation cache file
   * @param {string} [options.cwd=process.cwd()] Root path of the framework installation
   * @return {Promise}
   */
  async buildCourse ({
    sourceMaps = false,
    checkJSON = false,
    cache = true,
    outputDir = './build/',
    cachePath = './build/.cache',
    cwd = process.cwd()
  } = {})

Framework plugins

  /**
   * Installs multiple plugins
   * Can install from zip, source folder or bower registry
   * @param {Object} options
   * @param {[string]} [options.plugins=null] An array with name@version or name@path strings
   * @param {string} [options.cwd=process.cwd()] Root path of the framework installation
   * @return {Promise}
   */
  async installPlugins ({
    plugins = null,
    cwd = process.cwd()
  } = {})

  /**
   * Uninstalls multiple plugins
   * @param {Object} options
   * @param {[string]} [options.plugins=null] An array with name@version or name@path strings
   * @param {string} [options.cwd=process.cwd()] Root path of the framework installation
   * @return {Promise}
   */
  async uninstallPlugins ({
    plugins = null,
    cwd = process.cwd()
  } = {})

  /**
   * Updates multiple plugins
   * Can install from zip, source folder or bower registry
   * @param {Object} options
   * @param {[string]} [options.plugins=null] An array with name@version or name@path strings
   * @param {string} [options.cwd=process.cwd()] Root path of the framework installation
   * @return {Promise}
   */
  async updatePlugins ({
    plugins = null,
    cwd = process.cwd()
  } = {})

Generic utilities

  /**
   * Retrieves all schemas defined in the project
   * Clears schema cache
   * @param {string} [options.cwd=process.cwd()] Root path of the framework installation
   * @return {Promise<[string]>} Resolves with array of JSON schema filepaths
   */
  async getSchemaPaths ({
    cwd = process.cwd()
  } = {}) 

  /**
   * Retrieves named schema
   * Caches schemas for subsequent use
   * Call getSchemaPaths to reset cache
   * @param {string} options.name Schema filepath as returned from getSchemaPaths
   * @return {Promise<Object>} Resolves with the JSON schema contents
   */
  async getSchema ({
    name
  } = {})

  /**
   * Returns all installed plugins
   * @return {Promise<[Plugin]>}
   */
  async getInstalledPlugins ({
    cwd = process.cwd()
  } = {}

  /**
   * Gets the update information for installed plugins
   * @param {Object} options
   * @param {[string]} [options.plugins=null] An arrat of plugin names (if not specified, all plugins are checked)
   * @param {string} [options.cwd=process.cwd()] Root path of the framework installation
   * @return {Promise<[Plugin]>} Resolves plugin update info
   */
  async getPluginUpdateInfos ({
    plugins = null,
    cwd = process.cwd()
  } = {})

  /**
   * Returns an object representing the plugin at the path specified
   * @returns {Plugin}
   */
  async getPluginFromPath ({
    pluginPath,
    cwd = null
  } = {})

Plugin

  /**
   * @param {Object} options
   * @param {string} options.name
   * @param {string} options.requestedVersion
   * @param {Project} options.isContrib
   * @param {boolean} options.isCompatibleEnabled whether to target the latest compatible version for all plugin installations (overrides requestedVersion)
   * @param {Project} options.project
   * @param {string} options.cwd
   * @param {Object} options.logger
   */
  constructor ({
    name,
    requestedVersion = '*',
    isContrib = false,
    isCompatibleEnabled = false,
    project,
    cwd = (project?.cwd ?? process.cwd()),
    logger
  } = {})

  /**
   * the installed version is the latest version
   * @returns {boolean|null}
   */
  get isUpToDate ()

  /**
   * the most recent version of the plugin
   * @returns {string|null}
   */
  get latestSourceVersion ()

  /**
   * the installed version of the plugin
   * @returns {string|null}
   */
  get projectVersion ()

  /**
   * a list of tags denoting the source versions of the plugin
   * @returns {[string]}
   */
  get sourceVersions ()

  /**
   * plugin will be or was installed from a local source
   * @returns {boolean}
   */
  get isLocalSource ()

  /**
   * plugin will be or was installed from a local source zip
   * @returns {boolean}
   */
  get isLocalSourceZip ()

  /** @returns {boolean} */
  get isVersioned ()

  /**
   * is a contrib plugin
   * @returns {boolean}
   */
  get isContrib ()

  /**
   * whether querying the server or disk for plugin information worked
   * @returns {boolean}
   */
  get isPresent ()

  /**
   * has user requested version
   * @returns {boolean}
   */
  get hasUserRequestVersion ()

  /**
   * the supplied a constraint is valid and supported by the plugin
   * @returns {boolean|null}
   */
  get hasValidRequestVersion ()

  /** @returns {boolean} */
  get hasFrameworkCompatibleVersion ()

  async fetchSourceInfo ()

  async fetchLocalSourceInfo ()

  async fetchBowerInfo ()

  async fetchProjectInfo ()

  async findCompatibleVersion (framework)

  /**
   * @returns {string}
   */
  async getType ()

  async getTypeFolder ()

  async getRepositoryUrl ()

  /** @returns {string} */
  toString ()

  async getSchemaPaths ()

  clearExtractedSource ()

adaptlearning / adapt-cli