ngageoint / scale

Processing framework for containerized algorithms
http://ngageoint.github.io/scale/
Apache License 2.0
105 stars 45 forks source link

Basic Scale job how-to #1883

Closed Fizz11 closed 4 years ago

Fizz11 commented 4 years ago

While we have documentation for most parts of scale, it would be beneficial to have a start to finish tutorial of creating a simple job and incorporating it in scale. Instructions for the following tasks should be written out in a simple, friendly blog-style post with screenshots and code snippets where applicable. This tutorial will assume the scale infrastructure is already up and running.

  1. Create a simple program.

    • program should have one input and one output.
    • program should take two arguments, 'INPUT_FILE' which will be the filepath for the input file, and "OUTPUT' which will be the directory path for the output directory.
    • program should copy input file with a new name and write it to the output directory.
  2. Create a seed manifest file for the program

  3. dockerize program and build with seed-cli

    • create a dockerfile and base the image on alpine
    • build image using seed-cli command seed build
    • upload image to docker hub
  4. Add new image to seed-silo seed-silo documentation here: https://github.com/ngageoint/seed-silo#usage

  5. set up a new workspace in scale-ui this will be a local workspace pointing at a shared folder that all scale nodes have access to.

  6. create job-type for program in scale-ui

  7. create new recipe for job-type in scale-ui

  8. create new strike in scale-ui

  9. add files to input workspace and verify recipe and job run correctly