Implement Google TensorFlow Extended (TFX)

[slavakurilyak]

Goal(s)

As a developer, I want to implement Google's TensorFX, TensorFlow framework for training and serving machine learning models, so that I can better handle the hidden technical debt in machine learning systems.

Inspiration

Google recognized the importance of dealing with hidden technical debt in machine learning systems (YouTube, 2018). Machine learning developers must not only create ML Code, but also deal with the following:

1. Configuration
2. Data Verification
3. Data Collection
4. Feature Extraction
5. Monitoring
6. Analysis Tools
7. Process Management Tools
8. Serving Infrastructure
9. Machine Resource Management

To solve this problem, Google introduced TensorFlow Extended (TFX). Here is a short [4 minute video] which describes TFX.

As of today, Google is in Phase 1 (Google, 2017).

For Phase 2, Google will implement Data Digestion and Data Visualization & Validation (YouTube, 2018).

For Phase 3, Google will implement Integrated Frontend for Job Management, Monitoring, Debugging, Data/Model/Evaluation Visualization and Shared Configuration Framework and Job Orchestration (YouTube, 2018).

In the future, Google will release all of the tools necessary for this end-to-end framework (YouTube, 2018).

Since TFX has not been fully released, this task has the label waiting-for.