boozallen / aissemble

Booz Allen's lean manufacturing approach for holistically designing, developing and fielding AI solutions across the engineering lifecycle from data processing to model building, tuning, and training to secure operational deployment
Other
33 stars 8 forks source link

[BUG] Downstream projects may fail to build with build cache enabled under certain workflows #90

Closed peter-mcclonski closed 3 months ago

peter-mcclonski commented 3 months ago

Steps to reproduce

  1. Instantiate a new aissemble project with the following command:
    mvn archetype:generate -B -DarchetypeGroupId=com.boozallen.aissemble \
                          -DarchetypeArtifactId=foundation-archetype \
                          -DarchetypeVersion=1.7.0-SNAPSHOT \
                          -DartifactId=test-project-90\
                          -DgroupId=org.test \
                          -DprojectName='Test Cache' \
                          -DprojectGitUrl=test.org/test-project-90\
    && cd test-project-90
  2. Add the following pipeline as test-project-90-pipeline-models/src/main/resources/pipelines/SimplePipeline.json:
    {
    "name": "SimplePipeline",
    "package": "org.test",
    "type": {
        "name": "data-flow",
        "implementation": "data-delivery-spark"
    },
    "steps": [
        {
            "name": "Ingest",
            "type": "synchronous",
            "alerting": {
                "enabled": false
            },
            "dataProfiling": {
                "enabled": false
            },
            "provenance": {
                "enabled": false
            }
        }
    ]
    }
  3. Execute mvn clean install once, following all manual action notifications.
  4. Execute mvn clean generate-sources repeatedly, resolving all manual action notifications until none remain.
  5. Execute mvn clean install
  6. Observe that the simple-pipeline module fails to compile due to unresolved references.

Definition of Done

Test Steps

  1. Instantiate a new aissemble project with the following command:
    mvn archetype:generate -B -DarchetypeGroupId=com.boozallen.aissemble \
                          -DarchetypeArtifactId=foundation-archetype \
                          -DarchetypeVersion=1.7.0-SNAPSHOT \
                          -DartifactId=test-project-90\
                          -DgroupId=org.test \
                          -DprojectName='Test Cache' \
                          -DprojectGitUrl=test.org/test-project-90\
    && cd test-project-90
  2. Add the following pipeline as test-project-90-pipeline-models/src/main/resources/pipelines/SimplePipeline.json:
    {
    "name": "SimplePipeline",
    "package": "org.test",
    "type": {
        "name": "data-flow",
        "implementation": "data-delivery-spark"
    },
    "steps": [
        {
            "name": "Ingest",
            "type": "synchronous",
            "alerting": {
                "enabled": false
            },
            "dataProfiling": {
                "enabled": false
            },
            "provenance": {
                "enabled": false
            }
        }
    ]
    }
  3. Execute mvn clean install once, following all manual action notifications.
  4. Execute mvn clean generate-sources repeatedly, resolving all manual action notifications until none remain.
  5. Execute mvn clean install
  6. Observe that the build completes successfully.
ewilkins-csi commented 3 months ago

DoD looks good

peter-mcclonski commented 3 months ago

Closing as will not fix. The aiSSEMBLE build involves an enormous number of technologies, and abbreviating the build process using the fermenter generate-sources goal is likely to cause more problems than it solves. The desired mechanism for incepting a project and creating the generated code is to run the full mvn clean install process repeatedly.