Common: Isolate FCLI home folder for pipelines

[rsenden]

When fcli is used in non-containerized pipelines on a CI/CD system that may be running multiple pipelines simultaneously, any fcli command that reads or writes files to the FCLI home folder (~/.fortify/fcli by default) may be reading or overwriting data from other pipelines.

For example, if multiple pipelines issue session login and session logout commands with the same session name, one pipeline may log out of that session while another pipeline is still using it. This becomes even more of an issue if we introduce support for persistent data storage between command invocations, for example storing the last started scan id for use by subsequent commands; one pipeline might be reading the last started scan id that was persisted by another pipeline.

FcliHomeHelper already supports overriding the home folder by setting the FORTIFY_HOME environment variable, resulting in files being persisted to and read from <FORTIFY_HOME>/fcli. We should explicitly document that for non-containerized pipelines should set this variable to point to a folder that is unique for a particular pipeline run, for example pointing to a subdirectory of the pipeline workspace directory.

To make this more explicit, we could also introduce a new FCLI_PIPELINE_ID environment variable, resulting in something like <FORTIFY_HOME|~/.fortify>/fcli/pipeline-<FCLI_PIPELINE_ID> to be used as the home folder for that particular pipeline.

[rsenden]

Support for FCLI_PIPELINE_ID (and FCLI_HOME) environment variables has been implemented; remaining task is to properly document these environment variables.

[rsenden]

We should also add a command for cleaning up the pipeline directory after use to avoid dangling files; based on FCLI_PIPELINE_ID, this command should remove the corresponding directory (cleaning up all session and variable files)

[rsenden]

After second thought, creating separate directories for each pipeline may cause too many dangling files and directories if they are not being properly cleaned up. As such, this functionality has been removed again. Instead, we should recommend setting the FCLI_HOME variable to a pipeline-specific directory on shared systems (i.e. set FCLI_HOME to a subdirectory of the workspace directory, assuming this will be cleaned up after the pipeline finishes.

[MikeTheSnowman]

After second thought, creating separate directories for each pipeline may cause too many dangling files and directories if they are not being properly cleaned up. As such, this functionality has been removed again. Instead, we should recommend setting the FCLI_HOME variable to a pipeline-specific directory on shared systems (i.e. set FCLI_HOME to a subdirectory of the workspace directory, assuming this will be cleaned up after the pipeline finishes.

How do you feel about a fcli config command to help manage this kind of thing?

• fcli config pipeline clear [pipeline_id] # pipeline ID is optional. If no pipeline id is specified, it will try to get one from the env variable
• fcli config pipeline clear-all # clears all pipeline-<FCLI_PIPELINE_ID> folders

[rsenden]

I've thought about that approach as well, but I think it's just easier to have a recommendation to set the FORTIFY_HOME or FCLI_HOME to a pipeline-specific directory