award28
commented
1 year ago 1:1 JavaScript Functionality Proof of Concept Notes
I've been successful in creating a proof of concept. I'll breakdown the different requirements in order to get this working and how we can achieve cross-platform execution. You can find my branch with my poc progress here. Note that if you want to pull this down and run it (not recommended as of this writing, it's a mess😅), you'll need to copy the read_file function mentioned below into the generated core javascript package.
Clap
Action validator is heavily integrated with clap. clap has some fantastic abstractions that make it a really good choice for a rust-built CLI. At the same time, because of these abstractions, we need to understand the clap::Parser innards in order to replace some of the default behaviors with wasm-friendly alternatives.
Overriding the Default Arguments Input
std::env::args_os() is rust's builtin function for reading arguments. As described by the docstring, this function:
Returns the arguments that this program was started with (normally passed via the command line).
This returns a std::env::ArgsOs struct, but for our purposes all we care about is that it is an Iterable which can be converted into an OsString. This iterable is what gets passed into the clap::command. So for our purposes, we can replace the value of std::env::ArgsOs with an iterable that satisfies those conditions; thankfully, a Vec<String> converted into an iterable satisfies these conditions as well.
This means we can pass a list of strings to the javascript entrypoint function and have the CliConfig parse those instead! There is a catch here, however, as the clap::command doesn't simply return when a specific flag was seen. Instead, the flag is consumed behind the scenes and the program is exited. At this point, the javascript wasm build would raise an Unreachable error.
Making the --help and --version Flag Results Consumable
With the default behavior for ArgAction::Help and ArgAction::Version , the CliConfig::parse() function will short-circuit and print to the console. Clap treats the responses of these actions as a Result::Err. In order to work around this but still keep the same output, we need to capture the parsing response lower in the clap stack.
To maintain the existing functionality (and any future clap defaults), we don't want to manually re-write the default behavior for each builtin flag. Instead, we want to capture the output of that flag, and send that as the response for this invocation. To do this, we need to understand how clap handles these flags.
When clap runs into a "special" flag, it treats it as a Result::Err in a Result response. The value in that error has an exit method, which will format the error, print the error, and exit the program. The second step is where we want to intercept this exchange and capture the formatted error.
Code Example
// src/main.rs
use action_validator::CliConfig;
fn main() {
let res = CliConfig::parse_itr(std::env::args_os());
...
}
// src/config.rs
use std::ffi::OsString;
use clap::{CommandFactory, FromArgMatches, Error};
impl CliConfig {
fn format_error<I: CommandFactory>(err: Error) -> Error {
let mut cmd = I::command();
err.format(&mut cmd)
}
pub fn parse_itr<I, T>(itr: I) -> Result<Self, String>
where
I: IntoIterator<Item = T>,
T: Into<OsString> + Clone,
{
let arg_matches = <Self as CommandFactory>::command()
// Get the matching argument actions to build the CliConfig
.try_get_matches_from_mut(itr);
if let Err(e) = arg_matches {
return Err(format!("{e}"));
}
let mut matches = arg_matches.unwrap();
let res = <Self as FromArgMatches>::from_arg_matches_mut(&mut matches)
// Capture the builtin flags output
.map_err(Self::format_error::<Self>);
match res {
// If no errors occurred, capture the CliConfig instance
Ok(s) => Ok(s),
// This is a developer time exception - we may want to handle this one differently
Err(e) => Err(format!("{e}")),
}
}
}JavaScript
Passing CLI Arguments
As of this writing, all of the entrypoint functionality for action-validator is being manually written in javascript, which then calls the core functionality lower in the action-validator stack. Instead, we can pass action-validator a list of process arguments which will then be invoked with our above implementation of parse_itr. In node, we can use the process.argv var in order to get cli arguments. For node, the first argument in the list is always the node binary, and the second is the executed js file. To match the format of the rust cli arguments, we'll want a slice of this list. Namely, process.argv.slice(1).
If there are any errors, they will be written to stderr. Otherwise, nothing is written and the process will exit according to the response specified exit code.
Code Example
// packages/cli/cli.mjs
#!/usr/bin/env node
// @ts-check
import * as actionValidator from "@action-validator/core";
let response = actionValidator.entrypoint(
JSON.stringify({"args": process.argv.slice(1)})
);
if (response.errors) {
console.error(response.errors);
}
process.exit(response.exit_code);Reading File Contents
Because rust is unaware of the underlying OS platform when invoked through wasm, rust can't read files. Because of this, we need to externally accept a file reading function, and use that in order to get the contents. We also need to provide an abstraction for this reader in order to return rust-wasm compatible values.
Code Example
// src/lib.rs
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(catch)]
fn read_file(path: &str) -> Result<String, JsValue>;
}
pub fn run_path(config: &CliConfig, path: &PathBuf) -> Response {
let file_name = match path.file_name() {
Some(file_name) => file_name.to_str(),
None => {
return Response {
errors: Some(format!("Unable to derive file name from src!")),
exit_code: 1,
};
}
};
let src = &match read_file(path.to_str().unwrap()) {
Ok(src) => src,
...TODO: Need to move this to a non-generated file.
// package/core/action_validator.js
var fs = require('fs');
function read_file(path) {
return fs.readFileSync(path, { encoding: "utf8" });
}Action Validator
There are a few changes which will be required in order to make an input/output high-level interface for action-validator itself.
- Modify the run command to return a result rather than writing anything to stderr/stdout.
- Change all of the print statements to either:
- A wasm-bound function enabled when the
jsfeature is enabled. - The
runcommand returns a value rather than writing anything to stderr/stdout.
- A wasm-bound function enabled when the
We can use the js feature flag in order to execute with either rust (disabled; executed through src/main.rs) or javascript (enabled; executed through packages/cli/cli.mjs).
With the above code in place (and some other minor parts not mentioned above), I was able to run all flags through action-validator and handle all response errors.
npx action-validator --verbose .github/workflows/audit.yml
Treating audit.yml as a Workflow definition
WARNING: Glob validation is not yet supported. Glob at /on/push/paths will not be validated.
❯ npx action-validator -V
action-validator 0.0.0-git
❯ npx action-validator --version
action-validator 0.0.0-git
❯ npx action-validator --help
A validator for GitHub Action and Workflow YAML files
Usage: action-validator [OPTIONS] [path_to_action_yaml]...
Arguments:
[path_to_action_yaml]... Input file
Options:
-v, --verbose Be more verbose
-h, --help Print help information
-V, --version Print version information
bcheidemann
mpalmer
Background
As per the discussion with @bcheidemann in https://github.com/mpalmer/action-validator/pull/36#issuecomment-1464698569, this issue will track the necessary work to make the @action-validator/cli npm package behave equivalently to the native binary in terms of multi-file input support.
Scope of Work
Acceptance Criteria
When
action-validatoris installed through npm, multiple files can be passed as input.Example of Native Binary Behavior