search

moonrepo / moon

A build system and monorepo management tool for the web ecosystem, written in Rust.
https://moonrepo.dev/moon
MIT License
2.82k stars 154 forks source link

[bug] Running a task in workspace level get cached even with no inputs #880

Closed admin-ssg closed 1 year ago

admin-ssg commented 1 year ago

Describe the bug

I have a .moon/tasks.yml literally like this

tasks:
  test:
    command: 'yarn test:unit'

And no .moon.yml in the project level

When I run yarn moon run :test in the workspace level the test passed result gets cached, no matters what I change in the project level's code or test file, it will always pass due to the caching

Expected behavior

No caching

Environment npx envinfo starts, but does not output anything, even after couple of minutes. Seems to be stuck.

milesj commented 1 year ago

@admin-ssg Can you paste the --log trace output? Is there a demo of this somewhere I can test?

admin-ssg commented 1 year ago

These are the logs

Ran all test suites.
[TRACE 12:06:48] log  Processed action RunTarget(vgef:test) in 27.032167ms  log.target="moon:action-pipeline:batch:3:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=214
[DEBUG 12:06:48] log  Finished running 4 actions in 32.252667ms  log.target="moon:action-pipeline" log.module_path="moon_action_pipeline::pipeline" log.file="crates/core/action-pipeline/src/pipeline.rs" log.line=214
[TRACE 12:06:48] log  Cleaning up and deleting stale cache older than "7 days"  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=109
[TRACE 12:06:48] starbase_utils::fs  Removing stale contents from directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes
[TRACE 12:06:48] starbase_utils::fs  Reading directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes
[TRACE 12:06:48] starbase_utils::fs  Removing stale contents from directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/outputs
[TRACE 12:06:48] starbase_utils::fs  Reading directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/outputs
[TRACE 12:06:48] log  Deleted 0 files and saved 0 bytes  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=121
[TRACE 12:06:48] log  Writing report ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/runReport.json  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=151
[TRACE 12:06:48] starbase_utils::json  Stringifying JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/runReport.json
[TRACE 12:06:48] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/runReport.json

About the demo, unfortunately it's a private repo but if this logs doesn't help maybe I can try setting up a public one to test

milesj commented 1 year ago

I'd need the entire log to see what's going on. There should be log lines about what's files it's checking to see if it's affected.

admin-ssg commented 1 year ago

Hey, sorry! here are all the logs

 yarn test --log trace > logs.txt
[TRACE 2023-06-01 21:08:23] log  Attempting to find workspace root at ~/kasama/ssg_monorepo/async_backend/lambdaHandlers  log.target="moon:workspace" log.module_path="moon_workspace::workspace" log.file="crates/core/workspace/src/workspace.rs" log.line=33
[TRACE 21:08:23] starbase_utils::fs  Traversing upwards to find a file  file=".moon" dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers
[DEBUG 21:08:23] log  Creating workspace at ~/kasama/ssg_monorepo/async_backend/lambdaHandlers (from working directory ~/kasama/ssg_monorepo/async_backend/lambdaHandlers)  log.target="moon:workspace" log.module_path="moon_workspace::workspace" log.file="crates/core/workspace/src/workspace.rs" log.line=220
[TRACE 21:08:23] log  Loading .moon/workspace.yml from ~/kasama/ssg_monorepo/async_backend/lambdaHandlers  log.target="moon:workspace" log.module_path="moon_workspace::workspace" log.file="crates/core/workspace/src/workspace.rs" log.line=151
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/workspace.yml
[TRACE 21:08:23] starbase_utils::yaml  Parsing YAML  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/workspace.yml
[TRACE 21:08:23] log  Loading .moon/toolchain.yml from ~/kasama/ssg_monorepo/async_backend/lambdaHandlers  log.target="moon:workspace" log.module_path="moon_workspace::workspace" log.file="crates/core/workspace/src/workspace.rs" log.line=118
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/toolchain.yml
[TRACE 21:08:23] starbase_utils::yaml  Parsing YAML  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/toolchain.yml
[TRACE 21:08:23] log  Attempting to find .moon/tasks.yml in ~/kasama/ssg_monorepo/async_backend/lambdaHandlers  log.target="moon:workspace" log.module_path="moon_workspace::workspace" log.file="crates/core/workspace/src/workspace.rs" log.line=73
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/tasks.yml
[TRACE 21:08:23] starbase_utils::yaml  Parsing YAML  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/tasks.yml
[ WARN 21:08:23] log  Unknown fields in config file .moon/tasks.yml: langauge  log.target="moon:config" log.module_path="moon_config::helpers" log.file="crates/core/config/src/helpers.rs" log.line=110
[TRACE 21:08:23] log  Attempting to find .moon/tasks/*.yml in ~/kasama/ssg_monorepo/async_backend/lambdaHandlers  log.target="moon:workspace" log.module_path="moon_workspace::workspace" log.file="crates/core/workspace/src/workspace.rs" log.line=88
[DEBUG 21:08:23] log  Creating cache engine at ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=40
[TRACE 21:08:23] starbase_utils::fs  Traversing upwards to find a file  file=".git" dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers
[TRACE 21:08:23] starbase_utils::fs  Traversing upwards to find a file  file=".git" dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend
[TRACE 21:08:23] starbase_utils::fs  Traversing upwards to find a file  file=".git" dir=/Users/kasama.chenkaow/kasama/ssg_monorepo
[DEBUG 21:08:23] moon_process::command_inspector  Running command git remote get-url origin  env_vars={} working_dir="/Users/kasama.chenkaow/kasama/ssg_monorepo"
[TRACE 21:08:23] starbase_utils::fs  Traversing upwards to find a file  file=".moon" dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/versionCheck.json
[DEBUG 21:08:23] log  Creating project graph  log.target="moon:project-graph" log.module_path="moon_project_graph::project_builder" log.file="crates/core/project-graph/src/project_builder.rs" log.line=52
[TRACE 21:08:23] log  Cache hit for ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projects.json, reading  log.target="moon:cache:item" log.module_path="moon_cache::items::projects_state" log.file="crates/core/cache/src/items/projects_state.rs" log.line=26
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projects.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projects.json
[DEBUG 21:08:23] log  Loading names (aliases) from project package.json's  log.target="moon:node-platform" log.module_path="moon_node_platform::platform" log.file="crates/node/platform/src/platform.rs" log.line=116
[TRACE 21:08:23] log  Loading ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/package.json  log.target="moon:lang:config" log.module_path="moon_node_lang::package_json" log.file="crates/node/lang/src/package_json.rs" log.line=12
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/package.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/package.json
[TRACE 21:08:23] log  Writing hash manifest ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes/62ce6c7b562ff2fd87a4f30b8fda976b35a94a1f092691dc9b8b4c3dc9cd6795.json  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=137
[TRACE 21:08:23] starbase_utils::json  Stringifying JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes/62ce6c7b562ff2fd87a4f30b8fda976b35a94a1f092691dc9b8b4c3dc9cd6795.json
[TRACE 21:08:23] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes/62ce6c7b562ff2fd87a4f30b8fda976b35a94a1f092691dc9b8b4c3dc9cd6795.json
[DEBUG 21:08:23] log  Generated hash 62ce6c7b562ff2fd87a4f30b8fda976b35a94a1f092691dc9b8b4c3dc9cd6795 for project graph  log.target="moon:project-graph" log.module_path="moon_project_graph::project_builder" log.file="crates/core/project-graph/src/project_builder.rs" log.line=640
[TRACE 21:08:23] log  Writing cache ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projects.json  log.target="moon:cache:item" log.module_path="moon_cache::items::projects_state" log.file="crates/core/cache/src/items/projects_state.rs" log.line=26
[TRACE 21:08:23] starbase_utils::json  Stringifying JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projects.json
[TRACE 21:08:23] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projects.json
[DEBUG 21:08:23] log  Loading project graph with 1 projects from cache  log.target="moon:project-graph" log.module_path="moon_project_graph::project_builder" log.file="crates/core/project-graph/src/project_builder.rs" log.line=656
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projectGraph.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/projectGraph.json
[DEBUG 21:08:23] log  Querying for touched files  log.target="moon:query:touched-files" log.module_path="moon_cli::queries::touched_files" log.file="crates/cli/src/queries/touched_files.rs" log.line=37
[DEBUG 21:08:23] moon_process::command_inspector  Running command git branch --show-current  env_vars={} working_dir="/Users/kasama.chenkaow/kasama/ssg_monorepo"
[TRACE 21:08:23] log  Against locally touched  log.target="moon:query:touched-files" log.module_path="moon_cli::queries::touched_files" log.file="crates/cli/src/queries/touched_files.rs" log.line=76
[DEBUG 21:08:23] moon_process::command_inspector  Running command git status --porcelain --untracked-files -z  env_vars={} working_dir="/Users/kasama.chenkaow/kasama/ssg_monorepo"
[DEBUG 21:08:23] log  Filtering based on touched status "all"  log.target="moon:query:touched-files" log.module_path="moon_cli::queries::touched_files" log.file="crates/cli/src/queries/touched_files.rs" log.line=85
[DEBUG 21:08:23] log  Creating dependency graph  log.target="moon:dep-graph" log.module_path="moon_dep_graph::dep_builder" log.file="crates/core/dep-graph/src/dep_builder.rs" log.line=35
[TRACE 21:08:23] log  Adding RunTarget(vgef:test) to graph  log.target="moon:dep-graph" log.module_path="moon_dep_graph::dep_builder" log.file="crates/core/dep-graph/src/dep_builder.rs" log.line=310
[TRACE 21:08:23] log  Loading ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/package.json  log.target="moon:lang:config" log.module_path="moon_node_lang::package_json" log.file="crates/node/lang/src/package_json.rs" log.line=12
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/package.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/package.json
[DEBUG 21:08:23] log  Project vgef not within dependency workspaces, will be handled externally  log.target="moon:dep-graph" log.module_path="moon_dep_graph::dep_builder" log.file="crates/core/dep-graph/src/dep_builder.rs" log.line=110
[TRACE 21:08:23] log  Adding InstallNodeDepsInProject(18.0.0, vgef) to graph  log.target="moon:dep-graph" log.module_path="moon_dep_graph::dep_builder" log.file="crates/core/dep-graph/src/dep_builder.rs" log.line=135
[TRACE 21:08:23] log  Adding SetupNodeTool(18.0.0) to graph  log.target="moon:dep-graph" log.module_path="moon_dep_graph::dep_builder" log.file="crates/core/dep-graph/src/dep_builder.rs" log.line=417
[TRACE 21:08:23] log  Adding SyncNodeProject(vgef) to graph  log.target="moon:dep-graph" log.module_path="moon_dep_graph::dep_builder" log.file="crates/core/dep-graph/src/dep_builder.rs" log.line=434
[DEBUG 21:08:23] log  Running 4 actions across 3 batches  log.target="moon:action-pipeline" log.module_path="moon_action_pipeline::pipeline" log.file="crates/core/action-pipeline/src/pipeline.rs" log.line=93
[TRACE 21:08:23] log  Running 1 actions in batch 1  log.target="moon:action-pipeline:batch:1" log.module_path="moon_action_pipeline::pipeline" log.file="crates/core/action-pipeline/src/pipeline.rs" log.line=116
[TRACE 21:08:23] log  Processing action SetupNodeTool(18.0.0)  log.target="moon:action-pipeline:batch:1:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=36
[DEBUG 21:08:23] log  Setting up Node.js 18.0.0 toolchain  log.target="moon:action:setup-tool" log.module_path="moon_action_pipeline::actions::setup_tool" log.file="crates/core/action-pipeline/src/actions/setup_tool.rs" log.line=26
[TRACE 21:08:23] log  Cache hit for ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/toolNode-18.0.0.json, reading  log.target="moon:cache:item" log.module_path="moon_cache::items::tool_state" log.file="crates/core/cache/src/items/tool_state.rs" log.line=22
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/toolNode-18.0.0.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/toolNode-18.0.0.json
[DEBUG 21:08:23] proto_node::resolve  Resolving a semantic version for "18.0.0"
[TRACE 21:08:23] starbase_utils::fs:load_versions_manifest  Reading file metadata  file=/Users/kasama.chenkaow/.proto/temp/3d16d1693e83828f98bae178f181d5a01103b7f222db27cdeaec9b4950e951d7.json
[DEBUG 21:08:23] proto_core::resolver:load_versions_manifest  Loading versions manifest from local cache  cache_file=/Users/kasama.chenkaow/.proto/temp/3d16d1693e83828f98bae178f181d5a01103b7f222db27cdeaec9b4950e951d7.json
[TRACE 21:08:23] starbase_utils::fs:load_versions_manifest  Reading file  file=/Users/kasama.chenkaow/.proto/temp/3d16d1693e83828f98bae178f181d5a01103b7f222db27cdeaec9b4950e951d7.json
[TRACE 21:08:23] starbase_utils::json:load_versions_manifest  Parsing JSON  file=/Users/kasama.chenkaow/.proto/temp/3d16d1693e83828f98bae178f181d5a01103b7f222db27cdeaec9b4950e951d7.json
[DEBUG 21:08:23] proto_core::manifest:load  Loading manifest  file=/Users/kasama.chenkaow/.proto/tools/node/manifest.json
[TRACE 21:08:23] starbase_utils::fs:load  Opening file  file=/Users/kasama.chenkaow/.proto/tools/node/manifest.json
[DEBUG 21:08:23] proto_node::resolve  Resolved to 18.0.0
[DEBUG 21:08:23] proto_core::tool  Checking if tool is installed  install_dir=/Users/kasama.chenkaow/.proto/tools/node/18.0.0
[DEBUG 21:08:23] proto_core::tool  Tool has already been installed  install_dir=/Users/kasama.chenkaow/.proto/tools/node/18.0.0
[TRACE 21:08:23] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/.proto/bin/node
[TRACE 21:08:23] starbase_utils::fs  Updating file permissions  file=/Users/kasama.chenkaow/.proto/bin/node mode=493
[DEBUG 21:08:23] proto_core::shimmer  Created shim  file=/Users/kasama.chenkaow/.proto/bin/node
[TRACE 21:08:23] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/.proto/bin/npx
[TRACE 21:08:23] starbase_utils::fs  Updating file permissions  file=/Users/kasama.chenkaow/.proto/bin/npx mode=493
[DEBUG 21:08:23] proto_core::shimmer  Created shim  file=/Users/kasama.chenkaow/.proto/bin/npx
[DEBUG 21:08:23] log  Node.js has already been setup  log.target="moon_node_tool::node_tool" log.module_path="moon_node_tool::node_tool" log.file="crates/node/tool/src/node_tool.rs" log.line=166
[TRACE 21:08:23] log  Writing cache ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/toolNode-18.0.0.json  log.target="moon:cache:item" log.module_path="moon_cache::items::tool_state" log.file="crates/core/cache/src/items/tool_state.rs" log.line=22
[TRACE 21:08:23] starbase_utils::json  Stringifying JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/toolNode-18.0.0.json
[TRACE 21:08:23] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/toolNode-18.0.0.json
[TRACE 21:08:23] log  Processed action SetupNodeTool(18.0.0) in 3.954667ms  log.target="moon:action-pipeline:batch:1:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=214
[TRACE 21:08:23] log  Running 2 actions in batch 2  log.target="moon:action-pipeline:batch:2" log.module_path="moon_action_pipeline::pipeline" log.file="crates/core/action-pipeline/src/pipeline.rs" log.line=116
[TRACE 21:08:23] log  Processing action InstallNodeDepsInProject(18.0.0, vgef)  log.target="moon:action-pipeline:batch:2:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=36
[TRACE 21:08:23] starbase_utils::fs  Reading file metadata  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/yarn.lock
[TRACE 21:08:23] log  Cache hit for ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/depsNode.json, reading  log.target="moon:cache:item" log.module_path="moon_cache::items::dependencies_state" log.file="crates/core/cache/src/items/dependencies_state.rs" log.line=21
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/depsNode.json
[TRACE 21:08:23] log  Processing action SyncNodeProject(vgef)  log.target="moon:action-pipeline:batch:2:2" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=36
[DEBUG 21:08:23] log  Syncing project vgef  log.target="moon:action:sync-project" log.module_path="moon_action_pipeline::actions::sync_project" log.file="crates/core/action-pipeline/src/actions/sync_project.rs" log.line=32
[TRACE 21:08:23] log  Writing runfile ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/runfile.json  log.target="moon:cache:runfile" log.module_path="moon_cache::runfiles" log.file="crates/core/cache/src/runfiles.rs" log.line=18
[TRACE 21:08:23] starbase_utils::json  Stringifying JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/runfile.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/depsNode.json
[TRACE 21:08:23] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/runfile.json
[DEBUG 21:08:23] log  Lockfile has not changed since last install, skipping install  log.target="moon:action:install-deps" log.module_path="moon_action_pipeline::actions::install_deps" log.file="crates/core/action-pipeline/src/actions/install_deps.rs" log.line=160
[TRACE 21:08:23] log  Processed action InstallNodeDepsInProject(18.0.0, vgef) in 364.834µs  log.target="moon:action-pipeline:batch:2:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=214
[TRACE 21:08:23] log  Loading ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/tsconfig.json  log.target="moon:lang:config" log.module_path="moon_typescript_lang::tsconfig" log.file="crates/typescript/lang/src/tsconfig.rs" log.line=12
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/tsconfig.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/tsconfig.json
[TRACE 21:08:23] log  Loading ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/tsconfig.json  log.target="moon:lang:config" log.module_path="moon_typescript_lang::tsconfig" log.file="crates/typescript/lang/src/tsconfig.rs" log.line=12
[TRACE 21:08:23] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/tsconfig.json
[TRACE 21:08:23] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/tsconfig.json
[TRACE 21:08:23] log  Processed action SyncNodeProject(vgef) in 760.5µs  log.target="moon:action-pipeline:batch:2:2" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=214
[TRACE 21:08:23] log  Running 1 actions in batch 3  log.target="moon:action-pipeline:batch:3" log.module_path="moon_action_pipeline::pipeline" log.file="crates/core/action-pipeline/src/pipeline.rs" log.line=116
[TRACE 21:08:23] log  Processing action RunTarget(vgef:test)  log.target="moon:action-pipeline:batch:3:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=36
[TRACE 21:08:23] log  Cache hit for ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/test/lastRun.json, reading  log.target="moon:cache:item" log.module_path="moon_cache::items::run_targ
et_state" log.file="crates/core/cache/src/items/run_target_state.rs" log.line=27
nkaow/kasama/ssg_monorepo"
[TRACE 21:23:27] starbase_utils::fs  Traversing upwards to find a file  file="yarn.lock" dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder
[TRACE 21:23:27] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/yarn.lock
[TRACE 21:23:27] log  Loading ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/tsconfig.json  log.target="moon:lang:config" log.module_path="moon_typescript_lang::tsconfig" log.file="crates/typescript/lang/src/tsconfig.rs" log.line=12
[TRACE 21:23:27] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/tsconfig.json
[TRACE 21:23:27] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/tsconfig.json
[TRACE 21:23:27] log  Loading ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/tsconfig.json  log.target="moon:lang:config" log.module_path="moon_typescript_lang::tsconfig" log.file="crates/typescript/lang/src/tsconfig.rs" log.line=12
[TRACE 21:23:27] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/tsconfig.json
[TRACE 21:23:27] starbase_utils::json  Parsing JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/visitor-group-event-forwarder/tsconfig.json
[DEBUG 21:23:27] log  Generated hash 63ccb1c04153167e22f271387f2597350984d7be660b4e22ac3cc1dff413cbcb for target vgef:test  log.target="moon:runner" log.module_path="moon_runner::runner" log.file="crates/core/runner/src/runner.rs" log.line=434
[DEBUG 21:23:27] log  Cache hit for hash 63ccb1c04153167e22f271387f2597350984d7be660b4e22ac3cc1dff413cbcb, reusing previous build  log.target="moon:runner" log.module_path="moon_runner::runner" log.file="crates/core/runner/src/runner.rs" log.line=454
[TRACE 21:23:27] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/test/stdout.log
[TRACE 21:23:27] starbase_utils::fs  Reading file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/states/vgef/test/stderr.log
watchman warning:  Recrawled this watch 3 times, most recently because:
MustScanSubDirs UserDroppedTo resolve, please review the information on
https://facebook.github.io/watchman/docs/troubleshooting.html#recrawl
To clear this warning, run:
`watchman watch-del '/Users/kasama.chenkaow/kasama/ssg_monorepo' ; watchman watch-project '/Users/kasama.chenkaow/kasama/ssg_monorepo'`

 PASS  __tests__/main.test.ts
  handler
    ✓ should publish a message to SNS topic for each record in the event (10 ms)
    ✓ should throw an error if SNS_TOPIC environment variable is not set (5 ms)
    ✓ should not publish a message if the record does not have a NewImage property (3 ms)
    ✓ should throw an error if the send method fails (1 ms)

Test Suites: 1 passed, 1 total
Tests:       4 passed, 4 total
Snapshots:   0 total
Time:        1.331 s, estimated 2 s
Ran all test suites.
[TRACE 21:23:27] log  Processed action RunTarget(vgef:test) in 25.406625ms  log.target="moon:action-pipeline:batch:3:1" log.module_path="moon_action_pipeline::processor" log.file="crates/core/action-pipeline/src/processor.rs" log.line=210
[DEBUG 21:23:27] log  Finished running 4 actions in 31.718125ms  log.target="moon:action-pipeline" log.module_path="moon_action_pipeline::pipeline" log.file="crates/core/action-pipeline/src/pipeline.rs" log.line=213
[TRACE 21:23:27] log  Cleaning up and deleting stale cache older than "7 days"  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=109
[TRACE 21:23:27] starbase_utils::fs  Removing stale contents from directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes
[TRACE 21:23:27] starbase_utils::fs  Reading directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/hashes
[TRACE 21:23:27] starbase_utils::fs  Removing stale contents from directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/outputs
[TRACE 21:23:27] starbase_utils::fs  Reading directory  dir=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/outputs
[TRACE 21:23:27] log  Deleted 0 files and saved 0 bytes  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=121
[TRACE 21:23:27] log  Writing report ~/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/runReport.json  log.target="moon:cache" log.module_path="moon_cache::engine" log.file="crates/core/cache/src/engine.rs" log.line=151
[TRACE 21:23:27] starbase_utils::json  Stringifying JSON  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/runReport.json
[TRACE 21:23:27] starbase_utils::fs  Writing file  file=/Users/kasama.chenkaow/kasama/ssg_monorepo/async_backend/lambdaHandlers/.moon/cache/runReport.json
admin-ssg commented 1 year ago

Looks like it's because of this

[DEBUG 21:23:27] log  Generated hash 63ccb1c04153167e22f271387f2597350984d7be660b4e22ac3cc1dff413cbcb for target vgef:test  log.target="moon:runner" log.module_path="moon_runner::runner" log.file="crates/core/runner/src/runner.rs" log.line=434
[DEBUG 21:23:27] log  Cache hit for hash 63ccb1c04153167e22f271387f2597350984d7be660b4e22ac3cc1dff413cbcb, reusing previous build  log.target="moon:runner" log.module_path="moon_runner::runner" log.file="crates/core/runner/src/runner.rs" log.line=454

my guess is that it generate the cache based on this command purely vgef:test hence the same hash hit

my questions are

  1. Isn't according to this doc it shouldn't cache by default if I don't specify any inputs?
  2. I also tried to put the inputs (refers to the test file), but it's still using the same cache, what did I do wrong?
milesj commented 1 year ago

@admin-ssg Let's debug a bit. What's the contest of moon query hash 63ccb1c04153167e22f271387f2597350984d7be660b4e22ac3cc1dff413cbcb? That will show you what's being included in the hash.

As for inputs, by default its **/*, so changing files should trigger an affected change, if the file is in the current project. If the file is outside the project, it won't trigger a change.

admin-ssg commented 1 year ago

Nice! let's debug it! These are the content

[
  {
    "command": "yarn",
    "args": [
      "test:unit"
    ],
    "deps": {},
    "envVars": {},
    "inputs": {},
    "outputs": [],
    "projectDeps": [],
    "target": "vgef:test",
    "version": "1"
  },
  {
    "nodeVersion": "18.16.0",
    "dependencies": {
      "@aws-sdk/client-sns": [
        "sha512-7n+MV+2+ed/ZZPgHRfEmdsK+HZ8KCzw8mzE8XEcLZAQ2ihsVx8+BYql7PPMx864P3XDE0w943C2+nTHil5bKmw=="
      ],
      "@aws-sdk/util-dynamodb": [
        "sha512-k8DqABFcGjWZMIGGMq+7k6y7NrrzmaxSJZivwvTxuZ7TZYFBms2eWTvUgiAUZAalG5rG6PHrEMHzR5KYAh/8uA=="
      ],
      "@moonrepo/cli": [
        "^1.6.1"
      ],
      "@types/aws-lambda": [
        "sha512-kCZuFXKLV3y8NjSoaD5+qKTpRWvPz3uh3W/u1uwlw3Mg+MtaStg1NWgjAwUXo/VJDb6n6KF1ljykFNlNwEJ53Q=="
      ],
      "@types/jest": [
        "sha512-tEuVcHrpaixS36w7hpsfLBLpjtMRJUE09/MHXn923LOVojDwyC14cWcfc0rDs0VEfUyYmt/+iX1kxxp+gZMcaQ=="
      ],
      "@types/node": [
        "sha512-DqJociPbZP1lbZ5SQPk4oag6W7AyaGMO6gSfRwq3PWl4PXTwJpRQJhDq4W0kzrg3w6tJ1SwlvGZ5uKFHY13LIg==",
        "sha512-uZRomboV1vBL61EBXneL4j9/hEn+1Yqa4LQdpGrKmXFyJmVfWc9JV9+yb2AlnOnuaDnb2PDO3hC6/LKmzJxP1A=="
      ],
      "@typescript-eslint/eslint-plugin": [
        "~5.59"
      ],
      "@typescript-eslint/parser": [
        "~5.59"
      ],
      "aws-lambda": [
        "sha512-9GNFMRrEMG5y3Jvv+V4azWvc+qNWdWLTjDdhf/zgMlz8haaaLWv0xeAIWxz9PuWUBawsVxy0zZotjCdR3Xq+2w=="
      ],
      "eslint": [
        "~8.38"
      ],
      "eslint-config-prettier": [
        "~8.8"
      ],
      "eslint-plugin-jest": [
        "~27.2"
      ],
      "eslint-plugin-prettier": [
        "^4.2.1"
      ],
      "jest": [
        "sha512-juMg3he2uru1QoXX078zTa7pO85QyB9xajZc6bU+d9yEGwrKX6+vGmJQ3UdVZsvTEUARIdObzH68QItim6OSSQ=="
      ],
      "prettier": [
        "~2.8"
      ],
      "rimraf": [
        "sha512-OfFZdwtd3lZ+XZzYP/6gTACubwFcHdLRqS9UX3UwpU2dnGQYkPFISRwvM3w9IiB2w7bW5qGo/uAwE4SmXXSKvg=="
      ],
      "ts-api-utils": [
        "sha512-YKJeSx39n0mMk+hrpyHKyTgxA3s7Pz/j1cXYR+t8HcwwZupzOR5xDGKnOEw3gmLaUeFUQt3FJD39AH9Ajn/mdA=="
      ],
      "ts-jest": [
        "sha512-ZhNr7Z4PcYa+JjMl62ir+zPiNJfXJN6E8hSLnaUKhOgqcn8vb3e537cpkd0FuAfRK3sR1LSqM1MOhliXNgOFPA=="
      ],
      "tslib": [
        "sha512-5svOrSA2w3iGFDs1HibEVBGbDrAY82bFQ3HZ3ixB+88nsbsWQoKqDRb5UBYAUPEzbBn6dAp5gRNXglySbx1MlA==",
        "sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg=="
      ],
      "typescript": [
        "sha512-cW9T5W9xY37cc+jfEnaUvX91foxtHkza3Nw3wkoF4sSlKn0MONdkdEndig/qPBWXNkmplh3NzayQzCiHM4/hqw=="
      ]
    }
  },
  {
    "compilerOptions": {}
  }
]

Hmm if the default is **/*, then the sentence in doc says Our task above works, but isn't very efficient as it always runs, regardless of what has changed since the last time it has ran. is not very accurate right?

And also my test file is in the vgef project, that project doesn't have any .moon.yml though, I also specify tasks in the workspace level

milesj commented 1 year ago

@admin-ssg Ok I think this may be a bug, that inputs list should have something. It should look like this:

[
  {
    "command": "packemon",
    "args": [
      "--addExports",
      "--addFiles",
      "--declaration",
      "build"
    ],
    "deps": {},
    "envVars": {
      "NODE_ENV": "production"
    },
    "inputs": {
      ".moon/tasks/node.yml": "5f89768625d0626d7747eb17a2fdcd0d268109c8",
      ".moon/toolchain.yml": "a14fecbdbe15932846ab52c9cfd5d631033b54e2",
      ".moon/workspace.yml": "9c3b1defa1bc11b05b690bd7ad2101d609254ae9",
      "packages/types/package.json": "821042b356f40b1d9acf9514ebc9ffe6cb6dbe7e",
      "packages/types/src/common.ts": "918f6d85890ac66043c1c514a5a512cab6cf69d1",
      "packages/types/src/events.ts": "a7fd3ad88251f533b653db44bff93031ae8c2629",
      "packages/types/src/index.ts": "5d4752c198fca80c499278b6c47cd8b838b7a723",
      "packages/types/src/pipeline.ts": "97b545e4cc58b9b74336505cdc4d66d6c2995f6d",
      "packages/types/src/project-config.ts": "c91b4d8c7a934f202500a29ed98ae327427b0c30",
      "packages/types/src/project.ts": "4fde2a8ff33e0f48f44ce6ef78d3549ccb7d9404",
      "packages/types/src/template-config.ts": "2ab922bee6572bcc7be51779abdd866a7486afe8",
      "packages/types/src/toolchain-config.ts": "b3cec36ea25d56de056e8a3b77766a8c68a87829",
      "packages/types/src/workspace-config.ts": "01c074453e16559259f5113bef39c56ff546ddf3",
      "packages/types/tsconfig.cjs.json": "0765ec64cc5ab9c3896973696809b983e2bea841",
      "packages/types/tsconfig.json": "67b3494e5c4e9eebf9fbe77a967be1e5d385d46f",
      "tsconfig.options.json": "720b9c8f7dc0b6c38c33f4a6e26fff5fb5fc95d0"
    },
    "outputs": [
      "cjs"
    ],
    "projectDeps": [],
    "target": "types:build",
    "version": "1"
  },
]

If you add inputs: ['**/*'] does that fix the problem? There may be an inheritance problem here.

Also what does moon task vgef:test --json show?

admin-ssg commented 1 year ago

I added the inputs: ['**/*'] but it's still the same

here's the json

{
  "args": [
    "test:unit"
  ],
  "command": "yarn",
  "deps": [],
  "env": {},
  "flags": [],
  "globalInputs": [
    "/.moon/*.yml"
  ],
  "id": "test",
  "inputs": [
    "**/*"
  ],
  "inputGlobs": [
    "visitor-group-event-forwarder/**/*",
    ".moon/*.yml"
  ],
  "inputPaths": [],
  "inputVars": [],
  "options": {
    "affectedFiles": null,
    "cache": true,
    "envFile": null,
    "mergeArgs": "append",
    "mergeDeps": "append",
    "mergeEnv": "append",
    "mergeInputs": "append",
    "mergeOutputs": "append",
    "outputStyle": null,
    "persistent": false,
    "retryCount": 0,
    "runDepsInParallel": true,
    "runInCi": true,
    "runFromWorkspaceRoot": false,
    "shell": true
  },
  "outputs": [],
  "outputGlobs": [],
  "outputPaths": [],
  "platform": "node",
  "target": "vgef:test",
  "type": "test"
}
milesj commented 1 year ago

@admin-ssg Created an example of this but unable to reproduce, something else must be going on. When you edit a file in the project, what's the output of git status --porcelain --untracked-files -z?

admin-ssg commented 1 year ago

It says

❯ git status --porcelain --untracked-files -z
 M async_backend/lambdaHandlers/package.json M async_backend/lambdaHandlers/visitor-group-event-forwarder/src/main.test.ts%
milesj commented 1 year ago

@admin-ssg Everything looks good so far, this is a bit weird.

Hard to really say without interacting with it myself. If you're open to adding me as a contributor, I can quickly take a look at the problem.

admin-ssg commented 1 year ago

@milesj I added you, it'll be in https://github.com/Secure-Smart-Guard/ssg_monorepo/tree/39-dev-add-terraform-and-populate-events-to-our-own-serverless-opensearch-cluster/async_backend/lambdaHandlers

in the branch 39-dev-add-terraform-and-populate-events-to-our-own-serverless-opensearch-cluster

milesj commented 1 year ago

@admin-ssg Awesome thanks, was able to reproduce.

It looks like the problem is that some of the git commands are running in the wrong directory. I believe it's because the .git and .moon folders are at different depths. Should be an easy fix.

admin-ssg commented 1 year ago

Ah okay, nice that you figured out!

milesj commented 1 year ago

@admin-ssg Just released 1.7.3 that should fix this, let me know if there are more issues around it.

admin-ssg commented 1 year ago

@milesj It works, thanks a lot for your helps!