Closed dpastoor closed 5 months ago
running a local test shows a some sensitivity related failures exist in the test suite irrespective of this. We'll need to do some cleanup and add CI before we do too much automated gating.
> devtools::test()
ℹ Testing dvs
✔ | F W S OK | Context
⠏ | 0 | dvs_add Initialized empty Git repository in /tmp/RtmphkayHt/projects/add_single_file/.git/
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/data/dvs/add_single_file
⠴ | 1 5 | dvs_add [1] "Cleaning up /tmp/RtmphkayHt..."
Initialized empty Git repository in /tmp/RtmphkayHt/projects/add_multiple_file/.git/
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/data/dvs/add_multiple_file
[1] "Cleaning up /tmp/RtmphkayHt..."
Initialized empty Git repository in /tmp/RtmphkayHt/projects/add_two_diff_directories/.git/
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/data/dvs/add_two_diff_directories
[1] "Cleaning up /tmp/RtmphkayHt..."
Initialized empty Git repository in /tmp/RtmphkayHt/projects/no_add_meta_gitignore/.git/
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/data/dvs/no_add_meta_gitignore
⠙ | 1 31 | dvs_add [1] "Cleaning up /tmp/RtmphkayHt..."
Initialized empty Git repository in /tmp/RtmphkayHt/projects/file_dne/.git/
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/data/dvs/file_dne
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("at least one inputted file not found: /tmp/RtmphkayHt/projects/file_dne/data/derived/no_such_file.csv not found: No such file or directory (os error 2)")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at src/lib.rs:70:1:
explicit panic
[1] "Cleaning up /tmp/RtmphkayHt..."
Initialized empty Git repository in /tmp/RtmphkayHt/projects/no_repo/.git/
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/data/dvs/no_repo
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("git repository not found: could not find git repo root; make sure you're in an active git repository: .")
thread '<unnamed>' panicked at src/lib.rs:70:1:
explicit panic
[1] "Cleaning up /tmp/RtmphkayHt..."
Initialized empty Git repository in /tmp/RtmphkayHt/projects/no_init/.git/
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("configuration file not found (dvs.yaml): could not load configuration file, i.e. no dvs.yaml in directory; be sure to initiate dvs: No such file or directory (os error 2)")
thread '<unnamed>' panicked at src/lib.rs:70:1:
explicit panic
[1] "Cleaning up /tmp/RtmphkayHt..."
✖ | 1 48 | dvs_add
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Failure (test-dvs_add.R:38:3): can add a single file
is_near_time(dvs_json$time_stamp) is not TRUE
`actual`:
`expected`: TRUE
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
⠏ | 0 | dvs_get thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("metadata file not found for at least one file: missing for file")
thread '<unnamed>' panicked at src/lib.rs:526:1:
explicit panic
[1] "git_dir /tmp/RtmphkayHt/file-error/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/file-error_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/file-error"
[1] "/tmp/RtmphkayHt/file-error/file14b17e430082ca.txt"
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("metadata file not found for at least one file: missing for /tmp/RtmphkayHt/file-error/file14b17e430082ca.txt")
thread '<unnamed>' panicked at src/lib.rs:526:1:
explicit panic
[1] "git_dir /tmp/RtmphkayHt/file-dne/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/file-dne_stor_dir
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("metadata file not found for at least one file: missing for dne.txt")
thread '<unnamed>' panicked at src/lib.rs:526:1:
explicit panic
[1] "git_dir /tmp/RtmphkayHt/random/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/random_stor_dir
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("metadata file not found for at least one file: missing for random")
thread '<unnamed>' panicked at src/lib.rs:526:1:
explicit panic
[1] "git_dir /tmp/RtmphkayHt/file-error/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/file-error_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/file-error"
[1] "/tmp/RtmphkayHt/file-error/file14b17e592fdf59.txt"
warning: no files were queued
[1] "git_dir /tmp/RtmphkayHt/status-init/.git"
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("metadata file not found for at least one file: missing for file")
thread '<unnamed>' panicked at src/lib.rs:526:1:
explicit panic
[1] "git_dir /tmp/RtmphkayHt/multiple-files/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/multiple-files_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/multiple-files"
[1] "git_dir /tmp/RtmphkayHt/multiple-files-glob/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/multiple-files-glob_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/multiple-files-glob"
✔ | 11 | dvs_get
⠏ | 0 | dvs_init [1] "git_dir /tmp/RtmphkayHt/first-run-init/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/first-run-init_stor_dir
[1] "git_dir /tmp/RtmphkayHt/init_second_run/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/init_second_run_stor_dir
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("project already initialized: to change initialization settings, manually update dvs.yaml in project root")
thread '<unnamed>' panicked at src/lib.rs:48:1:
explicit panic
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("git repository not found: make sure you're in an active git repository. could not find git repo root; make sure you're in an active git repository: .")
thread '<unnamed>' panicked at src/lib.rs:48:1:
explicit panic
[1] "git_dir /tmp/RtmphkayHt/no-defaults/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/no-defaults_stor_dir
✔ | 11 | dvs_init
⠏ | 0 | dvs_status [1] "git_dir /tmp/RtmphkayHt/base-case/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/base-case_stor_dir
[1] "git_dir /tmp/RtmphkayHt/no-args/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/no-args_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/no-args"
[1] "/tmp/RtmphkayHt/no-args/file14b17e2aded1e.txt"
[1] "/tmp/RtmphkayHt/no-args/file14b17e2aded1e.txt.dvs"
[1] "git_dir /tmp/RtmphkayHt/explicit-arg/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/explicit-arg_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/explicit-arg"
[1] "/tmp/RtmphkayHt/explicit-arg/file14b17e480c70d4.txt"
[1] "/tmp/RtmphkayHt/explicit-arg/file14b17e480c70d4.txt.dvs"
[1] "git_dir /tmp/RtmphkayHt/glob-1/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/glob-1_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/glob-1"
[1] "/tmp/RtmphkayHt/glob-1/file14b17eed728cb.txt"
[1] "/tmp/RtmphkayHt/glob-1/file14b17eed728cb.txt.dvs"
[1] "git_dir /tmp/RtmphkayHt/glob-2/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/glob-2_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/glob-2"
[1] "/tmp/RtmphkayHt/glob-2/file14b17e63fcddb4.txt"
[1] "/tmp/RtmphkayHt/glob-2/file14b17e63fcddb4.txt.dvs"
[1] "git_dir /tmp/RtmphkayHt/status-init/.git"
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("configuration file not found (dvs.yaml): could not load configuration file, i.e. no dvs.yaml in directory; be sure to initiate dvs: No such file or directory (os error 2)")
thread '<unnamed>' panicked at src/lib.rs:294:1:
explicit panic
thread '<unnamed>' panicked at /cluster-data/user-homes/devin/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.6.0/src/robj/into_robj.rs:64:13:
called `Result::unwrap()` on an `Err` value: Other("git repository not found: could not find git repo root; make sure you're in an active git repository: .")
thread '<unnamed>' panicked at src/lib.rs:537:1:
explicit panic
⠴ | 16 | dvs_status [1] "git_dir /tmp/RtmphkayHt/file-error/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/file-error_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/file-error"
[1] "/tmp/RtmphkayHt/file-error/file14b17e3fa3aa13.txt"
[1] "git_dir /tmp/RtmphkayHt/multiple-files/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/multiple-files_stor_dir
[1] "proj_dir: /tmp/RtmphkayHt/multiple-files"
relative_path status size
1 file14b17e6f31a347.txt current 0
2 file14b17e33018306.txt current 0
3 file14b17e2b41939e.txt error NA
blake3_checksum
1 af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262
2 af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262
3 <NA>
add_time saved_by message
1 2024-06-09T01:19:13.123Z devin
2 2024-06-09T01:19:13.123Z devin
3 <NA> <NA> <NA>
absolute_path error
1 /tmp/RtmphkayHt/multiple-files/file14b17e6f31a347.txt <NA>
2 /tmp/RtmphkayHt/multiple-files/file14b17e33018306.txt <NA>
3 /tmp/RtmphkayHt/multiple-files/file14b17e2b41939e.txt file not added
error_message
1 <NA>
2 <NA>
3 metadata file not found - add the file to dvs to get status
input
1 <NA>
2 <NA>
3 /tmp/RtmphkayHt/multiple-files/file14b17e2b41939e.txt
[1] "git_dir /tmp/RtmphkayHt/multiple-files-glob/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/multiple-files-glob_stor_dir
✔ | 25 | dvs_status
⠏ | 0 | integrated1 [1] "git_dir /tmp/RtmphkayHt/update/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/update_stor_dir
[1] "inputted time: "
[1] "current time: "
[1] "time_difference: "
[1] "inputted time: "
[1] "current time: "
[1] "time_difference: "
difftime of length 0
[1] "inputted time: 2024-06-09 01:19:13.230247"
[1] "current time: 2024-06-09 01:19:13.230247"
[1] "time_difference: 0.000227689743041992"
[1] "inputted time: 2024-06-09 01:19:13.235493"
[1] "current time: 2024-06-09 01:19:13.235493"
[1] "time_difference: 5.0004575252533"
✖ | 3 8 | integrated1 [5.1s]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Failure (test-integrated1.R:18:5): add fn updates a file
near_system_time(init_add_time) is not TRUE
`actual`:
`expected`: TRUE
Backtrace:
▆
1. ├─withr::with_dir(...) at test-integrated1.R:4:3
2. │ └─base::force(code)
3. └─testthat::expect_true(near_system_time(init_add_time)) at test-integrated1.R:18:5
Failure (test-integrated1.R:43:5): add fn updates a file
near_system_time(update_time) is not TRUE
`actual`:
`expected`: TRUE
Backtrace:
▆
1. ├─withr::with_dir(...) at test-integrated1.R:4:3
2. │ └─base::force(code)
3. └─testthat::expect_true(near_system_time(update_time)) at test-integrated1.R:43:5
Failure (test-integrated1.R:47:5): add fn updates a file
init_add_time < update_time is not TRUE
`actual`:
`expected`: TRUE
Backtrace:
▆
1. ├─withr::with_dir(...) at test-integrated1.R:4:3
2. │ └─base::force(code)
3. └─testthat::expect_true(init_add_time < update_time) at test-integrated1.R:47:5
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
⠏ | 0 | integrated2 [1] "git_dir /tmp/RtmphkayHt/update/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/update_stor_dir
✔ | 7 | integrated2
⠏ | 0 | integrated3 [1] "git_dir /tmp/RtmphkayHt/no-op/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/no-op_stor_dir
✔ | 5 | integrated3
⠏ | 0 | integrated4 [1] "git_dir /tmp/RtmphkayHt/update/.git"
storage directory doesn't exist
creating storage directory...
initialized storage directory: /tmp/RtmphkayHt/update_stor_dir
✔ | 7 | integrated4
══ Results ═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Duration: 6.5 s
── Failed tests ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Failure (test-dvs_add.R:38:3): can add a single file
is_near_time(dvs_json$time_stamp) is not TRUE
`actual`:
`expected`: TRUE
Failure (test-integrated1.R:18:5): add fn updates a file
near_system_time(init_add_time) is not TRUE
`actual`:
`expected`: TRUE
Backtrace:
▆
1. ├─withr::with_dir(...) at test-integrated1.R:4:3
2. │ └─base::force(code)
3. └─testthat::expect_true(near_system_time(init_add_time)) at test-integrated1.R:18:5
Failure (test-integrated1.R:43:5): add fn updates a file
near_system_time(update_time) is not TRUE
`actual`:
`expected`: TRUE
Backtrace:
▆
1. ├─withr::with_dir(...) at test-integrated1.R:4:3
2. │ └─base::force(code)
3. └─testthat::expect_true(near_system_time(update_time)) at test-integrated1.R:43:5
Failure (test-integrated1.R:47:5): add fn updates a file
init_add_time < update_time is not TRUE
`actual`:
`expected`: TRUE
Backtrace:
▆
1. ├─withr::with_dir(...) at test-integrated1.R:4:3
2. │ └─base::force(code)
3. └─testthat::expect_true(init_add_time < update_time) at test-integrated1.R:47:5
[ FAIL 4 | WARN 0 | SKIP 0 | PASS 122 ]
This PR makes some breaking changes to the output formats to simplify the naming a bit.
we can simplify file_size_bytes to just size as we can document that the number corresponds to bytes + that is the only relevant. It also aligns with the common use of size in other linux utilities plus in
fs
for functions likedir_info()
.For the timestamp, this was already an unintentional breaking change from the storage spec from the v2 go implementation, which used timestamp instead of time_stamp. The time stamp is an ambiguous name, and therefore we are going to rename it to
add_time
similar to other time related modifiers often present such as modification_time, creation_time, etc for normal files.