Rperform R CMD Check

[EngineerDanny]

This PR creates a Github Action for setting up R, Rperform and its dependencies on the remote machine. The test is carried out on the stringr package.

The minimal test is to run Rperform on this package successfully.

[EngineerDanny]

Implementation Review

The main focus is to be able to run Rperform on other packages(eg. stringr ). On this repository, there will be a minimal testing of the package itself. The Github Action tests will be done on the repositories of other packages which I will fork. For conveniences, I will proceed the testing with the stringr package here.

[EngineerDanny]

The Rperform package has been failing R CMD checks.

As it can be seen, all the tests passed except running r-lib/actions/check-r-package@v2 After running the package check locally, even though the test didn't explicitly give me an error, it failed.

The only warning was :

checking Rd cross-references ... WARNING Missing link or links in documentation object 'Rperform-package.Rd': ‘[git2r:git2r-package]{git2r}’

See section 'Cross-references' in the 'Writing R Extensions' manual.

I fixed the warning by inserting the git2r official github link.

[EngineerDanny]

Initially, the issue was invalid link in the test-repo-metrics.Rfile. I found out that the hadley/stringr link does not exist. Even though, when you open the link you will be redirected to the actual repository which is the tidyverse/stringr .

My test results were inconsistent (Failing more times and occassionally passing) due to this issue. Therefore, I changed the github link to tidyverse/stringr

[EngineerDanny]

Although the issue with the link was fixed and R CMD Check on my local machine passed, the GitHub CI action still failed. So I started to dig into the tidyverse/stringr package tests.

Inside the https://github.com/tidyverse/stringr/blob/main/tests/testthat/test-dup.r file, there are three tests which are ran internally.

As shown above, all the other tests passed except the uses tidyverse recycling rules test. This led to my conclusion that, this may be an internal issue with the tidyverse/stringr package.

The first two tests however, could be used in checking Rperform.

[EngineerDanny]

After several debugging and testing, I finally found the final error that was causing the Rperform R CMD Check test to fail on GitHub CI. It turns out that during my adventures and explorations, I mistakenly created a file on the master branch
👉 [tests/testthat/test-plot-metrics.R].

As recorded from the logs above, in the file, there is an issue with the str_c function because it does not exist. Therefore, I have removed this file since it's not needed.

Now, all checks have passed ✅.

[tdhock]

also i'm not sure if this is the case for the stringr that you are using, but it would be good to have a test case in which there is a clear slowdown/speedup that can be observed, for example this branch/pr probly has a slow down https://github.com/tdhock/binsegRcpp/pull/10

[EngineerDanny]

My new commits to this repo fix most of the issues raised.

1. Indentation inconsistency in the R files have been fixed by using R studio's built-in indentation. (1)
2. Various binary files and directories have been added to the .gitignore file. (2)
3. Named reference approach has been adopted to reference list members by names instead of numeric indexes. (3)

[EngineerDanny]

@tdhock My recent tests on PR/branch with Rperform show that your assumption is right. Merging this PR made the code consistently slower.

Since they were on different branches, I ran the plot_branchmetrics(test_path = "tests/testthat/test-CRAN.R", metric = "time", branch1 = "l1loss", branch2 = "master", save_data = T, save_plots = F) function and observed the results.

I noticed that, the result was not depicted clearly on the graph. #4 However, the time_frame data set that was saved had 254 observations and 8 variables. This is the data 👉 time_frame.csv

[EngineerDanny]

BREAKING DOWN THE RESULTS The results were consistent. After the PR was merged, there was a clear slowdown that was observed.

1. The first test was "one data point has zero loss"

Average Time Taken Before PR : 0.009315 seconds. After merging PR : 0.0180096 seconds (Approximately 2 times increase)

2. "equal split cost is ok" test

Average Time Taken Before PR : 0.0351191 seconds. After merging PR : 0.057926 seconds (Approximately 1.6 times increase)

3. "error for 0 data" test

Average Time Taken Before PR : 0.0072106 seconds. After merging PR : 0.0352905 seconds (Approximately 5 times increase)

The list goes on and on, showing a consistent increase in time after merging the PR.