Error in post_plan: creates milestones but not issues and doesn't return data frame

[sctyner]

Sorry for this but I'm going insane. I'm sure it's just some dumb little YAML thing like an extra space or something but I cannot figure out for the life of me what is going on.

I copy-pasted your example from here to make a new plan. It worked great. (See okay below.)

Then I tried making my own, and totally failed. Well, not totally: the milestones were there but not the issues. There was an error message: #> Error in do.call(post_local, x): second argument must be a list and the data frame didn't print off at the end.

Then, I tried using the template and copy-pasting. (See broken below.) It STILL didn't work. (And now I have an additional error: #> Warning in mapply(FUN = function(x, y) {: longer argument not a multiple of length of shorter

I ended up stepping through all of your code, and everything seems to explode when the issues object is created in post_plan.

Do you have any clue why it's doing this? I'm seriously losing it because I am sure it's a simple YAML spacing thing or SOMETHING and I'm going nuts. Enjoy my reprex? 🤷‍♀ Here are some screengrabs of the result on github.

library(projmgr)
# connect to repo (emily i added you just in case)
my_ref <- create_repo_ref("sctyner", "magic-tidying-data")
#> Requests will authenticate with GITHUB_PAT

# run to make sure you can read/write
check_credentials(my_ref)
#> -- With provided credentials -- 
#> + Login: sctyner
#> + Type: User
#> -- In the magic-tidying-data repo -- 
#> + Admin: TRUE
#> + Push: TRUE
#> + Pull: TRUE

# yaml examples 

broken <-"- title: Enter a short, informative title of your goal
  description: >
    Enter a longer description of the objectives
  due_on: 2018-12-31T12:59:59Z
  issue:
    - title: Title of task 1 to complete
      body: Describe this step
      assignees: [sctyner]
      labels: [a, b, c]
    - title: Title of task 2 to comple
      body: Describe this step
      labels: [low]

- title: Enter another short, informative title of your goal
  description: >
    Enter a longer description of the objectives
  issue:
    - title: Title of task 1 to complete
      body: Describe this step
    - title: Title of task 2 to complete
      body: Describe this step
"

okay <- "- title: Data cleaning and validation
  description: >
    We will conduct data quality checks,
    resolve issues with data quality, and
    document this process
  due_on: 2018-12-31T12:59:59Z
  issue:
    - title: Define data quality standards
      body: List out decision rules to check data quality
      assignees: [sctyner]
      labels: [a, b, c]
    - title: Assess data quality
      body: Use assertthat to test decision rules on dataset
      labels: [low]
    - title: Resolve data quality issues
      body: Conduct needed research to resolve any issues

- title: Exploratory data analysis
  description: >
    Create basic statistics and views to better
    understand dataset and relationships
  issue:
    - title: Summary statistics
      body: Calculate summary statistics
    - title: Visualizations
      body: Create univariate and bivariate plots
"

my_plan <- read_plan(broken)
er_plan <- read_plan(okay)

# comparing my_plan and er_plan
is.list(my_plan)
#> [1] TRUE
my_plan
#> Plan: 
#> 1. Enter a short, informative title of your goal (2 issues) 
#> 2. Enter another short, informative title of your goal (2 issues)
er_plan
#> Plan: 
#> 1. Data cleaning and validation (3 issues) 
#> 2. Exploratory data analysis (2 issues)

milestones <- lapply(my_plan,
                     FUN = function(x) x[intersect( names(x), c("title", help_post_milestone()) )])
milestones
#> [[1]]
#> [[1]]$title
#> [1] "Enter a short, informative title of your goal"
#> 
#> [[1]]$description
#> [1] "Enter a longer description of the objectives\n"
#> 
#> [[1]]$due_on
#> [1] "2018-12-31T12:59:59Z"
#> 
#> 
#> [[2]]
#> [[2]]$title
#> [1] "Enter another short, informative title of your goal"
#> 
#> [[2]]$description
#> [1] "Enter a longer description of the objectives\n"

milestoneser <- lapply(er_plan,
                       FUN = function(x) x[intersect( names(x), c("title", help_post_milestone()) )])
milestoneser
#> [[1]]
#> [[1]]$title
#> [1] "Data cleaning and validation"
#> 
#> [[1]]$description
#> [1] "We will conduct data quality checks, resolve issues with data quality, and document this process\n"
#> 
#> [[1]]$due_on
#> [1] "2018-12-31T12:59:59Z"
#> 
#> 
#> [[2]]
#> [[2]]$title
#> [1] "Exploratory data analysis"
#> 
#> [[2]]$description
#> [1] "Create basic statistics and views to better understand dataset and relationships\n"

issues_per_milestone <- vapply(my_plan, FUN = function(x) length(x[["issue"]]), FUN.VALUE = integer(1), USE.NAMES = FALSE )
issues_per_milestone
#> [1] 2 2

issues_per_milestoneer <- vapply(er_plan, FUN = function(x) length(x[["issue"]]), FUN.VALUE = integer(1), USE.NAMES = FALSE )
issues_per_milestoneer
#> [1] 3 2

milestone_title <- vapply(milestones, FUN = function(x) x[["title"]], FUN.VALUE = character(1))
milestone_title
#> [1] "Enter a short, informative title of your goal"      
#> [2] "Enter another short, informative title of your goal"

milestone_titleer <- vapply(milestoneser, FUN = function(x) x[["title"]], FUN.VALUE = character(1))
milestone_titleer
#> [1] "Data cleaning and validation" "Exploratory data analysis"

# here is where it explodes 
issues <- unlist( sapply(my_plan, FUN = function(x) x[["issue"]]), recursive = FALSE )
issueser <- unlist( sapply(er_plan, FUN = function(x) x[["issue"]]), recursive = FALSE )
issues
#> $title
#> [1] "Title of task 1 to complete"
#> 
#> $body
#> [1] "Describe this step"
#> 
#> $assignees
#> [1] "sctyner"
#> 
#> $labels
#> [1] "a" "b" "c"
#> 
#> $title
#> [1] "Title of task 2 to comple"
#> 
#> $body
#> [1] "Describe this step"
#> 
#> $labels
#> [1] "low"
#> 
#> $title
#> [1] "Title of task 1 to complete"
#> 
#> $body
#> [1] "Describe this step"
#> 
#> $title
#> [1] "Title of task 2 to complete"
#> 
#> $body
#> [1] "Describe this step"
issueser
#> [[1]]
#> [[1]]$title
#> [1] "Define data quality standards"
#> 
#> [[1]]$body
#> [1] "List out decision rules to check data quality"
#> 
#> [[1]]$assignees
#> [1] "sctyner"
#> 
#> [[1]]$labels
#> [1] "a" "b" "c"
#> 
#> 
#> [[2]]
#> [[2]]$title
#> [1] "Assess data quality"
#> 
#> [[2]]$body
#> [1] "Use assertthat to test decision rules on dataset"
#> 
#> [[2]]$labels
#> [1] "low"
#> 
#> 
#> [[3]]
#> [[3]]$title
#> [1] "Resolve data quality issues"
#> 
#> [[3]]$body
#> [1] "Conduct needed research to resolve any issues"
#> 
#> 
#> [[4]]
#> [[4]]$title
#> [1] "Summary statistics"
#> 
#> [[4]]$body
#> [1] "Calculate summary statistics"
#> 
#> 
#> [[5]]
#> [[5]]$title
#> [1] "Visualizations"
#> 
#> [[5]]$body
#> [1] "Create univariate and bivariate plots"

post_plan(my_ref, my_plan)
#> Warning in mapply(FUN = function(x, y) {: longer argument not a multiple of
#> length of shorter
#> Error in do.call(post_local, x): second argument must be a list
post_plan(my_ref, er_plan)
#>   milestone_number              milestone_title issue_number
#> 1                3 Data cleaning and validation            1
#> 2                3 Data cleaning and validation            2
#> 3                3 Data cleaning and validation            3
#> 4                4    Exploratory data analysis            4
#> 5                4    Exploratory data analysis            5
#>                     issue_title
#> 1 Define data quality standards
#> 2           Assess data quality
#> 3   Resolve data quality issues
#> 4            Summary statistics
#> 5                Visualizations

^{Created on 2019-05-02 by the reprex package (v0.2.1)}

[emilyriederer]

Hey @sctyner - Wow! Thank you for this amazingly detailed issue. That was so helpful, and I really appreciate you're taking the time!

So, the solution here is actually pretty embarrassing. As you pointed out, it's "exploding" because I used sapply instead of lapply so R is encouraged to go ahead and flatten out the list when possible. I pushed a fix for this bug.

Your question also taught me a few things about my template_yaml() function. First, I did something dumb in that example and gave multiple issues the same title. This won't post using post_plan() unless distinct = FALSE. (Had I used the right *apply function, you probably would have gotten an error about this instead) More unexpected and unfortunate is that template_yaml("plan") looks correctly indented in the console but does not actually indent correctly when copy-pasted. I need to investigate what to do about this. But using either your reprex or copy/pasting the template, I had to manually line things back up as below for read_yaml() to work.

Regardless, the following code now is working for me. Can you please let me know if sapply -> lapply fixed it for you, too?

my_ref <- create_repo_ref("emilyriederer", "experigit")

broken <-
"
- title: Enter a short, informative title of your goal
  description: >
    Enter a longer description of the objectives
  due_on: 2018-12-31T12:59:59Z
  issue:
  - title: Title of task 1 to complete
    body: Describe this step
    assignees: [emilyriederer]
    labels: [a, b, c]
  - title: Title of task 2 to comple
    body: Describe this step
    labels: [low]

- title: Enter another short, informative title of your goal
  description: >
    Enter a longer description of the objectives
  issue:
  - title: Title of task 1 to complete for step 2
    body: Describe this step
  - title: Title of task 2 to complete for step 2
    body: Describe this step
"

my_plan <- read_plan(broken)
post_plan(my_ref, my_plan)

Sorry again for your time and trouble tracking down my error! Clearly this should have been caught by better testing, so its a good reminder to me that I've got to spend more time there. Really appreciate you're taking the time to report and help out so much debugging!

[sctyner]

Reran the reprex and got a new error:

Error in get_engine(api_endpoint = "/milestones", ref = ref, ...) : 
  object 'res' not found

[emilyriederer]

Wow, I am terrible. After I posted that, I introduced a different bug when I made a (thematically) unrelated change. Again, I really appreciate your help and am so sorry for the carelessness. The following reprex below works.

library(projmgr)

# connect to repo (emily i added you just in case)
my_ref <- create_repo_ref("emilyriederer", "experigit")
#> Requests will authenticate with GITHUB_PAT

# run to make sure you can read/write
check_credentials(my_ref)
#> -- With provided credentials -- 
#> + Login: emilyriederer
#> + Type: User
#> -- In the experigit repo -- 
#> + Admin: TRUE
#> + Push: TRUE
#> + Pull: TRUE

# yaml examples 
broken <-"
- title: Enter a short, informative title of your goal
  description: >
    Enter a longer description of the objectives
  due_on: 2018-12-31T12:59:59Z
  issue:
  - title: Title of task 1 to complete
    body: Describe this step
    assignees: [emilyriederer]
    labels: [a, b, c]
  - title: Title of task 2 to complete 
    body: Describe this step
    labels: [low]

- title: Enter another short, informative title of your goal
  description: >
    Enter a longer description of the objectives
  issue:
  - title: Title of task 1 to complete z
    body: Describe this step
  - title: Title of task 2 to complete z
    body: Describe this step
"

my_plan <- read_plan(broken)
post_plan(my_ref, my_plan)
#>   milestone_number                                     milestone_title
#> 1               29       Enter a short, informative title of your goal
#> 2               29       Enter a short, informative title of your goal
#> 3               30 Enter another short, informative title of your goal
#> 4               30 Enter another short, informative title of your goal
#>   issue_number                   issue_title
#> 1          344   Title of task 1 to complete
#> 2          345   Title of task 2 to complete
#> 3          346 Title of task 1 to complete z
#> 4          347 Title of task 2 to complete z

And continues to work with distinct = FALSE for duplicate issue names, as exist in my less-than-excellent template function.

library(projmgr)

# connect to repo (emily i added you just in case)
my_ref <- create_repo_ref("emilyriederer", "experigit")
#> Requests will authenticate with GITHUB_PAT

# run to make sure you can read/write
check_credentials(my_ref)
#> -- With provided credentials -- 
#> + Login: emilyriederer
#> + Type: User
#> -- In the experigit repo -- 
#> + Admin: TRUE
#> + Push: TRUE
#> + Pull: TRUE

# yaml examples 
broken <-"
- title: Enter a short, informative title of your goal a
  description: >
    Enter a longer description of the objectives
  due_on: 2018-12-31T12:59:59Z
  issue:
  - title: Title of task 1 to complete
    body: Describe this step
    assignees: [emilyriederer]
    labels: [a, b, c]
  - title: Title of task 2 to complete 
    body: Describe this step
    labels: [low]

- title: Enter another short, informative title of your goal b
  description: >
    Enter a longer description of the objectives
  issue:
  - title: Title of task 1 to complete z
    body: Describe this step
  - title: Title of task 2 to complete z
    body: Describe this step
"

my_plan <- read_plan(broken)
post_plan(my_ref, my_plan, distinct = FALSE)
#>   milestone_number                                       milestone_title
#> 1               29       Enter a short, informative title of your goal a
#> 2               29       Enter a short, informative title of your goal a
#> 3               30 Enter another short, informative title of your goal b
#> 4               30 Enter another short, informative title of your goal b
#>   issue_number                   issue_title
#> 1          356   Title of task 1 to complete
#> 2          357   Title of task 2 to complete
#> 3          358 Title of task 1 to complete z
#> 4          359 Title of task 2 to complete z

The spacing on the template is still a problem, but I need to research that more since it looks fine but copies incorrectly from the console.