Lab 3 `import::here()` and `%>%` errors

[Niagara1000]

I created a separate .R file within the labs/wk03 directory called lab3imports.R. I am trying to keep all the library() functions, variable declarations, functions such as create_dfs(), and more inside lab3imports.R to reduce clutter in the RMD file that I will submit for Lab 3.

1. One issue I ran into when knitting the RMD was I didn't know if I was supposed to include each library(for example, pander) in the import::here() in the RMD as pander or "pander" or if I'm supposed to store each library(..) inside a variable in the lab3imports.R file and call those specific variables of the functions that I need within the import::here() function in the RMD. I tried all 3 combinations and got the following error.

A package was loaded using 'library(...)' from within an import::*() module.
    Please rely on import::here() to load objects from packages within an 
    import::*() module.  See vignette(import) for further details.

2. I also have a question about the following part of the lab 3 tutorial

# load necessary objects
# note: all of these are R objects that will be used throughout this .rmd file

import::here("d", "d1", "d2","compare_dfs",
                       # notice the use of here::here() that points to the .R file where all these R objects are created
                       .from = here::here("labs/wk03/lab3imports.R"),
                       .character_only = TRUE)

I took the compare_dfs() function and put it inside the lab3imports.R file. However, when I am trying to knit the RMD document and also when I run this specific code - vars <- compare_dfs( df1=d1, df2=d2 ), an error shows up saying:

Error in intersect(var.names.1, var.names.2) %>% sort() :
could not find function "%>%"
Calls: <Anonymous> ... withCallingHandlers -> withVisible -> eval -> eval -> compare_dfs
Execution halted

What should I do to override these errors?

Thank you!

@cenuno , sorry forgot to tag you earlier!

[cenuno]

@Niagra1000

Hi Archana,

Great question

First, this is an awesome way to ask a question. Thank you for being so clear and sharing this information for others to see!

Imports are tricky

I think import::here() is totally fine with you loading the necessary libraries for your functions/objects within your .r file (just to be sure, check out my labs/utilities.R file and its use of library() statements.

However, within your RMD file, you may still need to import packages. For instance, if your lab3imports.R file contains functions that create data frames, it is okay to load packages via library() statement with your .rmd files because you're going to do something with the data frame that the reader needs to see.

In fact, this is exactly what I did during the project-data-steps.rmd file.

The reader, however, does not need to see you merging or cleaning data that gets used to create descriptive statistics or a plot. That work should live within your separate .r file.

This partially answers your second question: your function that you import via import::here() requires that in your global environment within the .rmd file you have the %>% function loaded. You can load it via library(magrittr).

Balance is key

Moving as much stuff as possible into the .r file is the goal. However, you will still need to ensure that you load the necessary packages within your .rmd file because sometimes your imports require explicit mention of a package.

[Niagara1000]

Hi @cenuno ,

Thank you for your explanation! I think that problem is fixed. I have a new issue now haha!

For this part of the code, the variables mhmval90 and mhmval00 are not found in d - again..

Adjust home values for inflation

(names(d) %>% sort())[160:170]          # to verify if mhmval00 and mhmval90 are inside `d`, and they are
mhv.90 <- d$mhmval90 * (1.34403)   #  inflation rate in $ from 1990 to 2000
mhv.00 <- d$mhmval00
mhv.change <- mhv.00 - mhv.90
df <- data.frame( MedianHomeValue1990=mhv.90, 
                  MedianHomeValue2000=mhv.00, 
                  Change.90.to.00=mhv.change )
stargazer( df, type=s.type, digits=0, summary.stat = c("min", "p25","median","mean","p75","max") )

The error that I get is:

Error in data.frame(MedianHomeValue1990 = mhv.90, MedianHomeValue2000 = mhv.00,  : 
  arguments imply differing number of rows: 59722, 0
Calls: <Anonymous> ... withCallingHandlers -> withVisible -> eval -> eval -> data.frame
Execution halted

[cenuno]

What is the length of each vector? I would expect mhv.90, mhv.00, and mhv.change to each have 59722. The error is in your df creation logic so that natural place to start debugging are the inputs into that object.

[Niagara1000]

Hi @cenuno ,

I have been trying various ways and somehow I got each of those variables as having length of 59722, as shown in my Global Enviroment (I re-ran the RMD several times).

But now, the error is different:

Error in data.frame(MedianHomeValue1990 = mhv.90, MedianHomeValue2000 = mhv.00,  : 
  object 'mhv.90' not found
Calls: <Anonymous> ... withCallingHandlers -> sys.source -> eval -> eval -> data.frame
Execution halted

This error shows up even though I can see mhv.90 and other objects in the Global Environment as shown in my screenshot below:

[cenuno]

This is very interesting. If the objects exist when you run them individually, there’s something happening either in the chunk above or in your current global environment that’s causing that object to “disappear”.

Try clearing your global environment. Restart R. And re run the RMD file.

Let me know what you find.

— Cristian E. Nuno

---

From: Niagara1000 @.> Sent: Friday, March 26, 2021 10:32:18 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

Hi @cenunohttps://github.com/cenuno ,

I have been trying various ways and somehow I got each of those variables as having length of 59722, as shown in my Global Enviroment (I re-ran the RMD several times).

But now, the error is different:

Error in data.frame(MedianHomeValue1990 = mhv.90, MedianHomeValue2000 = mhv.00, : object 'mhv.90' not found Calls: ... withCallingHandlers -> sys.source -> eval -> eval -> data.frame Execution halted

This error shows up even though I can see mhv.90 and other objects in the Global Environment as shown in my screenshot below:

[Screen Shot 2021-03-26 at 10 23 41 PM]https://user-images.githubusercontent.com/46457863/112711088-f4453100-8e82-11eb-8b5a-db633eaf3398.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808660126, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2S2FXQKCOWJUVHALKCDTFVUWFANCNFSM4Z32LR4A.

[cenuno]

One last thought: is your R file that you’re using with import::here modifying d in any way that is causing mhmval90 to go away? It’s worth confirming this either way.

— Cristian E. Nuno

---

From: Cristian Ernesto Nuno @.> Sent: Friday, March 26, 2021 11:28:13 PM To: DS4PS/cpp-528-spr-2021 @.>; DS4PS/cpp-528-spr-2021 @.> Cc: Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

This is very interesting. If the objects exist when you run them individually, there’s something happening either in the chunk above or in your current global environment that’s causing that object to “disappear”.

Try clearing your global environment. Restart R. And re run the RMD file.

Let me know what you find.

— Cristian E. Nuno

---

From: Niagara1000 @.> Sent: Friday, March 26, 2021 10:32:18 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

Hi @cenunohttps://github.com/cenuno ,

I have been trying various ways and somehow I got each of those variables as having length of 59722, as shown in my Global Enviroment (I re-ran the RMD several times).

But now, the error is different:

Error in data.frame(MedianHomeValue1990 = mhv.90, MedianHomeValue2000 = mhv.00, : object 'mhv.90' not found Calls: ... withCallingHandlers -> sys.source -> eval -> eval -> data.frame Execution halted

This error shows up even though I can see mhv.90 and other objects in the Global Environment as shown in my screenshot below:

[Screen Shot 2021-03-26 at 10 23 41 PM]https://user-images.githubusercontent.com/46457863/112711088-f4453100-8e82-11eb-8b5a-db633eaf3398.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808660126, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2S2FXQKCOWJUVHALKCDTFVUWFANCNFSM4Z32LR4A.

[malmufre]

Hello, I have moved the compare_dfs function to the utilities. R file, however when I try to load it in my rmd, I get the following error.

I am not sure if the problem is with the syntax or not.

[cenuno]

Hi Marah, You’re missing a closing parentheses within your here::here() call.

— Cristian E. Nuno

---

From: Marah @.> Sent: Saturday, March 27, 2021 1:16:39 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

Hello, I have moved the compare_dfs function to the utilities. R file, however when I try to load it in my rmd, I get the following error. [image]https://user-images.githubusercontent.com/59895354/112733663-55880580-8f52-11eb-94c3-6edccc9958ed.png

I am not sure if the problem is with the syntax or not.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808796515, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2S5VV6OZSZFAFAEGV2DTFY4KPANCNFSM4Z32LR4A.

[malmufre]

I have fixed that, and I got another error which is this

[cenuno]

Glad that helped!

For this error, clear your global environment, restart R, and rerun line 64. Also, please delete line 68 as it isn’t doing anything that you need.

— Cristian E. Nuno

---

From: Marah @.> Sent: Saturday, March 27, 2021 1:53:36 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

I have fixed that, and I got another error which is this [image]https://user-images.githubusercontent.com/59895354/112734555-8dde1280-8f57-11eb-9c46-57d94cf81273.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808800993, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2SZOMGR5Z6JPMY4ZZTDTFZAVBANCNFSM4Z32LR4A.

[cenuno]

Also for what it is worth, consider renaming that R file. I used labs/utilities.R because that was how the course directory was set up.

In your project directory, this R file should live within labs/wk03/.

— Cristian E. Nuno

---

From: Cristian Ernesto Nuno @.> Sent: Saturday, March 27, 2021 1:55:51 PM To: DS4PS/cpp-528-spr-2021 @.>; DS4PS/cpp-528-spr-2021 @.> Cc: Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

Glad that helped!

For this error, clear your global environment, restart R, and rerun line 64. Also, please delete line 68 as it isn’t doing anything that you need.

— Cristian E. Nuno

---

From: Marah @.> Sent: Saturday, March 27, 2021 1:53:36 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

I have fixed that, and I got another error which is this [image]https://user-images.githubusercontent.com/59895354/112734555-8dde1280-8f57-11eb-9c46-57d94cf81273.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808800993, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2SZOMGR5Z6JPMY4ZZTDTFZAVBANCNFSM4Z32LR4A.

[malmufre]

Perfect that worked! I cleared my environment and the code chunk ran fine. I think that we shouldn't be running any function in rmd as long as it's in a.R file.

In my project directory, the .R file lives in the labs folder and the utilities.R text file lives in the labs/wk03/ directory. Just to double-check with you , when you say "text file" , do you mean .R file? Because in my PC they are different. Here is a screenshot of what I have done earlier. The.R files are saved in the labs directory and the text files are saved in the labs/wk03/ directory.

[cenuno]

Glad that worked! Your conclusion is correct: looks like import::here() won’t overwrite existing objects in your environment.

Hm this a good question. Technically, both RMD and R files are text files. It isn’t too surprising to see that file explorer is showing the notepad icon with these files since most Window’s machines default text editor is the Notepad app.

I will say that all your relevant weekly lab files (RMD and R) should live in labs/wkXX folders as opposed to the top level labs/ folder. This way you can keep track of things on a week by week basis.

— Cristian E. Nuno

---

From: Marah @.> Sent: Saturday, March 27, 2021 2:08:22 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

Perfect that worked! I cleared my environment and the code chunk ran fine. I think that we shouldn't be running any function in rmd as long as it's in a.R file.

In my project directory, the .R file lives in the labs folder and the utilities.R text file lives in the labs/wk03/ directory. Just to double-check with you , when you say "text file" , do you mean .R file? Because in my PC they are different. Here is a screenshot of what I have done earlier. [image]https://user-images.githubusercontent.com/59895354/112734874-756ef780-8f59-11eb-9531-e199dac83eb3.png The.R files are saved in the labs directory and the text files are saved in the labs/wk03/ directory.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808802594, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2SYTUFZHLHZ67IABL7DTFZCMNANCNFSM4Z32LR4A.

[malmufre]

Great! This means that when you say text file , most of the time you are referring to a .R file or .RMD and not to an actual text file. I will move my .R and .RMD to labs/Wk03 and rename my utilities file.

Thank you!

[Niagara1000]

One last thought: is your R file that you’re using with import::here modifying d in any way that is causing mhmval90 to go away? It’s worth confirming this either way. — Cristian E. Nuno

Hi @cenuno ,

After numerous hours of trying to find what went wrong and almost completely giving up LOL, I realized the order of code in the lab3imports.R file matters so much. I had two chunks of code in the lab3imports.R file in the wrong order.

I did not catch this sooner because whenever I ran the entire lab3imports.R file, I had no errors. All variables and functions got populated in the Global Environment. However, when I knitted the .RMD file for the Lab 3 Tutorial, the objects were getting parsed in the wrong order and the required variables could not be found.

This mixup happened because of how I was transferring objects from my RMD to the .R file. The logical order of the code got a little messy, so I learned I need to be very careful when transferring objects and make sure they can still be understood in the RMD when knitting.

I hope that made sense haha

Thank you for your help!

-Archana

[Niagara1000]

Glad that worked! Your conclusion is correct: looks like import::here() won’t overwrite existing objects in your environment.

Hi @cenuno , I'm a little confused about what @malmufre said and your response because it sounds like the two are opposites

She said, "I think that we shouldn't be running any function in RMD as long as it's in a .R file."

I understood it as import::here() is dependent on what is and is not in the .R file

Your response seems to be the opposite.

Maybe I misunderstood what was said 🤔

[cenuno]

Hi Archana,

So glad you shared the resolution of your code. Order does matter so I hope others learn from you.

Your understanding of the import::here() function is correct. In my response to Marah, I wasn’t responding to her point about functions; only about the text files.

Say my R file has three functions (a, b, and c). When I use import::here(), I’m importing the functions necessary for me to show the non technical audience what it is that helps me make my point within the RMD file.

If that’s all three functions, then I should use a, b, and c throughout my RMD file. What should not happen is the creation of a new function (f) that is created and used within the RMD file.

F should be created within the R file and imported via import::here. I hope that clarifies things!

— Cristian E. Nuno

---

From: Niagara1000 @.> Sent: Saturday, March 27, 2021 10:12:17 PM To: DS4PS/cpp-528-spr-2021 @.> Cc: Cristian Ernesto Nuno @.>; Mention @.> Subject: Re: [DS4PS/cpp-528-spr-2021] Lab 3 import::here() and %>% errors (#18)

Glad that worked! Your conclusion is correct: looks like import::here() won’t overwrite existing objects in your environment.

Hi @cenunohttps://github.com/cenuno , I'm a little confused about what @malmufrehttps://github.com/malmufre said and your response because it sounds like the two are opposites

She said, "I think that we shouldn't be running any function in RMD as long as it's in a .R file."

I understood it as import::here() is dependent on what is and is not in the .R file

Your response seems to be the opposite.

Maybe I misunderstood what was said 🤔

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/DS4PS/cpp-528-spr-2021/issues/18#issuecomment-808845858, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZB2S525TAME34ORHD2I4TTF23DDANCNFSM4Z32LR4A.

[malmufre]

Hi @Niagara1000

Here is an explanation of what I meant : I took the compare_dfs to my .R file and then I tried to import it using import::here, however, because I ran that same function in my.RMD before I was getting the error mentioned above, therefore I cleared my environment. By doing so, the code chunk ran with no errors. Let me know if this clarifies what I said

[Niagara1000]

Hi @Niagara1000

Here is an explanation of what I meant : I took the compare_dfs to my .R file and then I tried to import it using import::here, however, because I ran that same function in my.RMD before I was getting the error mentioned above, therefore I cleared my environment. By doing so, the code chunk ran with no errors. Let me know if this clarifies what I said

Ah gotcha, so basically if I have a function definition in my .R file, then I should not have it in my .RMD file.

I guess the functions and variables get overwritten when their definitions are in more than 1 file

Thanks!