search

mattroumaya / surveymonkey

Access your SurveyMonkey data directly from R!
https://mattroumaya.github.io/surveymonkey/
Other
42 stars 10 forks source link

browse_surveys doesn't return include #79

Closed mikesamuelson32 closed 2 years ago

mikesamuelson32 commented 2 years ago

The line at the end of the script dplyr::select(sl, title, id, url = href, nickname) does not allow additional columns requested through the include parameter to be passed through. For instance, if I request 'date_modified,date_created', they are not being returned in the dataframe. However, if the line is modified to

dplyr::select(sl,title, id, url = href, nickname,everything()), additional columns are returned.

mattroumaya commented 2 years ago

Hi @mikesamuelson32, this is a great catch! I have a solution that works as follows:

Let me know what you think of this update, and I'll open a PR to resolve the issue tomorrow if it sounds good.

mikesamuelson32 commented 2 years ago

Matt,

I am not seeing that either of the workarounds you list below are working. Still getting just the 4 columns.

-- Mike

Please support my ride to end cancer. You can make a donation at www.pelotonia.org/mikesamuelsonhttp://www.pelotonia.org/mikesamuelson

From: Matt Roumaya @.> Sent: Tuesday, November 16, 2021 10:27 PM To: tntp/surveymonkey @.> Cc: mikesamuelson32 @.>; Mention @.> Subject: Re: [tntp/surveymonkey] browse_surveys doesn't return include (Issue #79)

Hi @mikesamuelson32https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmikesamuelson32&data=04%7C01%7C%7C30f11f3b183f4d92583208d9a97a2e16%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727164495654269%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RhIs6ilzaPoW%2Ff%2FguHATjIpqiXghUXWoXjy41Fmtq48%3D&reserved=0, this is a great catch! I have a solution that works as follows:

Let me know what you think of this update, and I'll open a PR to resolve the issue tomorrow if it sounds good.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftntp%2Fsurveymonkey%2Fissues%2F79%23issuecomment-971141263&data=04%7C01%7C%7C30f11f3b183f4d92583208d9a97a2e16%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727164495664230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=KtPqZgOtuwF1vDTM0P0KWL1G91xQmAyldx0dm31YX5s%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVRDOPFDD5DK4JZ7MMIP3LUMMOJ5ANCNFSM5IFIYW5A&data=04%7C01%7C%7C30f11f3b183f4d92583208d9a97a2e16%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727164495664230%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YazXb1t0Ob5%2F1EDT5aFMBTVnSUFgGLJ1U%2B9vE4HQeUY%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C30f11f3b183f4d92583208d9a97a2e16%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727164495674177%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=0zzuLSn4GRMb6vOkL2bEbYIfKBbYxt7P9Abh9qN8EJ4%3D&reserved=0 or Androidhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7C30f11f3b183f4d92583208d9a97a2e16%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637727164495674177%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ONB4WG7uo2w8mN0ngAyUFsLB6gI6WOYlrIVo81HGXxs%3D&reserved=0.

mattroumaya commented 2 years ago

Hi Mike,

I haven't submitted a pull request yet - was waiting to hear back to see if this sounds like a viable solution for you. Let me pull up the project and submit a pull request and I'll let you know when you can reinstall the package and give it a shot.

mikesamuelson32 commented 2 years ago

Matt,

Unfortunately, we have also found another issue as we have continued to dig...

The get_responses function is throwing the stop("Must specify 'oauth_token', Try using smlogin() first.") even though we are passing in a value for oauth_token.

We believe the issue is in lines 52-54, where the function calls itself recursively for each page of survey responses, but it doesn't pass in the oauth_token on those calls.

-- Mike

Please support my ride to end cancer. You can make a donation at www.pelotonia.org/mikesamuelsonhttp://www.pelotonia.org/mikesamuelson

From: Matt Roumaya @.> Sent: Friday, November 19, 2021 2:57 PM To: tntp/surveymonkey @.> Cc: mikesamuelson32 @.>; Mention @.> Subject: Re: [tntp/surveymonkey] browse_surveys doesn't return include (Issue #79)

Hi Mike,

I haven't submitted a pull request yet - was waiting to hear back to see if this sounds like a viable solution for you. Let me pull up the project and submit a pull request and I'll let you know when you can reinstall the package and give it a shot.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftntp%2Fsurveymonkey%2Fissues%2F79%23issuecomment-974385136&data=04%7C01%7C%7Cc9197f9806d44c29b7e308d9ab96d55e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729486578904894%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ibax5NwkxYLMN9Y3Qhav9dLTu2MTDkovfsmH%2BFLJuuk%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVRDOP2NIX2GIVNM3SRM73UM2T27ANCNFSM5IFIYW5A&data=04%7C01%7C%7Cc9197f9806d44c29b7e308d9ab96d55e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729486578904894%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JCBMnrDcMKLoHmOHbdvhcj%2FAqc215SifzRg8qlI8z2A%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7Cc9197f9806d44c29b7e308d9ab96d55e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729486578914857%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oBHEVbVCf5KVe3cOn92dwX3LwXxSKNdBJ5KR1ElMLb4%3D&reserved=0 or Androidhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7Cc9197f9806d44c29b7e308d9ab96d55e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729486578924807%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ScUEfEQvlWKW0DQORIoBqAoZYOf0%2B4WnmBQX%2FY1GRrg%3D&reserved=0.

mattroumaya commented 2 years ago

Hi Mike,

Hopefully the pull request that was just merged will fix the initial issue. You can test it out by reinstalling the package (devtools::install_github('tntp/surveymonkey'))

Can you provide some more info on what you're using get_responses() for? To pull a survey without editing your .Rprofile with the sm_oauth_token global variable, you could try something like:

survey <- 
  123456789 %>% 
  fetch_survey_obj(oauth_token = "your very long oauth token") %>% 
  parse_survey()
mikesamuelson32 commented 2 years ago

Matt,

We are actually using parse_survey, but of course it calls get_responses. In our case, we have the oauth token stored in a database and we pull it into a variable, then we pass that into parse_survey(). The problem is that when you get into that recursive call in get_responses(), it is no longer passing in the token, so the default of getOption("sm_oauth_token"), which of course then fails for us because we are not storing the token in the .Rprofile.

This is in contrast to fetch_survey_obj, where an oauth_token passed in works fine.

-- Mike

Please support my ride to end cancer. You can make a donation at www.pelotonia.org/mikesamuelsonhttp://www.pelotonia.org/mikesamuelson

From: Matt Roumaya @.> Sent: Friday, November 19, 2021 3:35 PM To: tntp/surveymonkey @.> Cc: mikesamuelson32 @.>; Mention @.> Subject: Re: [tntp/surveymonkey] browse_surveys doesn't return include (Issue #79)

Hi Mike,

Hopefully the pull request that was just merged will fix the initial issue.

Can you provide some more info on what you're using get_responses() for? To pull a survey without editing your .Rprofile with the sm_oauth_token global variable, you could try something like:

survey <- 123456789 %>% fetch_survey_obj(oauth_token = "your very long oauth token") %>% parse_survey()

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftntp%2Fsurveymonkey%2Fissues%2F79%23issuecomment-974437819&data=04%7C01%7C%7C4af4ee1a9d264c3b917408d9ab9c1435%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729509100956023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Hp2wZfbxyRHYHWwP%2BXmkFrsgpGtlcyMpcn4i4GOeeL4%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVRDOJE55ONTDJAZAK7OY3UM2YHZANCNFSM5IFIYW5A&data=04%7C01%7C%7C4af4ee1a9d264c3b917408d9ab9c1435%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729509100965979%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jZsZgjRTzDgEU0M3zULXEVuD2cFWw35xRGswuivNuxA%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C4af4ee1a9d264c3b917408d9ab9c1435%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729509100965979%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2F3LLkzVQjwnt%2FJlxzp23TzdrJNs5rrlTN6Jcb3HFrTU%3D&reserved=0 or Androidhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7C4af4ee1a9d264c3b917408d9ab9c1435%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729509100975934%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WvdU%2F0B7GLpcLpKJM8n3WVZ6mxPQujKE26OuoF3XomM%3D&reserved=0.

mattroumaya commented 2 years ago

Are you able to assign the token to a variable within the global environment (instead of in your .Rprofile) called sm_oauth_token?

If so, I think we could modify get_responses() so that it checks the global environment for a specific variable called sm_oauth_token and then assigns that to the token value before error-ing out.

What do you think?

mikesamuelson32 commented 2 years ago

We are currently passing in a variable called bearerToken in as the oauth_token parameter in parse_survey, which then passes it as oauth_token to get_responses.

In get_responses, I think you can easily fix it by changing this code, from lines 52-24

FROM THIS: rnext <- get_responses(id, page = page + 1, all_pages, per_page, start_created_at, end_created_at, start_modified_at, end_modified_at, sort_order, sort_by)

TO THIS: rnext <- get_responses(id, page = page + 1, all_pages, per_page, start_created_at, end_created_at, start_modified_at, end_modified_at, sort_order, sort_by, oauth_token = oauth_token)

-- Mike

Please support my ride to end cancer. You can make a donation at www.pelotonia.org/mikesamuelsonhttp://www.pelotonia.org/mikesamuelson

From: Matt Roumaya @.> Sent: Friday, November 19, 2021 4:17 PM To: tntp/surveymonkey @.> Cc: mikesamuelson32 @.>; Mention @.> Subject: Re: [tntp/surveymonkey] browse_surveys doesn't return include (Issue #79)

Are you able to assign the token to a variable within the global environment (instead of in your .Rprofile) called sm_oauth_token?

If so, I think we could modify get_responses() so that it checks the global environment for a specific variable called sm_oauth_token and then assigns that to the token value before error-ing out.

What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftntp%2Fsurveymonkey%2Fissues%2F79%23issuecomment-974478718&data=04%7C01%7C%7C3f52244e918a4f80a52308d9aba20abd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729534715415424%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=faw9AGtt%2BUBQDA6cxXZnsJb6M6Dye1wanHOXgtm5vRo%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVRDOJXMZ3R3RQNUILC6RTUM25H3ANCNFSM5IFIYW5A&data=04%7C01%7C%7C3f52244e918a4f80a52308d9aba20abd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729534715425379%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BME86eIo8j96qBvrnh7gigOsbPmy9ZYrklANh9vkxDE%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C3f52244e918a4f80a52308d9aba20abd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729534715425379%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xlNmYLzWsrVGmzBa9%2FNVZrLFA02l6YORyfL2mr0u7Sw%3D&reserved=0 or Androidhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7C3f52244e918a4f80a52308d9aba20abd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637729534715435334%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BaWeZtFj3gcYYn7QBfrUqfwxv%2Bm8YPfZyNyh3or1ooI%3D&reserved=0.

mattroumaya commented 2 years ago

@mikesamuelson32 - great idea, thank you! I made that change, check it out when you can.

mattroumaya commented 2 years ago

@mikesamuelson32 did the PR #83 fix your issue?

mattroumaya commented 2 years ago

Closing this for now as it seems to be resolved from #83