Closed brendanrbrown closed 3 years ago
What version of AzureGraph are you using? See if updating to the latest version helps.
Thanks for the reply. I did have the latest CRAN version of AzureGraph, 1.3.1. I just updated to the dev versions of both AzureGraph and Microsoft365R but get an identical error with the example above.
I can't reproduce this. Please post your sessioninfo. Also, does the same problem occur with get_personal_outlook()
?
This seems to be an issue with the build on my linux machines only. I could not reproduce it on a mac. In both cases below, I run the same code as in the issue post in a clean R session (after library(Microsoft365R)
).
sessionInfo()
for build that creates issue
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 20.04 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] Microsoft365R_2.2.1
loaded via a namespace (and not attached):
[1] httr_1.4.2 compiler_4.1.0 R6_2.5.0 tools_4.1.0 curl_4.3.2 rappdirs_0.3.3 jose_1.1.0
[8] jsonlite_1.7.2 AzureAuth_1.3.2 AzureGraph_1.3.1 openssl_1.4.4 askpass_1.1
I reproduce the issue as well on a separate machine running Ubuntu 20.04 LTS with the same package and R versions --- not surprising since Pop is built on Ubuntu.
Edit: Now able to reproduce on this machine
sessionInfo()
for build failing to reproduce issue
R version 4.1.0 (2021-05-18)
Platform: x86_64-apple-darwin17.0 (64-bit)
Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
|-- snip, remainder identical --|
I do not have a personal outlook account to test, and attempting to access my business account with get_personal_outlook
fails at authentication --- no account found --- which makes sense.
Update: I am able to reproduce the error also on the same platform as the first sessionInfo above, this time with R versions 3.6.3 and 4.0.5.
Unfortunately I'm still unable to repro the bug, both in Windows and Linux (WSL2).
How are you authenticating? Note that accessing your Outlook inbox only makes sense when there is a user account present ie you're using the authorization_code or device_code flows.
Try uninstalling and reinstalling both AzureGraph and Microsoft365R -- the error you're seeing is from a method defined in the former, and inherited in the latter.
Actually I suspect I know what's going on. Run the following and post the result:
outl <- get_business_outlook(*)
inb <- outl$get_inbox()
inb$do_operation("messages")$value[[1]]
This will retrieve your first email, so make sure there isn't anything confidential in there.
This does work as it should. Thoughts?
> outl <- get_business_outlook()
Loading Microsoft Graph login for default tenant
> inb <- outl$get_inbox()
> inb$do_operation("messages")$value[[1]]
$`@odata.etag`
[1] "<omitted>"
$id
[1] "<omitted>"
$createdDateTime
[1] "2021-06-28T02:16:05Z"
$lastModifiedDateTime
[1] "2021-06-28T02:16:05Z"
$changeKey
[1] "<omitted>"
$categories
list()
$receivedDateTime
[1] "2021-06-28T02:16:05Z"
$sentDateTime
[1] "2021-06-28T02:16:02Z"
$hasAttachments
[1] FALSE
$internetMessageId
[1] "<omitted>@<omitted>.namprd19.prod.outlook.com>"
$subject
[1] "dummy"
$bodyPreview
[1] "dummy"
$importance
[1] "normal"
$parentFolderId
[1] "<omitted>"
$conversationId
[1] "<omitted>"
$conversationIndex
[1] "<omitted>"
$isDeliveryReceiptRequested
[1] FALSE
$isReadReceiptRequested
[1] FALSE
$isRead
[1] FALSE
$isDraft
[1] FALSE
$webLink
[1] "https://outlook.office365.com/owa/?ItemID=<omitted>&exvsurl=1&viewmodel=ReadMessageItem"
$inferenceClassification
[1] "focused"
$unsubscribeData
list()
$unsubscribeEnabled
[1] FALSE
$body
$body$contentType
[1] "text"
$body$content
[1] "dummy"
$sender
$sender$emailAddress
$sender$emailAddress$name
[1] "Brendan Brown"
$sender$emailAddress$address
[1] "<omitted>"
$from
$from$emailAddress
$from$emailAddress$name
[1] "Brendan Brown"
$from$emailAddress$address
[1] "<omitted>"
$toRecipients
$toRecipients[[1]]
$toRecipients[[1]]$emailAddress
$toRecipients[[1]]$emailAddress$name
[1] "Brendan Brown"
$toRecipients[[1]]$emailAddress$address
[1] "<omitted>"
$ccRecipients
list()
$bccRecipients
list()
$replyTo
list()
$mentionsPreview
NULL
$flag
$flag$flagStatus
[1] "notFlagged"
For some of your previous comments, note that I am attempting to authenticate the exact same way for the same account both in the cases when I can and cannot produce the issue. I am able to produce the issue on a fresh install of Microsoft365R and AzureGraph, even an entirely fresh R install with only those two packages. I can post details of how I authenticate. Let me know. But perhaps you already know what the issue is here.
Ok, those properties look exactly like they should, which is puzzling.
Can you tell me what this returns:
# message properties from before
msg <- inb$do_operation("messages")$value[[1]]
# this should be an R6 generator for the `ms_outlook_email` class
AzureGraph::find_class_generator(msg)
# this should be TRUE only for the 'message' item
sapply(
ls(AzureGraph:::.graph_classes),
function(i) AzureGraph:::.graph_classes[[i]]$check(msg)
)
This does look like a namespace / method dispatch issue, but I have no idea why it would be platform-specific.
> ol <- get_business_outlook()
Loading Microsoft Graph login for default tenant
Access token has expired or is no longer valid; refreshing
> inb <- ol$get_inbox()
> msg <- inb$do_operation("messages")$value[[1]]
> AzureGraph::find_class_generator(msg)
<ms_outlook_email> object generator
Inherits from: <ms_outlook_object>
Public:
user_id: NULL
initialize: function (token, tenant = NULL, properties = NULL, user_id = NULL)
set_body: function (body = NULL, content_type = NULL)
set_subject: function (subject)
set_recipients: function (to = NULL, cc = NULL, bcc = NULL)
add_recipients: function (to = NULL, cc = NULL, bcc = NULL)
set_reply_to: function (reply_to = NULL)
add_attachment: function (object, type = c("view", "edit", "embed"), expiry = "7 days",
add_image: function (object)
get_attachment: function (attachment_name = NULL, attachment_id = NULL)
list_attachments: function (filter = NULL, n = Inf)
remove_attachment: function (attachment_name = NULL, attachment_id = NULL, confirm = TRUE)
download_attachment: function (attachment_name = NULL, attachment_id = NULL, ...)
send: function ()
create_reply: function (comment = "", send_now = FALSE)
create_reply_all: function (comment = "", send_now = FALSE)
create_forward: function (comment = "", to = NULL, cc = NULL, bcc = NULL, send_now = FALSE)
get_message_headers: function ()
copy: function (dest)
move: function (dest)
print: function (...)
clone: function (deep = FALSE)
Private:
make_attachment: function (object, inline, type, expiry, password, scope)
Parent env: <environment: namespace:Microsoft365R>
Locked objects: TRUE
Locked class: FALSE
Portable: TRUE
> sapply(ls(AzureGraph:::.graph_classes), function(i) AzureGraph:::.graph_classes[[i]]$check(msg))
aadUserConversationMember application attachment
FALSE FALSE FALSE
channel chatMessage device
FALSE FALSE FALSE
directoryRole drive driveItem
FALSE FALSE FALSE
fileAttachment group itemAttachment
FALSE FALSE FALSE
list listItem mailFolder
FALSE FALSE FALSE
message plan plan_bucket
TRUE FALSE FALSE
plan_task referenceAttachment servicePrincipal
FALSE FALSE FALSE
site team user
FALSE FALSE FALSE
Let's go through this step by step. This is the basic code to get the list of emails, see if it works when entered from the commandline:
lst <- inb$do_operation("messages")
pag <- inb$get_list_pager(lst, user_id=inb$user_id)
pag$value
All previous output is from the R console in the terminal initialized with R --no-save
.
Here is the issue again, from a fresh session:
library(Microsoft365R)
ol <- get_business_outlook()
inb <- ol$get_inbox()
lst <- inb$do_operation("messages")
pag <- inb$get_list_pager(lst, user_id=inb$user_id)
pag$value
Error in initialize(...) :
unused argument (user_id = "<omitted>")
Does this imply ... is not being captured correctly here? That is how I read it, from the ms_graph_pager
initialization, which should capture user_id
here.
Edit Actually, perhaps not or at least not so clearly. See below.
AzureGraph::ms_graph_pager$new(inb$token, lst, user_id = inb$user_id)
<Graph pager object>
output: object
has data: TRUE
edited to remove annoying >
The error occurs here:
pager <- AzureGraph::ms_graph_pager$new(inb$token, lst, user_id = inb$user_id)
AzureGraph:::extract_list_values(pager, 1)
Access token has expired or is no longer valid; refreshing
Error in initialize(...) :
unused argument (user_id = "<omitted>")
Edit
Actually this probably is an argument capture issue and lazy evaluation. Running AzureGraph::extract_list_values
line by line on object pager
above produces an error the first time I run it but succeeds the second time. I still have no idea why this would differ across platforms but perhaps something to do with the DOTSXP internals, of which I know little.
And further, once I have run the code below, if I then run AzureGraph:::extract_list_values(pager, 1)
I get the correct result.
pager <- AzureGraph::ms_graph_pager$new(inb$token, lst, user_id = inb$user_id)
n <- 1
res <- NULL
pager$output
[1] "object"
This produces the error the first time I run it. If I immediately run it again, it gives a correct <Outlook email>
object.
bind_fn <- base::c
while(pager$has_data() && NROW(res) < n) # not nrow()
res <- bind_fn(res, pager$value)
res
Furthermore, once I have the correct output res
in the previous line, AzureGraph:::extract_list_values(pager, 1)
also now works correctly.
The ms_graph_pager
object is an iterator; accessing the $value
component retrieves successive pages from the output of the messages
REST endpoint. In turn, extract_list_values
is just a convenience function that gets $value
repeatedly until all the data has been retrieved.
Note that running extract_list_values
on a pager object multiple times gives undefined results, meaning I don't know what the REST endpoint will pass you.
It looks like there may be a specific email message that is throwing off Microsoft365R. You can track it down by running the following:
lst <- inb$do_operation("messages")
pager <- inb$get_list_pager(lst, generate_objects=FALSE)
vals <- AzureGraph::extract_list_values(pager)
sapply(vals, function(v) AzureGraph::find_class_generator(v)$classname)
If everything is working correctly, the above should return ms_outlook_email
for all the messages.
Actually this probably is an argument capture issue and lazy evaluation. Running AzureGraph::extract_list_values line by line on object pager above produces an error the first time I run it but succeeds the second time.
This suggests that the offending email is in the 1st page of results. In that case, to simplify things and avoid retrieving your entire inbox, you can run
lst <- inb$do_operation("messages")
pager <- inb$get_list_pager(lst, generate_objects=FALSE)
vals <- pager$value
sapply(vals, function(v) AzureGraph::find_class_generator(v)$classname)
First -- Thanks for the back-and-forth. This does seem to be going somewhere.
You are correct that several items from the output of the final code chunk above are ms_object
class and not ms_outlook_email
. This perhaps is part of the problem. Some questions as follow-up, though:
Why should one platform produce an issue when another does not, if indeed the problem is with some particular offending email? Edit: I am able to reproduce it on the mac platform given above.
Suppose the offending email, one with ms_object
class, is at index 7 in the returned object from inb$do_operation("messages")
. Then why should inb$do_operation("messages")$value[[7]]
return the correct message output?
Edited
ms_outlook
changed to ms_object.
typo. apologies.
There shouldn't be anything with ms_outlook
. Did you mean ms_outlook_folder
?
Can you post the names
for one of the messages that returns ms_object
?
How it works is that the REST endpoint returns a list of fields containing the data for an object. The AzureGraph framework then looks at the structure of the fields to determine what type of object it is: an email, a file in Onedrive, an AAD user, etc. In this case, the test it uses for an email is the presence of a bodyPreview
field, and for whatever reason this is missing on some of the emails.
See below. bodyPreview
does not seem to be missing. In fact all messages have that field.
lst <- inb$do_operation("messages")
bad <- purrr::keep(lst$value,
~ !("bodyPreview" %in% names(.x))
)
print(length(bad))
[1] 0
Request from the previous comment:
msg <- inb$do_operation("messages")$value[[7]]
print(names(msg))
[1] "@odata.type" "@odata.etag" "id"
[4] "createdDateTime" "lastModifiedDateTime" "changeKey"
[7] "categories" "receivedDateTime" "sentDateTime"
[10] "hasAttachments" "internetMessageId" "subject"
[13] "bodyPreview" "importance" "parentFolderId"
[16] "conversationId" "conversationIndex" "isDeliveryReceiptRequested"
[19] "isReadReceiptRequested" "isRead" "isDraft"
[22] "webLink" "inferenceClassification" "unsubscribeData"
[25] "unsubscribeEnabled" "meetingMessageType" "type"
[28] "isOutOfDate" "isAllDay" "isDelegated"
[31] "responseRequested" "allowNewTimeProposals" "body"
[34] "sender" "from" "toRecipients"
[37] "ccRecipients" "bccRecipients" "replyTo"
[40] "mentionsPreview" "flag" "startDateTime"
[43] "endDateTime" "recurrence" "previousLocation"
[46] "previousStartDateTime" "previousEndDateTime"
That is odd. Can you also post the contents of that message? Remove anything confidential first, of course.
I particularly want to know what the @odata.type
field contains (AzureGraph uses this to determine the type of object, if it exists). Is this field also present for the other emails?
Another update: I am able to produce this error on the mac architecture as well now, which at least is more sane. I will edit the previous message.
Contents of inb$do_operation("messages")$value[[7]]
Loading Microsoft Graph login for default tenant
$`@odata.type`
[1] "#microsoft.graph.eventMessageRequest"
$`@odata.etag`
[1] "W/\"CwAAABYAAAD+X8am1PlvQapDtTGiQTcaAAALjSH8\""
$id
[1] "<omitted>"
$createdDateTime
[1] "2021-06-23T22:33:11Z"
$lastModifiedDateTime
[1] "2021-06-23T22:33:13Z"
$changeKey
[1] "<omitted>"
$categories
list()
$receivedDateTime
[1] "2021-06-23T22:33:11Z"
$sentDateTime
[1] "2021-06-23T22:33:04Z"
$hasAttachments
[1] FALSE
$internetMessageId
[1] "<omitted>@<omitted>.namprd19.prod.outlook.com>"
$subject
[1] "Stats Team Meeting"
$bodyPreview
[1] "________________________________________________________________________________\r\nMicrosoft Teams mee
ting\r\nJoin on your computer or mobile app\r\nClick here to join the meeting\r\nLearn More | Meeting option
s\r\n___________________________________________"
$importance
[1] "normal"
$parentFolderId
[1] "<omitted>"
$conversationId
[1] "<omitted>"
$conversationIndex
[1] "<omitted>"
$isDeliveryReceiptRequested
NULL
$isReadReceiptRequested
[1] FALSE
$isRead
[1] TRUE
$isDraft
[1] FALSE
$webLink
[1] "https://outlook.office365.com/owa/?ItemID=<omitted>=1&viewmodel=ReadMessageItem"
$inferenceClassification
[1] "focused"
$unsubscribeData
list()
$unsubscribeEnabled
[1] FALSE
$meetingMessageType
[1] "meetingRequest"
$type
[1] "singleInstance"
$isOutOfDate
[1] FALSE
$isAllDay
[1] FALSE
$isDelegated
[1] FALSE
$responseRequested
[1] TRUE
$allowNewTimeProposals
NULL
$body
$body$contentType
[1] "html"
$body$content
[1] "<html><head>\r\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\"><meta content=\"
text/html; charset=us-ascii\"></head><body><div><br><br><br><div style=\"width:100%; height:20px\"><span sty
le=\"white-space:nowrap; color:#5F5F5F; opacity:.36\">______________________________________________________
__________________________</span> </div><div class=\"me-email-text\" style=\"color:#252424; font-family:'Seg
oe UI','Helvetica Neue',Helvetica,Arial,sans-serif\"><div style=\"margin-top:24px; margin-bottom:20px\"><spa
n style=\"font-size:24px; color:#252424\">Microsoft Teams meeting</span> </div><div style=\"margin-bottom:20
px\"><div style=\"margin-top:0px; margin-bottom:0px; font-weight:bold\"><span style=\"font-size:14px; color:
#252424\">Join on your computer or mobile app</span> </div><a class=\"me-email-headline\" href=\"https://nam
12.safelinks.protection.outlook.com/ap/t-59584e83/?url=<omitted>;sdata=<omitted>
src=\"https://teams.microsoft.com/l/meetup-join/<omitted>
" target=\"_blank\" rel=\"noreferrer noopener\" title=\"https://teams.microsoft.com/l/meetup-join/<omitted>" style=\"font-size:14p
x; font-family:'Segoe UI Semibold','Segoe UI','Helvetica Neue',Helvetica,Arial,sans-serif; text-decoration:u
nderline; color:#6264a7\">Click here to join the meeting</a> </div><div style=\"margin-bottom:24px; margin-t
op:20px\"><a class=\"me-email-link\" target=\"_blank\" href=\"https://nam12.safelinks.protection.outlook.com
/?url=<omitted>%2FJoinTeamsMeeting&data=<omitted>\" originalsrc=\"https://aka.ms/JoinTeamsMeeting\" shash=\"<omitted>
=\"font-size:14px; text-decoration:underline; color:#6264a7; font-family:'Segoe UI','Helvetica Neue',Helvet
ica,Arial,sans-serif\">Learn More</a> | <a class=\"me-email-link\" target=\"_blank\" href=\"https://nam1<omitted>">
<span style=\"white-space:nowrap; color:#5F5F5F; opacity:.36\">_____________________________________________
___________________________________</span> </div></div></body></html>"
$sender
$sender$emailAddress
$sender$emailAddress$name
[1] "<omitted>"
$sender$emailAddress$address
[1] "<omitted>"
$from
$from$emailAddress
$from$emailAddress$name
[1] "<omitted>"
$from$emailAddress$address
[1] "<omitted>"
$toRecipients
$toRecipients[[1]]
$toRecipients[[1]]$emailAddress
$toRecipients[[1]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[1]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[2]]
$toRecipients[[2]]$emailAddress
$toRecipients[[2]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[2]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[3]]
$toRecipients[[3]]$emailAddress
$toRecipients[[3]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[3]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[4]]
$toRecipients[[4]]$emailAddress
$toRecipients[[4]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[4]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[5]]
$toRecipients[[5]]$emailAddress
$toRecipients[[5]]$emailAddress$name
[1] "Brendan Brown"
$toRecipients[[5]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[6]]
$toRecipients[[6]]$emailAddress
$toRecipients[[6]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[6]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[7]]
$toRecipients[[7]]$emailAddress
$toRecipients[[7]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[7]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[8]]
$toRecipients[[8]]$emailAddress
$toRecipients[[8]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[8]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[9]]
$toRecipients[[9]]$emailAddress
$toRecipients[[9]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[9]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[10]]
$toRecipients[[10]]$emailAddress
$toRecipients[[10]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[10]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[11]]
$toRecipients[[11]]$emailAddress
$toRecipients[[11]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[11]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[12]]
$toRecipients[[12]]$emailAddress
$toRecipients[[12]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[12]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[13]]
$toRecipients[[13]]$emailAddress
$toRecipients[[13]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[13]]$emailAddress$address
[1] "<omitted>"
$toRecipients[[14]]
$toRecipients[[14]]$emailAddress
$toRecipients[[14]]$emailAddress$name
[1] "<omitted>"
$toRecipients[[14]]$emailAddress$address
[1] "<omitted>"
$ccRecipients
$ccRecipients[[1]]
$ccRecipients[[1]]$emailAddress
$ccRecipients[[1]]$emailAddress$name
[1] "<omitted>"
$ccRecipients[[1]]$emailAddress$address
[1] "<omitted>"
$bccRecipients
list()
$replyTo
list()
$mentionsPreview
NULL
$flag
$flag$flagStatus
[1] "notFlagged"
$startDateTime
$startDateTime$dateTime
[1] "2021-06-23T18:00:00.0000000"
$startDateTime$timeZone
[1] "UTC"
$endDateTime
$endDateTime$dateTime
[1] "2021-06-23T19:00:00.0000000"
$endDateTime$timeZone
[1] "UTC"
$recurrence
NULL
$previousLocation
NULL
$previousStartDateTime
$previousStartDateTime$dateTime
[1] "2021-06-23T18:00:00.0000000"
$previousStartDateTime$timeZone
[1] "UTC"
$previousEndDateTime
$previousEndDateTime$dateTime
[1] "2021-06-23T19:00:00.0000000"
$previousEndDateTime$timeZone
[1] "UTC"
Right, microsoft.graph.eventMessageRequest
is a calendar invite, which is technically not an email (although it has the same structure as one). Currently Microsoft365R doesn't have support for calendars and events. However, it should do something more sensible than throwing an error when it finds an invite.
I've pushed a workaround in the outlook-calendar-fix
branch. Can you give it a go and see if it fixes the error?
Ah, that at least does make some sense. I agree there should be more information than an error, but more practically: It would be good to simply skip these in the iterator yielding messages, since I don't know apriori which elements are calendar invitations vs. genuine messages. Perhaps just a try-catch and pass on error is enough, or similar, in extract_list_values
Yes, I can confirm it works. Setting default_generator = ms_outlook_email
in get_list_pager
seems to have done the trick. Nice!
Hello,
fails with the message
Error in initialize(...) : unused argument (user_id = "<id-omitted>")
The
list_emails
method ultimately callsms_object$get_list_pager()
in AzureGraph, which in turn callsms_graph_pager$new()
.At first glance it seems to me the
user_id
argument should be captured here then passed toclass_gen$new
, but that is where I lose the trail as I'm unsure which R6 classfind_class_generator
is resolving to at this point.Thanks, and happy to pass along more info.