search

nmarus / node-ews

A simple JSON wrapper for the Exchange Web Services (EWS) SOAP API.
MIT License
116 stars 52 forks source link

Streaming Request? #48

Open kcastrotech opened 7 years ago

kcastrotech commented 7 years ago

I'm playing around with the EWS streaming subscription and am wondering if there is a way to access the streaming request as data comes in instead of waiting until it finishes?

For example, if I create a subscription then use the "GetStreamingEvents" function like so:

        ews.run("GetStreamingEvents",{
            SubscriptionIds:{
                SubscriptionId:this.subscriptionId
            },
            ConnectionTimeout:1
        }).then(events => {
            //Do something with events
        }, err => {
            //Ooops, something went wrong!
        });

The "then(events => {..." isn't called until after the connection times out (e.g. 1 minute) and the response at that point contains multiple envelopes with or without events:

<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:Notifications>
                        <m:Notification>
                            <t:SubscriptionId>FQBjYmNkY21haWwwMy5jYmNrYy5vcmcQAAAAvxnM8b2nh0C6cdPzWYkpp2wPU82pgtQIEAAAACGRLXGSJlZJn4ZVwqpDTtI=</t:SubscriptionId>
                            <t:CreatedEvent>
                                <t:TimeStamp>2017-04-13T20:15:37Z</t:TimeStamp>
                                <t:ItemId Id="AAH8Bh99AAA=" ChangeKey="CQAAAA==" />
                                <t:ParentFolderId Id="AAAqoK/AAA=" ChangeKey="AQAAAA==" />
                            </t:CreatedEvent>
                            <t:NewMailEvent>
                                <t:TimeStamp>2017-04-13T20:15:37Z</t:TimeStamp>
                                <t:ItemId Id="yHL73BAAH8Bh99AAA=" ChangeKey="CQAAAA==" />
                                <t:ParentFolderId Id="AAAAqoK/AAA=" ChangeKey="AQAAAA==" />
                            </t:NewMailEvent>
                            <t:ModifiedEvent>
                                <t:TimeStamp>2017-04-13T20:15:37Z</t:TimeStamp>
                                <t:FolderId Id="3iAAqoK/AAA=" ChangeKey="AQAAAA==" />
                                <t:ParentFolderId Id="3iAAAAqoK8AAA=" ChangeKey="AQAAAA==" />
                                <t:UnreadCount>2</t:UnreadCount>
                            </t:ModifiedEvent>
                        </m:Notification>
                    </m:Notifications>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:ConnectionStatus>OK</m:ConnectionStatus>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:ConnectionStatus>OK</m:ConnectionStatus>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:Notifications>
                        <m:Notification>
                            <t:SubscriptionId>FQBjYmNkY21haWwwMy5jYmNrYy5vcmcQAAAAvxnM8b2nh0C6cdPzWYkpp2wPU82pgtQIEAAAACGRLXGSJlZJn4ZVwqpDTtI=</t:SubscriptionId>
                            <t:ModifiedEvent>
                                <t:TimeStamp>2017-04-13T20:16:59Z</t:TimeStamp>
                                <t:ItemId Id="AA99AAA=" ChangeKey="CQAAAA==" />
                                <t:ParentFolderId Id="AAMkADc/AAA=" ChangeKey="AQAAAA==" />
                            </t:ModifiedEvent>
                            <t:ModifiedEvent>
                                <t:TimeStamp>2017-04-13T20:16:59Z</t:TimeStamp>
                                <t:FolderId Id="AAMkAAAA=" ChangeKey="AQAAAA==" />
                                <t:ParentFolderId Id="AAMoK8AAA=" ChangeKey="AQAAAA==" />
                                <t:UnreadCount>1</t:UnreadCount>
                            </t:ModifiedEvent>
                        </m:Notification>
                    </m:Notifications>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:ConnectionStatus>OK</m:ConnectionStatus>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:ConnectionStatus>OK</m:ConnectionStatus>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
    <soap11:Header xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <ServerVersionInfo xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="0" MajorBuildNumber="1076" MinorBuildNumber="9" Version="V2_22" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" />
    </soap11:Header>
    <soap11:Body xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/">
        <m:GetStreamingEventsResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages">
            <m:ResponseMessages>
                <m:GetStreamingEventsResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:ConnectionStatus>Closed</m:ConnectionStatus>
                </m:GetStreamingEventsResponseMessage>
            </m:ResponseMessages>
        </m:GetStreamingEventsResponse>
    </soap11:Body>
</Envelope>

So what I need to do is tap into the request as it's running and parse the streaming data as it comes in.

I've been digging through the node-ews and node-soap code but am not seeing anything that can allow me to do that without forking it. I was hoping someone here may have another idea or knows of something that I've missed?

Thanks in advance!

nmarus commented 7 years ago

We are already on a modified fork of node-soap to fix a recursion issue with stack overflow when parsing the MS wdsl. If it makes sense to, we can modify that fork? Otherwise, been thinking of replacing node-soap completely, but yet to find the time to research another library.

kcastrotech commented 7 years ago

I ended up putting this on the back burner and focusing on push notifications instead which I was able to get working with minimal modifications. However, I think I've got an idea on how to do this.

It would appear that the node-soap HttpClient (soap\lib\http.js:21) accepts options.request as an option that later gets used to make the eventual request to the soap server. I then tracked the wsdlOptions object and it would appear that it gets passed along to wsdl.open_wsdl whereby it is then used to create a new HttpClient (soap\lib\wsdl.js:2149). I think I can tap into that by passing a custom function as wsdlOptions.request that wraps the request module. Then I should be able to intercept the request and pipe (https://github.com/request/request#streaming) it to a function that can process the incoming stream.

If that works then no modifications should be needed. I'll keep everyone posted on how it goes.

kcastrotech commented 7 years ago

Also, have you seen strong-soap? (https://github.com/strongloop/strong-soap) I haven't played with it yet but it might be a suitable alternative and would seem to fit in node-ews perfectly with little to no modification. I'm just not sure about the stack overflow issue. But if I get a chance I'll replace node-soap with that and let you know how it goes. Can you link me to the original issue and fix for the stack overflow problem so I can be sure to test properly?

nmarus commented 7 years ago

Issue #17 and fork of node this lib uses is here: https://github.com/CumberlandGroup/node-soap with modified code here: https://github.com/CumberlandGroup/node-soap/commit/aafad446344ef858d424126061287376893edc8e

The issue is node-soap does a LOT of recursive function without a process.nexttick, while the MS wsdl has a lot of self referencing logic. (=boom) See here for some details on this with javascript and recursion: http://stackoverflow.com/questions/28349548/how-does-process-nexttick-keep-my-stack-from-blowing-up

kcastrotech commented 7 years ago

Oh yeah... I remember that issue now! :)

nmarus commented 7 years ago

Moved over to the updated version of node-soap (19.2) in version 3.2.x. Im curious if this is working now?

PatrickUrlberger commented 3 years ago

Has this issue been fixed aready? I still encounter the Problem