Open plamoni opened 11 years ago
plamoni
commented
11 years ago I really did not want this to turn into a focal point for violations of the developer NDA. Please do not discuss unreleased Apple software on this forum. For now on I'm just going to delete any such posts without warning or modification.
elvisimprsntr
commented
11 years ago Do not upgrade your devices to iOS 7 GM Seed if SiriProxy is important to you. Please keep in mind that iOS 7 is not yet a public release and we are still bound by an iOS developer NDA. We are investigating any issues for possible solutions. We will provide updates when we have relevant information we can disclose.
alexrmc92
commented
11 years ago Now that iOS 7 has officially been released lets get this working. It's been a few months since i've looked at the source. I don't think iOS7 resolve to guzzoni.apple.com anymore. With my local DNS server handling that domain, siri will still function on iOS7. Perhaps they changed the servers DNS name, or possibly they internalized the ip address for it. They could have also set iOS7 to use a specific name server for Siri.
I'll do some packet tracing at work today and see what i can find out.
elvisimprsntr
commented
11 years ago I know you are trying to be helpful, but please refrain from posting conjecture or speculation. It does not serve anyone any good and dilutes resources from real solutions. Pete is already looking into it.
P.S. I'll be sure to let your boss know you are using company resources and time to diagnose the problem. ;)
Sikozu
commented
11 years ago Guys I came across this while looking for a solution to get siri-proxy working whilst I was in the beta;
https://github.com/zhangyuanwei/node-siri/issues/13
The server is now daryl.apple.com, which I got also from my own tests.
elvisimprsntr
commented
11 years ago I know everyone is anxious and trying to be helpful, but once again please refrain from conjecture and speculation. What someone may have discovered while iOS was in beta is not likely to apply to the public release. From now on any information or posts that are known to be inaccurate will automatically be deleted.
fablanglet
commented
11 years ago The struture of the siri's object have been changed. "Properties" doesn't exist now. The property "class" become "$class", for "group" and "v" it's the same case. See the example below. Moreover, the domain daryl.apple.com has been used for the beta version, now it's ever guzzoni.apple.com but the version change, and the user-agent change IOS 6.1: User-Agent: Assistant(iPhone/iPhone4,1; iPhone OS/6.1.3/10B329) Ace/2.1 IOS 7: User-Agent: Assistant(iPhone/iPhone4,1; iPhone OS/7.0/11A465) Ace/3.0
It's my observation, i modified the siriproxy code, but the protocol have more changes, because even i modified the structure of the object, siriproxy doesn't work. If someone has more information, let me know and we can work together (@Fab_Lan or html5webapp.blogspot.com).
Example for SpeechRecognized (IOS7): {"$group"=>"com.apple.ace.speech", "$class"=>"SpeechRecognized", "$v"=>"3.0", "aceId"=>"GUID", "refId"=>"GUID", "recognition"=> {"$group"=>"com.apple.ace.speech", "$class"=>"Recognition", "$v"=>"3.0", "phrases"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Phrase", "$v"=>"3.0", "interpretations"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Interpretation", "$v"=>"3.0", "tokens"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Token", "$v"=>"3.0", "confidenceScore"=>804.0, "endTime"=>780, "originalText"=>"Teste", "removeSpaceAfter"=>false, "removeSpaceBefore"=>true, "startTime"=>0, "text"=>"Teste"}]}], "lowConfidence"=>false, "speechRepairApplied"=>false}, {"$group"=>"com.apple.ace.speech", "$class"=>"Phrase", "$v"=>"3.0", "interpretations"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Interpretation", "$v"=>"3.0", "tokens"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Token", "$v"=>"3.0", "confidenceScore"=>808.0, "endTime"=>1080, "originalText"=>"Siri", "removeSpaceAfter"=>false, "removeSpaceBefore"=>false, "startTime"=>780, "text"=>"Siri"}]}], "lowConfidence"=>false, "speechRepairApplied"=>false}, {"$group"=>"com.apple.ace.speech", "$class"=>"Phrase", "$v"=>"3.0", "interpretations"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Interpretation", "$v"=>"3.0", "tokens"=> [{"$group"=>"com.apple.ace.speech", "$class"=>"Token", "$v"=>"3.0", "confidenceScore"=>883.0, "endTime"=>2780, "originalText"=>"proxy", "removeSpaceAfter"=>false, "removeSpaceBefore"=>false, "startTime"=>1080, "text"=>"proxy"}]}], "lowConfidence"=>false, "speechRepairApplied"=>false}], "sentenceConfidence"=>514}, "sessionId"=>"GUID"}
bgottsch
commented
11 years ago I've noticed 2 different headers with the new iOS 7 (I run the non-beta version). Maybe this can help someone. Also, in another test, I changed some of the SiriProxy code and had what I believe were problems with the SSL connection with the iDevice.
(the log below is from the non-modified SiriProxy server and in this test I was able to get a response from Siri)
[Info - Server] DNS Server started, tainting 'guzzoni.apple.com' with 10.0.1.25
[Info - Server] Starting SiriProxy on 0.0.0.0:443...
[Info - Server] SiriProxy up and running.
[Info - Plugin Manager] Plugins loaded: Example, Computer, Personal
[Info - iPhone] SSL completed for iPhone
[Header - iPhone] ACE /ace HTTP/1.0
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Content-Length: 2000000000
[Header - iPhone] User-Agent: Assistant(iPhone/iPhone5,1; iPhone OS/7.0/11A465) Ace/3.0
[Header - iPhone] Accept-Language: en-US
[Header - iPhone] X-Ace-Host: st14p02sa:405549ce-fcc8-4779-b412-1ddf9eaacfed
[Header - iPhone]
[Debug - iPhone] Found end of headers
[Info - Plugin Manager] Plugins loaded: Example, Computer, Personal
[Info - iPhone] SSL completed for iPhone
[Header - iPhone] ACE /ace HTTP/1.0
[Header - iPhone] Host: guzzoni.apple.com
[Header - iPhone] Content-Length: 2000000000
[Header - iPhone] User-Agent: Assistant(iPhone/iPhone5,1; iPhone OS/7.0/11A465) Ace/3.0
[Header - iPhone] Accept-Language: en-US
[Header - iPhone] X-Ace-Host: st14p02sa:405549ce-fcc8-4779-b412-1ddf9eaacfed
[Header - iPhone]
[Debug - iPhone] Found end of headers
[Info - iPhone] Received Object: (group: , ref_id: , ace_id: F24C73B3-0681-476B-AAC5-1126D671BE26)
{"$class"=>"LoadAssistant",
"connectionType"=>"Wifi",
"speechId"=>"539335ad-021b-40a4-9496-b74e8c1806a5",
"assistantId"=>"a1746c70-7c6e-4dea-8401-e43f4c8eaa9f",
"activationToken"=>
"\xB1\xA2\xF6\xE6\xE2\x10\xC5 >j\xBD\x126\xA1\x97 \xB5j$",
"aceId"=>"F24C73B3-0681-476B-AAC5-1126D671BE26",
"$group"=>"com.apple.ace.system",
"sessionValidationData"=>
"\x02\xB5\xEB\xDB\x7F\x84\xA1{\xAE\xD3\x02s\x8B5\xC1\x05\xA4\xA2iT\xAF\x9D\x83V\xA0f\xB8L\x83c$\xF5h\x00\x00\x02\x00\x06\x00\x00\x00O\x00\x00\x00\x80\xEF\xAEP\xB1\xF06\xD1\xDD\xF0\x9D\x8E\xD3\xDA\xDB\x86\x05T\xEB\xFA\x1D\x8B\xA6\xB9z\b\xEB\x10.\x04'F\x11\xDDZ\x02\t\x96j\xEDJ\x90\xA2R\xDC\xB7\x03\x1ADo\xC5\xD2\xCD\x1EK\xD6\xE2&\xC6\xAB\xCE\xBE\x00]-\xA71=Bb\xA5\xE9\x98\e\xE7\xD2s!\xD0\xE7\xD2\xD6Q\xA2\x92Yw\xDF\x13W1\x9EH{\xD7P\xA6\x00\xFEM\xC3dG\r\x14\xEC1\x9D\xE3I\xB8\xEE/\x11[\xFB\f\f @M\xC4]\xCC'\x0F\x95\xA6\x00\x00\x01\x14<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n<plist version=\"1.0\">\n
bgottsch
commented
11 years ago Also another issue I discovered was that Siri on my iPhone, when connect to the SiriProxy over wifi, uses cellular data to get the requests. When I have cellular + wifi enabled Siri works and presents 2 different headers. When I disable it (wifi only), it shows only one header and does not work. I tested it on my wifi iPad and I get the same result as the iPhone with cellular off, and in this log it appears to be doing circles, differently from the one I posted above. (the above one was using wifi + 3g)
elvisimprsntr
commented
11 years ago Siri was known to "leak" over the cellular connection in iOS 6 when the cellular connection was determined to be better, which was often the case for me when connected to LTE. Thus one had to disable the cellular data connection.
I came across this quote by Olivier Bonaventure, one of, if not the leading researcher in the field of Multi Path TCP. Olivier Bonaventure writes: "Besides changes in UI, multitasking and other features that the press discusses, iOS7 also includes support for Multipath TCP. Multipath TCP is a major extension to TCP that is able to use different interfaces for the same connection. Until now, Multipath TCP has been mainly used by researchers with a modified Linux kernel. iOS7 changes that, with millions of Multipath-TCP enabled devices that can switch from 3G to WiFi without losing existing TCP connections. This is not yet the case on iOS7, which currently seems to only enable it for SIRI, but other use cases will likely appear in the future." Source: http://m.slashdot.org/story/191809
I am not an expert, but MPTCP seems to offer multiple connections over multiple IP addresses (e.g. Cellular, Wifi) to provide more robust services when one connection quality is poor or is dropped. While these reports suggest Apple is limiting MPTCP between Wifi and cellular and only for Siri, I can see the benefits of providing more robust connection to services when switching cell towers, when for a brief moment your phone has multiple cellular data connections. Also could be a used for persistent MPTCP LTE connections to improve throughput and robustness.
Regardless, from what little I have read on MPTCP, it seems turing off the cellular connection to limit Siri to using Wifi only would thwart Siri trying to use both.
bgottsch
commented
11 years ago @elvisimprsntr Maybe this could represent a problem. I'm looking into MPTCP. Therefore, it is possible that the iPhone prefers wifi over cellular, so if the SiriProxy connection is working it won't be a problem. Another way I though to fix this is to use VPN. While I using iOS 6, I used VPN over cellular and it worked perfectly with SiriProxy. Although, with or without the MPTCP, SiriProxy is yet not able to work with iOS 7. I think the MPTCP problems will be better figured out when it is working. For now we will have to disable cellular while we try to find a fix to SiriProxy. When I have more info of how Siri on iOS 7 works I'll let you know. In the mean time, I'll have a look into MPTCP.
elvisimprsntr
commented
11 years ago @bgottsch I used a VPN to invoke SiriProxy in iOS 6 with success as well. At the moment I am having trouble getting Siri to route thru the VPN connection. It seems Siri ignores the "route all traffic thru VPN" setting, or at least the symptoms appear that way.
bgottsch
commented
11 years ago @elvisimprsntr I just tested it using VPN (I run iVPN) over 3G and the SiriProxy identifies a connection from my iPhone. I found 2 headers in the log:
[Header - iPhone] ACE /ace HTTP/1.0 [Header - iPhone] Host: guzzoni.apple.com [Header - iPhone] Content-Length: 2000000000 [Header - iPhone] User-Agent: Assistant(iPhone/iPhone5,1; iPhone OS/7.0/11A465) Ace/3.0 [Header - iPhone] Accept-Language: en-US
and
[Header - iPhone] HEAD /ace HTTP/1.1 [Header - iPhone] Host: guzzoni.apple.com [Header - iPhone] Accept-Encoding: gzip, deflate [Header - iPhone] Accept: / [Header - iPhone] Content-Length: 0 [Header - iPhone] Connection: keep-alive [Header - iPhone] Accept-Language: en-us [Header - iPhone] User-Agent: AssistantServices/1 CFNetwork/672.0.2 Darwin/14.0.0
When I ask Siri something over this VPN connection it does not respond, so I think it is working. On my VPN setup, I point my computer running SiriProxy as the DNS server and run RubyDNS from the SiriProxy on the same address. Maybe this is preventing your iPhone to connect to the server.
Also I have found this link http://www.networkworld.com/news/2013/091913-ios7-multipath-273995.html which says "MPTCP is spreading the connection’s data across several subflows, sending it over the least congested paths."
elvisimprsntr
commented
11 years ago @bgottsch I use DD-WRT for both my VPN server and to perform a transparent guzzoni.apple.com redirect, which worked well under iOS 6.
The source of the information all points to Professor Olivier Bonaventure, http://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/18/mptcp.html I wonder if he actually consulted Apple and once the "Cone of Silence" was lifted when iOS 7 became public, allowed him to publish Apple's use of MPTCP. If Apple is using his implementation of MPTCP, it would certainly help his case over the competing implementations.
Regardless, it seems like getting SiriProxy working with iOS 7 will be more work than anticipated. I see it as a challenge and opportunity to learn.
bgottsch
commented
11 years ago @elvisimprsntr It seems that iOS 7 avoids using VPN to connect Siri. I am having problems with the VPN connection and Siri. It appears to be avoiding, as you said before. The DNS + wifi-only solution is working. For now I'll use that to see whats happening to SiriProxy. I agree, it will be a challenge.
bgottsch
commented
11 years ago @elvisimprsntr @plamoni I could be wrong, but I think the problem is on EventMachine and on the SSL connections.
plamoni
commented
11 years ago I'd be kind of surprised if the issue was with the SSL. The SSL is the lowest level thing (from an OSI layer prospective) that gets logged. It tosses bunches of logging when you're having connection issues, even if those are actually TCP or DNS related. If the SSL were failing, we'd be having trouble getting any data at all.
I think it's more likely that the problem is related to the multi-path TCP used by Siri in iOS7. It sounds like the VPN use-case might not be viable anymore. Again, I've had just enough free time to follow these threads, I haven't had time to do any real investigation yet.
My planned approach is to do basically the opposite of what I did when building the proxy. Essentially, when I started out, I would build the protocol encoders/decoders in one at a time, starting with a simple passthrough that handles the SSL MITM, then building on various elements of the protocol stack. If you look into the connection class, you'll see this behavior and should be able to short-circuit it. By cutting out the higher level decoders (which might be broken), it should be easier to determine at what level things are breaking down.
My worry is that the problem occurs within the TCP layer itself, which is actually handled by the kernel of the host OS. This is a sort-of worst-case-scenario. The only real solution would probably be to require people to implement MP-TCP on their servers, which would eliminate support for OSX and Windows hosts. Only Linux hosts would work and you'd probably have to build a customized kernel to add support for MP-TCP.
Again, I've done no research at all beyond reading these threads. So don't panic. I just need time. Hope to have some this weekend.
Also, I have an iPhone 5S on order. Should get here in a couple weeks.
C4Wiz
commented
11 years ago elvisimprsntr
commented
11 years ago Here is a good write up on how MPTCP actually works. http://lwn.net/Articles/544399/
elvisimprsntr
commented
11 years ago Installation options for various OS distributions: http://multipath-tcp.org/pmwiki.php?n=Users.HowToInstallMPTCP
P.S. Sorry native Windows users, you may be out of luck unless you are willing to run Linux on a VM
bgottsch
commented
11 years ago I wonder if OSX 10.9 Mavericks has MPTCP. I'm also installing the VM so I can test it.
elvisimprsntr
commented
11 years ago There seems to be a MPTCP virtualization method. https://github.com/multipath-tcp/mptcp-virtual
bgottsch
commented
11 years ago @elvisimprsntr I've seen your link. Awesome. I'm installing it (since I run on a Mac). I'll see if I can find something new with it.
plamoni
commented
11 years ago This might be a solution for testing to see if MP-TCP is at the core of the problem (I'm still far from sure it is), but it's not a sustainable solution. Requiring people to install a (second in many cases) virtual machine just to handle the base connection is a pain. Most of the improvements I've made to SP over the last couple years are focused on trimming dependancies (on things like third-party DNS servers). I would hate to have a new dependency of "boot up a whole new (virtual) computer..."
On Fri, Sep 20, 2013 at 3:41 PM, Beno notifications@github.com wrote:
@elvisimprsntr https://github.com/elvisimprsntr I've seen your link. Awesome. I'm installing it (since I run on a Mac). I'll see if I can find something new with it.
— Reply to this email directly or view it on GitHubhttps://github.com/plamoni/SiriProxy/issues/542#issuecomment-24845908 .
bgottsch
commented
11 years ago @plamoni @elvisimprsntr I agree. Using a VM is too complicated. I think of two solutions to this. One is to disable cellular, which will make Siri use wifi and hence connect to SiriProxy. The second is to fix the server for iOS 7 and see if Siri automatically uses wifi (since it's a more reliable connection it should be automatic). Though, for this options or any other to work, SiriProxy must first be fixed. So in the mean time, while doing the testings, using a VM is not a bad idea. Another work around would be to disable cellular. I'm currently checking the connection.rb file to see if I can find where the problem is. Let you know if I have found something.
bgottsch
commented
11 years ago @plamoni @elvisimprsntr I think I found the problem. It may be stupid, but I think guzzoni is refusing/disconnecting from the SiriProxy. If analysed the connection.rb and it does almost everything correctly. The first and most important issue is with the new objects, which are a bit different and make the SiriProxy not interpret them. The other (which I think is the key issue) is that the iPhone, even with cellular disabled, creates new headers when it completes the SSL to guzzoni. This makes the SiriProxy/iPhone reconnect resulting in a loop hole where it never connects to guzzoni. I'm not an expert on servers so I might be wrong, but in all my testing I was not able to send/receive objects from guzzoni with iOS 7. In the same server I was able to do so with my iPad running iOS 6 (to verify if it was not another problem). A question I have is the cause of this disconnection, if it is the iPhone or guzzoni who made it.
Ps.: sorry to bother on a Saturday.
C4Wiz
commented
11 years ago any updates on the development of this?
plamoni
commented
11 years ago Not yet. :-)
plamoni
commented
11 years ago Some good news:
The important part is here:
In the comments on Bonaventure's blog post, Kristian Evensen writes that it looks like Siri doesn't actually use MPTCP to send packets over Wi-Fi and 3G/LTE at the same time. Rather, it sets up MPTCP and then continues to communicate over Wi-Fi. If the Wi-Fi network then goes down or stops working, the communication can switch seamlessly from Wi-Fi to 3G or LTE without the need to reconnect, renegotiate a secure connection, and retransmit the audio making up the question to Siri.
This is very good news. Basically, if it's correct, it means that MPTCP is not killing SiriProxy and that it's just a change to the protocol that's breaking stuff. SiriProxy should continue to work as it does today for our purposes as long as it determines it can talk to the Apple servers over the WiFi.
Also, consider that since the proxy happens via DNS, it means that the IP address the Siri client connects to is that of the proxy server. So it may try to open multiple paths to that server, but those attempted over 3G/4G/LTE should be unsuccessful (unless your SP server is public -- which is still a bad idea). Therefore, it should limit itself to the SP server and make everyone pretty happy.
So just need to figure out why the protocol interpretation is broken (not necessarily easy) and hopefully we'll be back in business.
Hopefully. :-)
elvisimprsntr
commented
11 years ago The question I have is, will iOS 7 still expect a response from SP that it supports MPTCP, thus still require a MPTCP kernel.
plamoni
commented
11 years ago That could be an issue, but I'm not sure. I think that given there's only a single path to the server it will work just like regular TCP. It won't find a second path so it won't attempt to open a second connection.
Things might be different in situations where you have two valid paths to your server (e.g. if your server is on a public IP that can be reached by 3G).
elvisimprsntr
commented
11 years ago Or for example, a person has both Wifi and cellular enabled, which iOS attempts to open a second connection to Apple over cellular to "join" the existing connection which does not exist, thus drops both connections, and starts the cycle over again. I suspect one of the limitations might be one must disable cellular data to prevent the second connection, but still may require a MPTCP kernel.
bgottsch
commented
11 years ago If the MPTCP kernel is the issue it will be a problem since SP uses EventMachine to run the connections and I don't know if it supports MPTCP. Until now I wasn't able to SSL to guzzoni and send packets to it. I think the problem is with the iPhone creating multiple headers which make the SP disconnect and reconnect in an endless cycle. Even with cellular disabled it creates multiple headers. I think this is the main problem.
plamoni
commented
11 years ago Or for example, a person has both Wifi and cellular enabled, which iOS attempts to open a second connection to Apple over cellular to "join" the existing connection which does not exist, thus drops both connections, and starts the cycle over again.
I'm not sure that is the case. Remember that TCP operates just above IP in the network stack. It operates with IP addresses instead of DNS names. Therefore, given that the phone's DNS client will resolve guzzoni.apple.com as, say, 10.0.0.1, it will try to open the second subflow to 10.0.0.1 via the cellular connection. However, given that 10.0.0.1 is not globally routable, that subflow will fail. I assume (but I really don't know) that this means that things just fall back to something close enough to TCP as to not matter to SiriProxy.
JaredSartin
commented
11 years ago This could be problematic if the initial resolve was done via cellular. But again, this is all speculative.
plamoni
commented
11 years ago You're right. However, you can just turn on airplane mode for a second or two to reset all your connections. Having the DNS resolution happen over cellular (or another untainted connection) has always been an issue with SiriProxy.
bgottsch
commented
11 years ago I was able to send some objects to guzzoni and receive some also. Therefore they had no effect on the iPhone and were all the same. I had to disable cellular data and made a minor change to the code in connection.rb file. My assumption is that the objects are not being recognised since they are different (they were version 2.1 now they are 3.0). Here they go:
[Info - Guzzoni] Received Object: (group: , ref_id: 7EE2EB4A-CDD2-421D-A41D-F90C56E47C37, ace_id: 37f33937-834e-4979-b852-bb176c6aba0c)
{"$group"=>"com.apple.ace.system",
"$class"=>"CommandFailed",
"$v"=>"3.0",
"aceId"=>"37f33937-834e-4979-b852-bb176c6aba0c",
"refId"=>"7EE2EB4A-CDD2-421D-A41D-F90C56E47C37",
"errorCode"=>722,
"reason"=>
"9EE4F32A-6768-47FA-B9AC-7DE1AD77B3D5 was not the last request, cannot roll back"}[Info - Guzzoni] Received Object: (group: , ref_id: ABAEB531-C332-41D2-9824-33CF6976D0A6, ace_id: 1ce3eae7-83ea-448c-8c44-277340e65f70)
{"$group"=>"com.apple.ace.assistant",
"$class"=>"AddViews",
"$v"=>"3.0",
"aceId"=>"1ce3eae7-83ea-448c-8c44-277340e65f70",
"refId"=>"ABAEB531-C332-41D2-9824-33CF6976D0A6",
"dialogPhase"=>"Error",
"displayTarget"=>"PrimaryDisplay",
"scrollToTop"=>false,
"temporary"=>false,
"views"=>
[{"$group"=>"com.apple.ace.assistant",
"$class"=>"AssistantUtteranceView",
"$v"=>"3.0",
"deferredRendering"=>false,
"speakableText"=>"Sorry, I missed that.",
"dialogIdentifier"=>"Error#speechOtherFailure",
"text"=>"Sorry, I missed that."}]}[Info - Guzzoni] Got invalid data (non-ACE protocol?), terminating the connection.[Header - Guzzoni] HTTP/1.0 406 Not Acceptable[Header - Guzzoni] HTTP/1.0 200 OK[Info - Guzzoni] Received Object: (group: , ref_id: 93D8362E-999D-4D97-897E-66D1453D2DF4, ace_id: 2978dfbd-51d6-4b64-b832-2e5cf46cc929)
{"$group"=>"com.apple.ace.system",
"$class"=>"SetConnectionHeader",
"$v"=>"3.0",
"aceId"=>"2978dfbd-51d6-4b64-b832-2e5cf46cc929",
"refId"=>"93D8362E-999D-4D97-897E-66D1453D2DF4",
"aceHostHeader"=>"nk33p01sa:35d4eea6-cb68-40eb-981b-61ecbdcd7c95",
"reconnectNow"=>false}[Info - Guzzoni] Received Object: (group: , ref_id: 4CAA272C-20B4-4EF2-955B-CB9DDFD90A64, ace_id: 605ad257-db53-437f-9e0c-9320e1609494)
{"$group"=>"com.apple.ace.system",
"$class"=>"AssistantLoaded",
"$v"=>"3.0",
"aceId"=>"605ad257-db53-437f-9e0c-9320e1609494",
"refId"=>"4CAA272C-20B4-4EF2-955B-CB9DDFD90A64",
"dataAnchor"=>"114644da49843606dcfc1ad93ee14cf1152a66fb",
"requestSync"=>false,
"syncAnchors"=>
[{"$group"=>"com.apple.ace.sync",
"$class"=>"Anchor",
"$v"=>"3.0",
"count"=>124,
"generation"=>"710",
"key"=>"com.apple.siri.applications",
"validity"=>"181FD265-497B-4EF1-B7AE-85AE57088D18"},
{"$group"=>"com.apple.ace.sync",
"$class"=>"Anchor",
"$v"=>"3.0",
"count"=>147,
"generation"=>"759 320",
"key"=>"com.apple.contact.people",
"validity"=>"2 05C123C3-C57C-4267-BA55-9CF04D0356ED"},
{"$group"=>"com.apple.ace.sync",
"$class"=>"Anchor",
"$v"=>"3.0",
"count"=>1,
"generation"=>"0",
"key"=>"com.apple.reminder.list.name",
"validity"=>"2131785503"},
{"$group"=>"com.apple.ace.sync",
"$class"=>"Anchor",
"$v"=>"3.0",
"count"=>30,
"generation"=>"32",
"key"=>"com.apple.media.stations",
"validity"=>"290701048"},
{"$group"=>"com.apple.ace.sync",
"$class"=>"Anchor",
"$v"=>"3.0",
"count"=>28,
"generation"=>"4242138264",
"key"=>"com.apple.alarm.label",
"validity"=>"4242138264"},
{"$group"=>"com.apple.ace.sync",
"$class"=>"Anchor",
"$v"=>"3.0",
"count"=>1690,
"generation"=>"7462",
"key"=>"com.apple.media.entities",
"validity"=>"758685915002176979"}],
"version"=>
"20130910-72848-branches/innsbruck-i0?cnxn=95701801-af40-4ab4-ad5a-67e966ea4555"}
ghost
commented
11 years ago I hope we get some solution!!! i updated my Phone to IOS7 and i can't use siriProxy, i will to come back to IOS6
plamoni
commented
11 years ago @bgottsch - Is Siri functional? Plugin support will take some refactoring to deal with the new object formats, but if Siri is working via SiriProxy, I'd be really happy to get a pull request with your fixes.
bgottsch
commented
11 years ago @plamoni I was able to send objects to guzzoni and receive a response to them on the SiriProxy only. I wasn't able to send them to my iPhone. I'm still trying to do so. But I already had some progress by communicating with guzzoni. Still its not a success. I'll try some more things this weekend. For now what I think is the problem is that SiriProxy is not interpreting the objects correctly and is sending incorrect ones to both sides. I assume this because I received an object which had the text "Sorry, I missed that.", which is usually what guzzoni responds when the packets (objects in this case) are not recognised. Here is the object:
[Info - Guzzoni] Received Object: (group: , ref_id: ABAEB531-C332-41D2-9824-33CF6976D0A6, ace_id: 1ce3eae7-83ea-448c-8c44-277340e65f70)
{"$group"=>"com.apple.ace.assistant",
"$class"=>"AddViews",
"$v"=>"3.0",
"aceId"=>"1ce3eae7-83ea-448c-8c44-277340e65f70",
"refId"=>"ABAEB531-C332-41D2-9824-33CF6976D0A6",
"dialogPhase"=>"Error",
"displayTarget"=>"PrimaryDisplay",
"scrollToTop"=>false,
"temporary"=>false,
"views"=>
[{"$group"=>"com.apple.ace.assistant",
"$class"=>"AssistantUtteranceView",
"$v"=>"3.0",
"deferredRendering"=>false,
"speakableText"=>"Sorry, I missed that.",
"dialogIdentifier"=>"Error#speechOtherFailure",
"text"=>"Sorry, I missed that."}]}I think this might be something worth exploring:
[Info - Guzzoni] Got invalid data (non-ACE protocol?), terminating the connection.That message comes from here:
https://github.com/plamoni/SiriProxy/blob/master/lib/siriproxy/connection.rb#L59-L65
Basically, it's a catch-all for data it doesn't understand. So I would suggest you have it dump a stack trace from the exception to see what might be causing the exception to be thrown. My guess is that it might have to do with that second connection. It may not be zipped. Or, more likely, it doesn't have the 4-byte ACE header:
https://github.com/plamoni/SiriProxy/blob/master/lib/siriproxy/connection.rb#L52-L56
We'll probably have to start paying attention to the headers and process the two connections differently.
It might also be useful to attach some info about which connection is which to the logging. Sadly, that won't be simple. I never got around to unifying the logging. But maybe this is our chance. :-)
bgottsch
commented
11 years ago @plamoni NIce idea. Will do that.
dtomcat
commented
11 years ago You guys get this working... and you'll definitely earn some beer... my treat!
fideladio
commented
11 years ago @plamoni Hi!! Ive been trying to setup Siriproxy on Ubuntu 13.04, I Think the phone is connecting to the terminal because everytime I press the home button to bring up Siri the terminal reacts to it but Siri doesnt understan, Ive been stuck with this for a week, can you send me an email with some help, fideladio@hotmail.com , thanks in advanced!!
elvisimprsntr
commented
11 years ago @fideladio SiriProxy does bit work with iOS 7 yet.
fideladio
commented
11 years ago @elvisimprsntr Hey Elvis, im not running iOS7. The problem im having is that, the terminal seems to communicate with my phone BUT when I test Siri with the command of "Test Siri Proxy", Siri doesnt understand.
elvisimprsntr
commented
11 years ago Then open a new issue so I can try to help you. This issue is specific to iOS 7.
Post the follow information in a new issue.
E
On Oct 4, 2013, at 17:36, Fidel Eladio Mendez notifications@github.com wrote:
@elvisimprsntr Hey Elvis, im not running iOS7. The problem im having is that, the terminal seems to communicate with my phone BUT when I test Siri with the command of "Test Siri Proxy", Siri doesnt understand.
— Reply to this email directly or view it on GitHub.
ferrygun
commented
11 years ago Wondering if you guys have already any outcome to make SiriProxy works on iOS 7?
vibroboy
commented
11 years ago any news on ios7 support ?
Franchie
commented
11 years ago Hello,
I've been playing around with iOS7 Siri for the past few evenings, using custom perl code for a simple proxy. I find that I am able to get correct data to and back from the Guzzoni server, with correct SpeechRecognized packets being sent back (with valid speech recognition). However I find that the iPhone disconnects from my proxy quite abruptly and seemingly randomly, and is therefore not receiving the response correctly.
Using a direct proxy (iPhone->proxy->Guzzoni->proxy->iPhone), I find that the iPhone closes the connection to the proxy, and re-initiates a number of new connections (including the occasional "salt > Salty!" and what I assume to be keep-alive ping requests) to it thereafter before finally abandoning. I am therefore unable to complete the link, even before doing any packet modifications.
I do not as yet know why this disconnect occurs, and why a simple bypass proxy does not work. While it may be an error in my proxy code, I was wondering whether you had encountered similar experiences and had any insight into the underlying reasons? Since the iPhone initiates what is essentially a streaming channel, why does it close it, even without the proxy sending anything back?
Otherwise, what problems are you encountering using the 'official' proxy? Are you also seeing disconnects, or is it merely problems with the changes in the object format? Basically, how can we help? :)
Regards, F.
UPDATE:
Thanks for everyone's continued interest in SiriProxy. I know lots of people are really excited to get it "up and running" with iOS7. Now that the software is public, you're welcome to discuss bugs/issues on on this forum. However, keep two things in mind:
1: Only public releases of iOS are supported by SiriProxy I don't have a developer account with Apple right now, so I'm not under NDA (I never had access to pre-release iOS7), but my understanding is that the pre-release versions of the software point to a different set of servers from the production versions. Therefore, you'll have a number of issues as
guzzoni.apple.comis hard-coded in in several places (and is an integral part of the SSL MITM cert generation process). This being the case, I highly recommend upgrading to the public release (NOT the GM release from earlier this week) before you file any bugs with SiriProxy.2: SiriProxy has no full time development staff At this point, I'm the sole developer on SiriProxy. ElvisImpersonator does an AMAZING job at supporting the SiriProxy developer community, but doesn't work on the core development. Both of us have full time jobs that keep us extremely busy and neither of us get paid to work on SiriProxy. It's a labor of love (one that you're free to participate in through pull requests!).
That being said, I am on call this week and Elvis has, um, left the building (he's traveling for work), so it might be a week or more before I even get a chance to do testing on iOS7. So, as with before: UPDATE TO iOS7 AT YOUR OWN RISK. There's no official support for iOS7 in SiriProxy at this time and I can't make any guarantees it will ever work on iOS7. As with any new release of iOS, Apple could (any may already have) put in measures to block SiriProxy from working.
Again, thank you for your continued interest in my project. SiriProxy started out as a fun short-term project that kept me busy while I was between jobs and has turned into much more than I ever would have expected. I will continue to try and keep things working as long as I can and I'm always every excited to see the awesome projects that you, the SiriProxy development community, create! Keep up the great work and thanks again!!!
-Pete
Original:
Okay, so I'm watching the WWDC right now and there's lots of change to Siri coming. This means two things:
1: NEW BETA -- NO DISCUSSION
There's a beta out today for developers. Please remember that as an Apple Developer, you must agree to an NDA regarding unreleased applications (e.g. iOS7). This means that you should refrain from discussion of changes to the Siri protocol or functionality until iOS7 is publicly released. Once it's out for the public we can start working on fixing whatever it might break. Please do not open issues regarding problems when using iOS7 until it is publicly released. Elvis and I will close these tickets.
2: SiriProxy might break with beta software
As mentioned, a new iOS might mean changes to the Siri protocol. The iOS6 changes were minimal and I was able to fix them pretty quickly. No promises for iOS7 or future software. Hopefully it's something I'll be able to work with, but don't go upgrading devices if SiriProxy is important to you.
Thanks everyone for your interest!
-Pete