Closed shreeve closed 13 years ago
I'm trying to maintain a high test coverage ratio and I don't have access to BlazeDS, so I wouldn't be able to create tests for this. If you could write some test or at the very least add some binary data dump to spec/fixtures/object
I can certainly investigate merging this in.
Is there any reason you didn't reuse the existing AbstractMessage
class in lib/rocketamf/values/messages.rb
?
As I read your message this morning, I see that the github code is much newer than the gem that I was using when I made my changes. All of this may thus be irrelevant. :-)
Nevertheless, I'll try to describe what I did below. I'm certainly no expert on BlazeDS or AMF, but from what I gather there is both a "normal" way that messages are sent and a "shorthand" way. The normal way appears to indicate the message type by its class name (as a string) and then all of the fields follow (as strings) and then all of the actual values for the fields (as objects). In the shorthand way, instead of spelling out the whole class name, such as "flex.messaging.messages.AbstractMessage", a shorthand version called "DSK" is sent. Since this is a shorthand message, it tries to save space by not actually spelling out all of the field names as strings. Instead, following the "DSK", there are one or more bytes whose bits represent the presence of specific fields. So, for example if bits 1 and 4 of the first byte are on, then that means that two specific fields will follow. I assume the whole motivation is to shorten the length of the resulting message. From what I saw in RocketAMF, there is currently no way to deserialize these "shorthand" message types, which is what I added.
Here are the shorthand message mappings that I can see:
DSK - AcknowledgeDSMessage DSA - AsyncDSMessage DSC - CommandDSMessage DSD - DataDSMessage DSP - PagedDSMessage DSQ - SequencedDSMessage DSU - UpdateCollectionDSMessage
Here is a brief hierarchy of messages, along with the static fields defined in each class:
Abstract body clientId destination headers messageId timestamp timeToLive
Command < Abstract operation
Async < Abstract correlationId
Data < Async identity operation
UpdateCollection < Data collectionId replace updateMode
Acknowledge < Async
Sequenced < Acknowledge dataMessage dynamicSizing sequenceId sequenceProxies sequenceSize
Paged < Sequenced pageCount pageIndex
In my case, I needed to deserialize the "DSK" message, which needs the following minimal setup. The other classes above could follow the same pattern.
class AbstractMessage
ExternalizedFields = [
%w[ body clientId destination headers messageId timestamp
timeToLive ], # byte 1: bits 1 through 7 %w[ clientIdBytes messageIdBytes ] # byte 2: bits 1 and 2 ]
class AsyncMessage
ExternalizedFields = [
%w[ correlationId correlationIdBytes ], # byte 1: bits 1 and 2
]
class AcknowledgeMessage
ExternalizedFields = nil
In order to deserialize (which I called "internalize") a shorthand message, I modified the code in "../pure/deserializer.rb" to:
if traits[:externalizable]
if obj.respond_to?(:internalize)
obj.internalize(source)
else
obj.externalized_data = deserialize(source)
end
else
Then each externalizable class has a method such as:
def internalize(source)
super
populate(source, ExternalizedFields)
end
So, to deserialize a message like DSK, we:
1) internalize AcknowledgeMessage, which will 2) internalize AsyncMessage, which will 3) internalize AbstractMessage, which will 4) read flags for AbstractMessage and deserialize the specified objects and then 5) read flags for AsyncMessage and deserialize the specified objects and then 6) read flags for AcknowledgeMessage and deserialize the specified objects
In order to read the flags and deserialize the specified objects, I use:
def populate(source, fields)
flags = []
loop do
flags << source.read(1).unpack('C').first
break if flags.last < 128
end
if fields && !fields.empty?
fields.each_with_index do |list, i|
list.each_with_index do |name, j|
if flags[i].ord[j] > 0
data = RocketAMF.deserialize(source, 3)
send("#{name}=", data)
end
end
end
end
end
The flags[i].ord[j] syntax checks bit j of byte i in the flags array. If it's set, then that specific object needs to be deserialized.
Since some bit fields correspond to bytearrays that need to also be accessible as UUID's, some helper methods are needed, such as:
def clientIdBytes=(obj)
@clientId = to_pretty_uuid(obj)
@clientIdBytes = obj
end
In order to handle the other direction, I added others such as:
def clientId=(obj)
@clientIdBytes = to_binary_uuid(obj)
@clientId = obj
end
I'm not sure if this is still helpful or relevant, but I send it over just in case it's helpful. I've also attached a few
Cheers,
Steve Shreeve
ps - Here is the content of the message included in one of the images, if it's helpful...
00 03 00 00 00 01 00 0B 2F 33 2F 6F 6E 52 65 73 75 6C 74 00 00 FF FF FF FF 11 0A 0B 07 44 53 4B 09 62 6F 64 79 06 83 43 3C 65 6E 76 3A 45 6E 76 65 6C 6F 70 65 20 78 6D 6C 6E 73 3A 65 6E 76 3D 27 68 74 74 70 3A 2F 2F 73 63 68 65 6D 61 73 2E 78 6D 6C 73 6F 61 70 2E 6F 72 67 2F 73 6F 61 70 2F 65 6E 76 65 6C 6F 70 65 2F 27 3E 3C 65 6E 76 3A 48 65 61 64 65 72 3E 3C 2F 65 6E 76 3A 48 65 61 64 65 72 3E 3C 65 6E 76 3A 42 6F 64 79 3E 3C 76 61 6C 69 64 61 74 65 53 65 73 73 69 6F 6E 52 65 73 70 6F 6E 73 65 20 78 6D 6C 6E 73 3D 22 75 72 6E 3A 63 6F 6D 3A 6D 79 63 61 3A 73 69 22 3E 3C 72 65 73 75 6C 74 3E 74 72 75 65 3C 2F 72 65 73 75 6C 74 3E 3C 2F 76 61 6C 69 64 61 74 65 53 65 73 73 69 6F 6E 52 65 73 70 6F 6E 73 65 3E 3C 2F 65 6E 76 3A 42 6F 64 79 3E 3C 2F 65 6E 76 3A 45 6E 76 65 6C 6F 70 65 3E 11 63 6C 69 65 6E 74 49 64 06 49 33 39 35 39 34 44 41 41 2D 41 39 30 39 2D 35 32 30 42 2D 43 42 44 32 2D 44 41 41 41 35 37 39 46 45 30 43 33 1B 63 6F 72 72 65 6C 61 74 69 6F 6E 49 64 06 49 35 46 31 41 34 41 42 37 2D 38 36 43 41 2D 33 32 44 35 2D 39 46 41 46 2D 44 31 32 30 42 33 37 46 45 31 31 37 17 64 65 73 74 69 6E 61 74 69 6F 6E 01 0F 68 65 61 64 65 72 73 0A 0B 01 01 13 6D 65 73 73 61 67 65 49 64 06 49 33 39 35 39 35 37 34 43 2D 37 42 30 44 2D 34 37 31 36 2D 45 39 37 39 2D 41 36 36 43 45 45 39 31 34 30 42 41 15 74 69 6D 65 54 6F 4C 69 76 65 04 00 13 74 69 6D 65 73 74 61 6D 70 05 42 73 06 CA FD BA B0 00 01
----- Original Message ----- From: "warhammerkid" reply@reply.github.com To: steve.shreeve@gmail.com Sent: Tuesday, June 07, 2011 10:08 PM Subject: Re: [rocketamf] Support externalized messages (#1)
I'm trying to maintain a high test coverage ratio and I don't have access to BlazeDS, so I wouldn't be able to create tests for this. If you could write some test or at the very least add some binary data dump to
spec/fixtures/object
I can certainly investigate merging this in.Is there any reason you didn't reuse the existing
AbstractMessage
class inlib/rocketamf/values/messages.rb
?Reply to this email directly or view it on GitHub: https://github.com/rubyamf/rocketamf/pull/1#issuecomment-1324049
The provided binary dump doesn't mark class DSK as externalizable. When I converted it to a data file and deserialized it without any changes to RocketAMF, it properly identified all of the fields by name and deserialized it.
env = RocketAMF::Envelope.new.populate_from_stream(File.read('input.dat'))
puts "'"+env.messages[0].data.type+"'"
# => 'DSK'
puts env.messages[0].data.inspect
# => {:messageId=>"3959574C-7B0D-4716-E979-A66CEE9140BA", :timestamp=>1307493522347.0, :correlationId=>"5F1A4AB7-86CA-32D5-9FAF-D120B37FE117", :timeToLive=>0, :headers=>{}, :clientId=>"39594DAA-A909-520B-CBD2-DAAA579FE0C3", :body=>"<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Header></env:Header><env:Body><validateSessionResponse xmlns=\"urn:com:myca:si\"><result>true</result></validateSessionResponse></env:Body></env:Envelope>", :destination=>nil}
Do you have any other dump that actually contains the bit fields and externalizable flag?
Oops! Try this message... it should be marked as externalizable:
00 03 00 00 00 01 00 0C 2F 33 33 2F 6F 6E 52 65 73 75 6C 74 00 00 FF FF FF FF 11 0A 07 07 44 53 4B A1 03 06 83 3F 3C 65 6E 76 3A 45 6E 76 65 6C 6F 70 65 20 78 6D 6C 6E 73 3A 65 6E 76 3D 27 68 74 74 70 3A 2F 2F 73 63 68 65 6D 61 73 2E 78 6D 6C 73 6F 61 70 2E 6F 72 67 2F 73 6F 61 70 2F 65 6E 76 65 6C 6F 70 65 2F 27 3E 3C 65 6E 76 3A 48 65 61 64 65 72 3E 3C 2F 65 6E 76 3A 48 65 61 64 65 72 3E 3C 65 6E 76 3A 42 6F 64 79 3E 3C 67 65 74 43 6F 6E 66 69 67 53 74 72 69 6E 67 52 65 73 70 6F 6E 73 65 20 78 6D 6C 6E 73 3D 22 75 72 6E 3A 63 6F 6D 3A 6D 79 63 61 3A 73 69 22 3E 3C 72 65 73 75 6C 74 3E 34 38 3C 2F 72 65 73 75 6C 74 3E 3C 2F 67 65 74 43 6F 6E 66 69 67 53 74 72 69 6E 67 52 65 73 70 6F 6E 73 65 3E 3C 2F 65 6E 76 3A 42 6F 64 79 3E 3C 2F 65 6E 76 3A 45 6E 76 65 6C 6F 70 65 3E 05 42 73 02 41 54 59 E0 00 0C 21 88 14 A0 67 FE 0D 3A 9C A2 74 4A AE D9 BD 7B 0B 0C 21 88 17 EE F6 BE 0D 84 62 17 F1 38 B6 A4 34 14 DE 02 0C 21 7B B0 1B C0 C8 36 8F 4D 7B 47 24 15 43 35 71 09 00
It should have the following YAML output when processed:
--- !ruby/object:RocketAMF::Envelope amf_version: 3 headers: []
messages:
!ruby/object:RocketAMF::Message
data: !ruby/object:RocketAMF::Values::AcknowledgeMessage
body: <env:Envelope
xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'>env:Header/env:Headerenv:Body<getConfigStringResponse
xmlns="urn:com:myca:si">
correlationId: 7BB01BC0-C836-8F4D-7B47-241543357109 correlationIdBytes: !ruby/object:StringIO {}
destination: headers: {}
messageId: 8817EEF6-BE0D-8462-17F1-38B6A43414DE messageIdBytes: !ruby/object:StringIO {}
timeToLive: 0 timestamp: 1306275431838.0 response_uri: "" target_uri: /33/onResult
I downloaded the BlazeDS source to double check the implementation and added support for the shorthand AcknowledgeMessage, CommandMessage, and AsyncMessage to the existing message classes. Thanks for the data dump! Let me know if you run into issues with the new code.
Great! Thanks, man.
I am hitting a BlazeDS server that is wrapping a Java based server. I see a lot of the messages with types of "DSK", "DSA", "DSC", etc. These look like a shorthand message type, that uses bitmapped fields instead of string-based keys. After looking through the reference implementation and also the source code for Charles (the web proxy), I added support so that RocketAMF can deserialize these messages. For certain fields, such as messageIdBytes (which is a byte array), I have adjusted the setter methods to also set a human-readable version called (for example) messageId (which is just a string version of the UUID from the byte array). For completeness, a reverse method should be added so that if you set messageId, it'll create a 16-byte byte array and store the bytes directly.
By the way, one other random helper (not sure if this is necessary, but I added it) is this:
convenience for instantiating TypedHash objects
Hash.class_eval { def +@; RocketAMF::Values::TypedHash.new("").update(self); end }
Let me know if any of this is helpful.
Thanks,
Steve Shreeve