moov-io / iso8583

A golang implementation to marshal and unmarshal iso8583 message.
https://moov.io
Apache License 2.0
349 stars 104 forks source link

subfield decoding support #57

Closed vtolstov closed 3 years ago

vtolstov commented 3 years ago

some fields have TLV stuff, do you have plans to add ability to define subfield and parsing capability?

krishishah commented 3 years ago

In addition to TLVs, various Data Elements (DEs) also consist of subfields and subelements. It would be great to add support to pack/unpack these into the library.

Take an example of DE 3 (Processing Code). The ISO specification describes its structure as such:

image

However, Spec87 defines it as:

field.NewNumeric(&field.Spec{
    Length:      6,
    Description: "Processing Code",
    Enc:         encoding.ASCII,
    Pref:        prefix.ASCII.Fixed,
    Pad:         padding.Left('0'),
})

Though this is technically correct and will parse successfully, we could extract more granular data from messages by decoding their subfields directly. Note that this is a simplified example - however, they get a lot more complex especially for subelements. To extract subfield data using this library, I propose that we introduce a new field type as such:

package field 

// ComplexField represents a Data Element (DE)                                                                                                                                                                    
// with subelements or subfields within it.   
// This struct implements the 'Field' interface.    
// Open to naming suggestions.                                                                                                                                                                
type ComplexField struct {                                                                                                                                                                                        
    spec   *Spec                                                                                                                                                                                                  
    Fields map[int]Field                                                                                                                                                                                          
}   

func NewComplexField(spec *Spec) field.Field { ... }

// Logic to implement the 'Field' interface. 
func (*ComplexField) Spec() *Spec { }
...

The above could also technically be defined separately by the user if we wish to keep such logic out of this library. However, the design pattern above would still require an enhancement moov’s field.Spec struct definition to hold sub-fields as such:

type Spec struct {                                                                                                                                                                                                
    Length      int                                                                                                                                                                                               
    Description string                                                                                                                                                                                            
    Enc         encoding.Encoder                                                                                                                                                                                  
    Pref        prefix.Prefixer                                                                                                                                                                                   
    Pad         padding.Padder                                                                                                                                                                                    
+   Fields      map[int]Field                                                                                                                                                                                      
} 

The great bit about this design is that mapping of Spec.Fields to sub-fields in the ComplexField may then be left to the ComplexField to implement via SetSpec itself. In addition, this would be a non-breaking change.

It may then be used when defining a MessageSpecification as such:

spec := &iso8583.MessageSpec{
    Fields: map[int]field.Field{
        ...

        3: field.NewComplexField(&field.Spec{
            Length:      6,
            Description: "Processing Code",
            Pref:        prefix.ASCII.Fixed,
            Fields:  map[int]field.Field{
                1: field.NewString(&field.Spec{
                    Length:      2,
                    Description: "Transaction Type",
                    Enc:         encoding.ASCII,
                    Pref:        prefix.ASCII.Fixed,
                }),
                2: field.NewString(&field.Spec{
                    Length:      2,
                    Description: "From Account",
                    Enc:         encoding.ASCII,
                    Pref:        prefix.ASCII.Fixed,
                }),
                3: field.NewString(&field.Spec{
                    Length:      2,
                    Description: "To Account",
                    Enc:         encoding.ASCII,
                    Pref:        prefix.ASCII.Fixed,
                }),
            }
        }),
        ...
    },
}

Keen to hear what you guys think of the above. Are you currently able to extract information from fields such as DE 48? I haven't implemented the above myself but I'm open to contributing if required.

vtolstov commented 3 years ago

I think that this will be cool. Can you provide some pr to test proposed solution?

alovak commented 3 years ago

@krishishah thanks for such a detailed suggestion! It looks great!

alovak commented 3 years ago

It would be really nice if you can contribute :D Let me know if I can help you somehow.

krishishah commented 3 years ago

I'll try to get something out in the coming few days 😃

krishishah commented 3 years ago

Raised https://github.com/moov-io/iso8583/pull/68 to address part of this requirement. Unfortunately, it does vary a bit from my proposal above and involves breaking changes to the library.

Let me know what you guys think!

alovak commented 3 years ago

@vtolstov #68 was merged and released as v0.4.0. Please, let us know how it works for you.

alovak commented 3 years ago

Btw, there is no TLV support, but approach implemented in #68 can be used/applied for TLV fields as well.

jmptrader commented 2 years ago

The specifications of the main brands (VISA, Mastercard, for example) specify subfields that can be in different ways: fixed, variable, bitmap, bitstrings, tlv type, etc. Even subfields that are from another form subfields in turn. One way to deal with this is by using a recursive approach.