IPM Format EBCDIC, RDW

[Zellcl]

Hi, i have a problem with a ipm file from mastercard, because the format of the file is distinct to the structure 1014 (EBCDIC, RDW), then the process indicates this error " line 165, in _iso8583_to_dict message_type_indicator, binary_bitmap, message_data = struct.unpack(struct.error: bad char in struct format". There is another way to read a file with this EBCDIC structure?

[adelosa]

My understanding is RDW format is not 1014 blocked. Try adding the --no1014blocking option to your command and let me know how you go.. Otherwise you will need to provide more details so I can replicate the issue.

[Zellcl]

by adding the --no1014blocking option to the command gets an other error.

"C:\Users\gabriel.canales\Desktop\IPM>mci_ipm_to_csv --no1014blocking --in-encoding cp500 --out-encoding utf_8 4.ipm Traceback (most recent call last): File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts\mci_ipm_to_csv.exe__main.py", line 7, in File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\cardutil\cli\mci_ipm_to_csv.py", line 10, in cli_entry cli_run(vars(cli_parser().parse_args())) File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\cardutil\cli\mci_ipm_to_csv.py", line 22, in cli_run mci_ipm_to_csv(in_ipm=in_ipm, out_csv=out_csv, config=config, kwargs) File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\cardutil\cli\mci_ipm_to_csv.py", line 79, in mci_ipm_to_csv dicts_to_csv( File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\cardutil\cli\mci_ipm_to_csv.py", line 50, in dicts_to_csv for data_item in data_list: File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\cardutil\mciipm.py", line 315, in next__ vbs_record = super(IpmReader, self).next__() File "C:\Users\gabriel.canales\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\cardutil\mciipm.py", line 272, in next record = self.vbs_data.read(record_length) ValueError: read length must be non-negative or -1"

so, what details you need to replicate the issue?

[adelosa]

Attach a file with test data and the command that caused the issue. If you have issues attaching file to GitHub issue, send to me at adelosa@gmail.com.

[Zellcl]

Hi, this is the ipm file i can´t decode with cardutil. Test.zip and the command i used is this. mci_ipm_to_csv --in-encoding cp500 --out-encoding utf_8 test.ipm or mci_ipm_to_csv --in-encoding cp500 --no1014blocking --out-encoding utf_8 test.ipm

[adelosa]

The package documentation provides an overview of the 2 formats provided -- vbs and 1014 blocked. https://cardutil.readthedocs.io/en/latest/api.html#mastercard-file-formats

Your file is neither. The issue looks like you are missing the record lengths (RDW) that precede each record.

For example, the start of your file is as follows (CP500 - this is hex, first 16 bytes) F1 F6 F4 F4 80 00 01 00 00 01 00 00 02 00 00 00 -- 1644 then bitmap. This should have a record length before the 1644 Mastercard MTI field.

If you are pulling these files off a mainframe using FTP, then it will tend to drop the RDW.. To force it to pass, you need to tell FTP to not drop them. I think a site command like this is what will fix it. https://www.ibm.com/docs/en/zos/2.2.0?topic=protocol-rdw-ftp-client-server-statement

Let me know how you go.

[Zellcl]

after raising the problem with mastercard, the problem with the ipm files, was the format sent, ,after correcting the files the process runs smoothly. so thank´s a lot for your help! and keep up the good work with this project

[adelosa]

Hi @Zellcl. Thanks for the update I am interested to know.. If the file you received was from Mastercard, is it a valid file format. If so, I can add support for that file type. Off the mainframe, variable length records without RDW are common but I was not aware this was an option available from mastercard. If so, can you tell me how they describe that file format so I can locate the manuals.

[Zellcl]

@adelosa , sorry for the delay in response. mastercard explained to us, they exist for them 3 formats of IPM, that they can send to process. formats are: EBCDIC with RDW EBCDIC without RDW EBCDIC 1014

we currently work with the format EBCDIC 1014.