Wrong regions

[DKanarsky]

I have opened the issue #29 and i have also tried to comment exception code. But the issue seems to be deeper. Besides some unrecognized regions there are wrong detected recognized regions. It fails for example when it's trying to read CardNumberRegion and returning something like "???j?????\u001f??51". I tried to figure out what's going wrong comparing executing on "good" and "bad" files. I found out that sometimes it can find region's identification bytes (magic) twice, but the second one is "just the signature". When it processes "bad" files it also can find "the second" magic but the first byte of this second region doesn't equils 0x01 it equils 129 (int) and it tries to process it and it fails. I also tried to read docs, but found nothing about it. Ohh... And i have no other way to open and see the data inside those files

[DKanarsky]

Ok. If i set first byte = 129 (int) to be processed as "just the signature" it works (at least doesn't crash). It can't recognize some data:

<VehicleRegistrationNation>0</VehicleRegistrationNation>
<VehicleRegistrationNumber>?????????????</VehicleRegistrationNumber>

But most of data seems to be read right. codepage variable for VehicleRegistrationNumber is set to 255 (int). Does it mean that it starts with FF and the region is empty?

[jugglingcats]

I didn't encounter this problem before. I wonder if your file is corrupt somehow, but it sounds like this is happening with multiple files. Are you sure to process driver and vehicle cards with the right classes and not mix them up? Is the issue with driver or vehicle files?

The way the code is designed, it looks for a magic at the top of the file and consumes each region it finds according to the region configuration. This should leave the read head precisely at the magic for the next region, so as long as all magics are defined you should not see the error. But of course if a region is not fully processed the read head will be in the wrong place and bad things will happen. It will scan forward and may randomly find two bytes that are a magic but which is not in fact a proper region. That is why @davispuh put the exception code in I suspect.

I hope this helps. Get yourself a good hex viewer and the spec! If I get some time I'll run some old files through the latest code and check for issues...

[jugglingcats]

I tested some old files and didn't see any issues.

I've made some changes. I've added some messages when magics are not recognised.

I've also added a simple command line prog to scan folders for driver or vehicle files, or you can tell it to process a particular file.

Let me know how it goes.

[DKanarsky]

I opened my .ddd files in HEX viewer and examined them. I found out that all of them don't have "DF Tachograph" (ID 0x0500) and "EF Application_Identification" (ID 0x0501) regions (perhaps not only these regions). Region "EF Card_Certificate" (ID 0xC100) goes right after "EF IC" (ID 0x0005) and despite all it works. But some files even don't have "EF Card_Certificate" (ID C100). There goes uncrecognized code 0xC200 after "EF IC" (ID 0005) region and it seems to spoil next parsing. I went through the documentation and didn't find any information about 0xC200 region. I haven't updated my repository yet and haven't seen your changes. Anyway, thanks for advice.

[DKanarsky]

I tried to fix the problem with files having some additional/wrong information (regions). Firstly, I found out that two bytes "magic" window should have byte step or the window can be off-set. For example: ... | 00 05 | 20 00 00 ... next magic is: ... 00 05 | 20 00 | 00 ... where 05 20 - region identificator and | XX XX | - "magic window" Secondly, I added additional byte check after region identificator. It seems to work for me now. Sorry for my English)

[jugglingcats]

Closing for now - feel free to re-open or better still create a pull request if issue remains