Gen 2, Summarize informations

[Phoryn]

Hello

I want to summarize and consult all information which I need to write update to gen2.

-The structures of bytes for the segments is the same like in gen1. -In the segment is only one change, in new generation type can be also 0x02(content) and 0x03(sygnature). -On the one card might be also types of gen1 and gen2

On the beggining I thought that I can compare length of data "application_Identification" becouse it is exacly 10 byte in gen1(pic.2) and 15 byte in gen2(pic.3) . After select generation, choosen config. But I found information that in the card can be mixed information (gen1 and gen2 segments).

*"segment" mean one block of data like "application_Identification"

*@Dilemma725 wrote a program which to be able to check wchich generation card is. In his code he checking first and second byte of all card byte data. -if first 0x76 and second 0x06 - G1 -if first 0x00 and second 0x02 - G2 Why? First block of data is the same for both generation(pic1)

My question is: Did the best way do update this application to generation 2 is getting each segment one by one and checking his type?

Originally posted by @Phoryn in https://github.com/jugglingcats/tachograph-reader/issues/42#issuecomment-1195802860

[cbrede]

Hi,

the drivercards (gen1) I saw until now starts with 0x76 0x01.

If you want to restart this project I think that we have to identity the type of the data first (Drivercard or Tacho then Gen x) and load the appropriate .config

I also make some research on and move it .net Core and make some clean ups.

[belamenso]

Hi, I probably need to implement partial support for the version 2 of the specification. Can someone share with me at least a few gen2 .ddd files? I'm having trouble finding them. I promise that I will not publish them anywhere and will use them only for the development, which I might contribute back if it gets somewhere.

@jugglingcats @davispuh @rimutaka @diogocp

[davispuh]

I don't work at company that works with Tachographs anymore. Pretty sure they have gen2 files but it's not possible to share them because they contain customer data and GDPR all that.

[mpi-wl]

*@Dilemma725 wrote a program which to be able to check wchich generation card is. In his code he checking first and second byte of all card byte data. -if first 0x76 and second 0x06 - G1 -if first 0x00 and second 0x02 - G2 Why? First block of data is the same for both generation(pic1)

76 01/21, 76 02/22, 76 03/23 etc... are identifiers of data blocks in the ddd file from the tachograph (01 - 1gen, 21 - 2 gen).

In the ddd file from the driver card, it may start with 00 02. This is the EF ICC identifier. This data block is not mandatory, so it doesn't have to be at the beginning of the file. Also, the order of the blocks is not mandatory.

[OscarKro]

What file is the file you used to get those pictures? We as a company are also busy trying to decipher gen 2, and when I figure it out I plan on implementing it on the Jugglingcat lib so nobody ever has to do this work again hopefully for the coming years.

[jugglingcats]

@OscarKro the images from @Phoryn will be from the specification. I uploaded the gen1 spec to this repo. If someone can link me to the gen2 spec am happy to upload it too. I'm not working on this project any more but happy to facilitate and can add collaborators if/when appropriate!

[OscarKro]

Yeah but thats the problem. I cant find a similar spec like on the first generation anywhere. I used ECE/TRANS/SC.1/2006 for the first generation, and then appendix VII gives a pretty clear documentation on what the file looks like when downloaded. But there isnt a similar document to be found for the second generation as far as i can tell

[graealex]

I can upload some test data that are free of any identifying information.

[OscarKro]

That would probably be very helpfull!

[graealex]

Here is sample data. Be advised that the certificates used are test certificates, so they will not properly validate. The dataset contains driver cards and transfers from a vehicle unit. None of the data contain real persons, so there are no privacy concerns. SampleData.zip

[OscarKro]

Wow thanks man! great help. To give some context on what i'm doing: I'm currently working for a small company where we create devices to remote download the tachographs in trucks. I've been asked to check how we can validate the downloaded files. We were using the @jugglingcats lib for testing and finding out how it all comes together. It worked great for GEN 1. Now i've been asked to look into GEN 2. For research purposes i'm now creating some simple python code to pull the data from the gen 1 and 2 files, gen 1 works, but now still gen 2. When that is done, I need to find out how to validate it all, but thats the real part I can't make heads or tails of. The getting data from the GEN 2 file is already succeeding bit by bit. I find the record arrays actually pretty handy dandy.

[graealex]

Well, you can run through all of the data structures and verify plausibility, to one degree or another, but generally it is enough to verify the signatures to have full confidence in the download not having been corrupted or tampered with.

[OscarKro]

Yeah, in the Netherlands we say "je slaat de spijker op zijn kop". Which roughly translates to "you hit the nail on its head with your comment". Verifying the signatures is exactly the part I don't understand. But I'll get to it someday, and when I do I want to update the @jugglingcats library so nobody ever has to do this sheit again.

[davispuh]

Yeah but thats the problem. I cant find a similar spec like on the first generation anywhere. I used ECE/TRANS/SC.1/2006 for the first generation, and then appendix VII gives a pretty clear documentation on what the file looks like when downloaded. But there isnt a similar document to be found for the second generation as far as i can tell

See #42, there I linked latest doc at the time - https://eur-lex.europa.eu/eli/reg_impl/2016/799/2018-04-17

It's not exactly that readable but info is there you just need decypher it in your mind :joy: back then I went over it many times and got pretty good understanding but it was many many years ago.

[davispuh]

Looks like this https://eur-lex.europa.eu/eli/reg_impl/2016/799/2023-08-21 (interestingly date in future :D) is the most newest one now.

[Camillus83]

@mpi-wl Is there any chance that you will share some information how to read gen2 files? I mean, I can read Gen1 files based on that information. But when I'm trying to read a gen 2 files (that ones which starts with 76 21) I cannot find any data.

[graealex]

@mpi-wl Is there any chance that you will share some information how to read gen2 files? I mean, I can read Gen1 files based on that information. But when I'm trying to read a gen 2 files (that ones which starts with 76 21) I cannot find any data.

0x7621 is "Transfer data overview structure generation 2, version 1 (TREP 21 Hex)" - here is a direct link to the information: CELEX:02016R0799-20230821 (page 343 if it doesn't open on its own)

It will look something like this:

[Camillus83]

Hi @graealex, thanks for your response.

I must have missed this piece in the documentation, which isn't difficult since it is 600 pages long lol. I don't know if I understand this correctly, so there are not only Gen1 and Gen2 files but Gen1, Gen2V1, and Gen2V2?

My problem is that I can't read Gen 2 files manually. I have a couple of files, for example, gen 1 which starts with 0x00 and 0x02. This type of file seems quite understandable to me. When I wanted to find DriverCardApplicationIdentification, i just take a look on documentation. It says that Application_Identifications starts at 0501. So I open that file in some hexViewer and read that manually like here.

But when I've opened the second file which starts with 0x76 and 0x21 nothing make sense to me. In documentation DriverCardApplicationIdentification also starts at 0501 but with SFID 1. There is information that SFID is given in decimal, so I've transferred it to the hex, and cannot find many bytes 0501 but without that SFID or any other informations which make sense.

Am I doing something wrong? On what should I focus on?

[mpi-wl]

Files with sections starting with 76 01 or 21, 76 02 or 22, 76 03 or 23... etc. are tachograph readings. These files have a different structure than the driver card. 05 01 is the Application_Identifications section in the driver card.

[graealex]

As @mpi-wl correctly wrote, 0x7621 indicates that this is a download directly from a VU (vehicle unit - the actual device memory). All these files come as .DDD files, but can be just a card or from a VU.

And yes, there is Gen 1, Gen 2 V1 and Gen 2 V2. And for VU downloads, the format has actually changed quite a bit between Gen 1 and Gen 2, notably because every block is now a record array, and never just the block itself.

[OscarKro]

I have written all the code for the company I work for which parses all three gens into usefull data objects so we can just extract everything. I happen to know a little bit about it now, but I don't really understand your question? Yes there are multiple generations, mainly gen 1 and gen 2, and a latter gen 2.1 where gen 2.1 mostly just adds GNSS data to different fields. Could you summarize what your question is?

[Camillus83]

I'm ashamed to admit but I was sure that the files with the extension .ddd are only files from the driver's card. This completely changes the form of things, because I tried to read files that began with 0x76 and 0x21 in the same way as the driver's card.

For a few days, I'm trying to understand the structure of these files, file types, manual reading, etc. Thank guys, this information is very helpful for me. Trying to build a parser, like many people before me, from what I can see in this repo.

[graealex]

@Camillus83 There is no shame in not knowing that, because it's not like any of the tachograph stuff is particularly well documented. Generally there is a download protocol ("Appendix 7 DATA DOWNLOADING PROTOCOLS" in the EU spec) where a device requests blocks from the VU to download, and one such answer is the 0x7621 "Transfer data overview structure generation 2, version 1 (TREP 21 Hex)" block. These answer blocks then get written into a DDD file.

[OscarKro]

Ah okay, No, as said before, Both the driver card or the tachograph mass memory can be downloaded via the tachopgrahs interface. Both are called 'ddd' files. But they are completely different. The Driver card looks exactly like the structure described in the documents. And is basically just all card data in a specified order. LIke any card file system. Gen 1 memory files are very different from Gen 2. Where gen 1 is just bytes in a specified order and gen 2 consists of so called 'record arrays' where each record array first specifies the type, length and amount of fields giving you the chance to see the length of a section before actually looking at the data. The data then is about the same as it is in Gen 1. But there are some differents between all data in gen 1 and gen 2, so be mindfull of that. Gen 2 mostly has more data for specific fields then gen1. Gen 2.1 is about the same as gen 2 except for added GNSS data. If you need more detailed info I can help.

Someday I will upload all my code giving everyone an easy to use library to parse gen 1, 2 and 2.1 but for now im under NDA. I have to rebuild it in a different way to do this, but yeah. Time and all.

[Camillus83]

@graealex @OscarKro

As I said before, thank you for the explanation. Now I am able to read the files.

It seems that the generation changes and the rebuilding of the structure made a lot of mess in the existing solutions and in the creation of a compatibility solution with all generations. Going back to work on it, most likely I will have more questions later. So don't be surprised by notifications 😸

[Camillus83]

Hi Guys, it's again me.

After couple of weeks I've finally understood and built parser for Driver Cards. Now I started working on VU files, and again have some problems and misunderstandings. I've started with VuOverview Gen1 and everything was clear and fine.

First main issue was when I was trying to read VuActivities block from VU Gen1 file. Reading the documentation we can deduce that VuActivities Block is built by

• TimeReal (4 bytes)
• OdometerValueMidnight (3 bytes)
• VuCardIWData (x bytes)
• etc.

Block structure from my file looks like: 76 02 63 D1 C2 7F 05 7F 45 00 01 XX XX XX Where:

• 7602 is block ID
• 63 D1 C2 7F is TimeReal
• 05 7F 45 is OdometerValueMidnight
• 00 01 is noOfIWRecords

But when I've opened that file in HexEditor I saw that there are many more blocks which looks exactly the same like VuCardIWRecord and have the same prefix as 00 01.

It seems to me that I don't understand or wrongly reads file, because later other blocks after parsing have meaningless data.

[OscarKro]

well I would have to look it up in the code and documentation again, its been a while for me. But vehicle unit card iw record stands for "insertion and withdrawal" record if i remember correctly. And those can be multiple. hense the number of iw records uint16_t there.

You cant deduce the data of gen 2 files by just looking at the hex. You will see that all data records look more or less the same, as the whole idea of gen 2 and up files are made using the same first few bytes, indicating the type, length of data and amount of data blocks. Its only after the first few bytes that it is different. So if you just look at the hex and say 'hey, i see a lot the same', yeah that's the idea.

[Camillus83]

@OscarKro

According to the first part of message, the data blocks are built with noOfRecords, then recordsData(noOfRecords*recordData). That's why I'm surprised that noOfRecords says there is only 1 record like this, but on hexEditor I can see that next is almost identical structure, probably only values of TimeReal bytes are different.

According to the second part of your message, in DriverCard i've recognized generations based on the size etc. But In VU it's probably kind easier. 76 01 - Activites Gen1 76 21 - Activities Gen 2 etc.

[graealex]

Can you share the data you are trying to parse, or is it a privacy concern?

[Camillus83]

Unfortunately, it's confidential data from the customer :/

[graealex]

As always. Btw that was the reason I got commissioned to create a library to read and write DDD files - so they can artificially create driver card and VU storage data for demonstration purposes, without privacy or confidentiality concerns. Anyway, without data, I can't really tell you what might be the problem. I also never properly tested against Gen 1, and as I have written before, Gen 1 vs 2 VU data has a pretty drastic change, as ALL structures are placed as record arrays in Gen 2. Maybe this screenshot helps understanding the structure and what to expect:

[OscarKro]

Yes VU files are a lot easier to recognize gen 2. I'm currently at work, if I find the time and see no further solving in this thread about your issue i'll try to look it up in my code to see how i did it.

[Camillus83]

@graealex @OscarKro VuBlocks gen2 at first glance looks more civilized, but we'll see lol. Anything can be expected when working with those tacho files. Every time I think i already understand it, they show me that I don't.

Thanks guys, I'm going back to work on it, hoping that maybe I missed something small.

Thanks for screenshot, will definitely come in handy when working with gen2.

[graealex]

Just structurally, Gen 2 VU isn't much different. So you'd expect a similar visual result, just that the parsing is different because of record arrays. In Gen 1, where multiple records might be present, like for VuCardIWRecord, you have a parent VuCardIWData structure that simply has a single ushort giving the number or records to expect, whereas in Gen 2, all records are put in a RecordArray which specifies type, size and number of records, regardless of whether it actually makes sense for more than one record to appear.

[graealex]

@Camillus83 Maybe I understand your problem now - if you have 00 01 for noOfIWRecords, then that means you have to read one VuCardIWRecord (you have to determine the size to read yourself), and then the next 00 01 indicates that you also have one ActivityDailyRecord to parse next, because that is the next element in VuActivities.

[Camillus83]

@graealex Yes, I just did that and it parse almost well, got 14 activityChangeInfo records. (some issues with block sizes). But one thing still doesn't give me peace of mind that blocks which looks like VuCardIWDataRecord i'm parsing as a ActivityChangeInfoRecord (On hexEditor preview I can see the first and last name of driver). Also, all noOfRecords will be uint16? And there is a possibility to have few VuActivities blocks?

[graealex]

I think you have an offset problem. From what I can tell, a conforming VU would write at least one IW record (insertion-withdrawal), and two ActivityChangeInfo records, one for driver, one for co-driver, at 00h00 indicating what activity was chosen for that slot at the time. ActivityChangeInfo has no card holder information, so if you are seeing names, you probably haven't parsed the preceding IW records completely.

noOfRecords will be uint16

Not sure. Look at the EU reference and make sure you have the right data type. Also remember it's Big-Endian...

[sirasjad]

Thanks everyone for asking and answering questions here! I'm glad other people are having the same struggles as me. 😄

I have read through all previous comments and maybe I missed something, but I have a question. I'm currently working on a driver card parser for generation 1 and 2, and I want my program to detect if the DDD-file is gen 1 or 2.

At first I thought this information was provided in "DriverCardApplicationIdentification" under the "CardStructureVersion", but for some reason the value is always 0. The documentation says that it should be "00" for Gen1 and "01" for Gen2, right? This is frustrating, I'm not seeing the "01" value even though I'm very sure that the provided file is Gen2.

So my question is, how can I detect the generation version and separate between gen 1 and 2?

I appreciate all help, thanks in advance!

[Camillus83]

@graealex @OscarKro Hi again guys, I hope you are both well and you haven't had enough of me yet. I have a question, have you solved the signature verification problem? I'm working on it for a few days, and still got a few things I'm not sure about.

I mean, all Gen2 and Gen2V2 files are signed with ECC algorithm. For example, If I want to verify signature of CardDownload block from DriverCard, i need to:

• Take a data from that block, in this example it would be only bytes corresponding to TimeReal so it would be 4 bytes.
• Take a signature from that block, in this example it would be 128 bytes on the end of the file.
• Take a Public Key remainder from CardCertificate block (58 bytes)

Then I need to use for example OpenSSL or Cryptography module from Python to verify the signature BUT: What ElipticCurve type should I use? It is possible to read the type from PublicKey? I've used SECP256R1 or BrainpoolP256R1 but got some error about Unsupported eliptic curve point type. How should I store the publickey, signature, data? Now i'm passing all of them as a string containg rawdata.

[Camillus83]

@OscarKro @graealex Is there any chance to contact with any of you guys? I stucked again with Signature Verification, for Gen2 ECDSA is quite easy and transparent. But RSA verification is pain, I've stucked on Certification Verification. There is a step " open Sign with CA Public Key: Sr'= X.CA.PK [Sign] " I got an Sign (First 128 bytes of CardCertificate) and public key generated from ERCA (n,e). But what means that open? Firstly i was thinking about encryption, but every encryption returns different message! Relating that to next step which tells check Sr' starts with ‘6A’ and ends with ‘BC’.

[graealex]

CVCs ("Card Verifiable Certificates") are defined in ISO/IEC 7816-8. "Certificate verification and unwrapping" is described in detail in ISO/IEC 9796-2. Unless you find a prior implementation somewhere, I don't think you have an easy way around simply buying these specs, especially the latter. The document from which you are citing only gives a very rough overview, basically only half a page describing the method.

Edit: Here is an old version of the aforementioned document free of charge on Internet Archive: https://archive.org/details/gov.in.is.iso.iec.9796.2.2002

[Camillus83]

@graealex Thank you very much for your reply. I'm starting to read with the hope of finally moving something along. These things make you depressed...

[graealex]

Well, while standardization is a good thing, the fact that nearly all of these documents are behind a paywall makes it hard again. Also when you buy a document, your name will be embedded in the document, so that's why no one is sharing them. Besides the legal aspect, obviously.

[Camillus83]

@graealex

I read the document you uploaded and found the answer to how to verify the signature. It is explained quite clearly. Once again, a big thank to you.

Nevertheless, I am still having trouble with verification. Because documentation says that we need to get $J^*$.

Which is an $J^* = \Sigma^v \ mod\ n$. $\Sigma$ - integer value of given Signature $v$ - public exponent $n$ - the modulus

Signature 0x18, 0x21, 0xA5, 0x1B, 0xA8, 0xAA, 0xEE, 0xAC, 0x93, 0x2A, 0xB5, 0xDF, 0xC4, 0xF0, 0xD6, 0x72, 0x75, 0x65, 0x76, 0xEA, 0xAA, 0xD7, 0x6C, 0xAA, 0x0F, 0xB7, 0x13, 0xA8, 0x30, 0xEB, 0x78, 0x17, 0xA6, 0xDC, 0x36, 0x7A, 0x55, 0x26, 0x56, 0x90, 0xB9, 0xBA, 0x24, 0xD5, 0xD0, 0x86, 0x52, 0xDB, 0x7F, 0x5D, 0x0C, 0x84, 0xB7, 0x87, 0x58, 0x23, 0x0A, 0x2B, 0x22, 0xE2, 0xB6, 0x03, 0xA7, 0x04, 0x61, 0x58, 0xE4, 0x4F, 0x51, 0x3A, 0x17, 0xFB, 0x88, 0x5B, 0x1C, 0xB4, 0x58, 0x62, 0x7C, 0xD0, 0xB7, 0x13, 0xC0, 0xDA, 0xD9, 0xAB, 0xD5, 0x04, 0xA8, 0x62, 0xA8, 0x91, 0x07, 0x85, 0x65, 0x67, 0x5F, 0x77, 0x92, 0x38, 0xA7, 0xA7, 0x49, 0x8B, 0x38, 0x34, 0x7B, 0xDA, 0x1B, 0x99, 0x08, 0x87, 0x4D, 0x7E, 0x11, 0x31, 0x5E, 0xDF, 0x4A, 0xEE, 0xE7, 0xFB, 0xF5, 0x82, 0x25, 0xFD, 0xBE, 0x94 converted to integer value is

39871213657556712785743798399678659729728462930070485080026276996237931233957214655329281058127998080188609422467155620875643995926287856704772605574419436962902653026391993371019986085376311278191228766790165694547487105168431139174600037348108043676741863493392549357225865748487737088872217765342447831663

Public expotent: 65537 Modulus: 163970542987064517096427732657586873311121103259339977822075834226043831831992997085797193024881480944179959166654675047314960206132215825542205470319185636239474606486605079901001892267884946038281940713954094412760812467846977693082812813826815101351434431397964803402960143753961809885769074524190245968807

So with such large numbers, I don't know if there isn't some error along the way because it is $J^* = 39871213657556712785743798399678659729728462930070485080026276996237931233957214655329281058127998080188609422467155620875643995926287856704772605574419436962902653026391993371019986085376311278191228766790165694547487105168431139174600037348108043676741863493392549357225865748487737088872217765342447831663^{65537}\ mod\ 163970542987064517096427732657586873311121103259339977822075834226043831831992997085797193024881480944179959166654675047314960206132215825542205470319185636239474606486605079901001892267884946038281940713954094412760812467846977693082812813826815101351434431397964803402960143753961809885769074524190245968807$

Anyway, digs into it further and maybe something will come up. Big thanks

[graealex]

Take a look at these implementations: https://github.com/bcgit/bc-java/blob/main/core/src/main/java/org/bouncycastle/crypto/signers/ISO9796d2Signer.java https://github.com/bcgit/bc-java/blob/main/core/src/main/java/org/bouncycastle/crypto/signers/ISO9796d2PSSSigner.java

That's the most accessible example I could find, and implements all three schemes.

[aRRoYo4]

Hello, great work and it has been of great help to me. I have problems being able to correctly translate the coordinate value read from a file. 3bytes for each Lat and Lon, I convert them to decimal but that's where I stop because I don't know how to convert that decimal into the coordinate value according to the information in the documentation.

Can you Help me?

Example: GNSSPlaceGeoCoordinates Lat: 41116 /// Lon: 16776023 - Hex: Latitude - 00a09c /// Longitude - fffb57

[graealex]

@aRRoYo4

Example: GNSSPlaceGeoCoordinates Lat: 41116 /// Lon: 16776023 - Hex: Latitude - 00a09c /// Longitude - fffb57

Latitude seems plausible. You just divide by 1000, so 41116 becomes 41.116 - that could be Greece, Italy or Spain.

The other one is negative, represented as One's Complement. If you invert the bits (XOR with 0xFFFFFF), you get 0x4A8, or decimal 1192. Divide by 1000 again, gives you -1.192. Which would be here, a road in eastern Spain.

I could be wrong though, as the three MM.M denote the minutes, which isn't exactly just a decimal fraction. So maybe check that multiple coordinates are plausible.

[aRRoYo4]

Thanks @graealex ,

But we need extract first sign, and i don't know how.

First i convert 3bytes from latitude to decimal, but in longitude how i detect if the decimal negative?

you know any function in python to do this?

[graealex]

But we need extract first sign, and i don't know how.

I thought it was clear. If the first (MSB) bit is one instead of zero, then it's a negative number, and you need to invert the bits. Please read the article on One's Complement which I linked. That's literally how all signed numbers work in computers, with the exception that here you only have 24 bits instead of the typical 16 bit short or 32 bit int types.

Another shortcut would be to shift left 8 bits, then use normal signed int logic, and then shift right 8 bits again.

[mpi-wl]

@aRRoYo4 Are you sure you read GeoCoordinates correctly? Could you paste the entire GNSSAccumulatedDrivingRecord?