jugglingcats / tachograph-reader

Read driver and vehicle card data and convert to a more usable XML format
81 stars 62 forks source link

DTCO 4.0 Structure ? #42

Closed cbrede closed 4 years ago

cbrede commented 5 years ago

Hi, from 14 JUN 2019 new Tacho will be install in the trucks. There also will new drivercard. Has some already some test data? How will be the implemtation?

BR

Carsten

mpi-wl commented 4 years ago

I'm not intenting to check the signatures, rather to check all pointers and blocks in the files to make sure these are correct.

This is not a good solution. What if the length of the data received is correct but the content is incorrect? For example: the correct date is 5E C6 CF B0 (Thursday, 21 May 2020 19:00:00), it is enough that one byte 5E C0 CF B0 will be wrong and the result is different (Sunday, 17 May 2020 05:46:24).

Another example: 4C 61 73 74 20 6E 61 6D 65 = 'Last name'. Changing one byte 58 61 73 74 20 6E 61 6D 65 = 'Xast name'

When downloading data from older tachographs, such errors may occur. Also when downloading data remotely.

The signature is generated by the card / tachograph for correct data and only the signature verification confirms the correctness of the file. This can be compared to a checksum.

Dilemma725 commented 4 years ago

I am aware of this. The current verification is not perfect indeed. However, that does not mean that it is not a good solution. TISWeb marks files as "corrupt" when uploading the file to the server. The files that have been marked as incorrect up until now, were all structurally incorrect because of incomplete files (in our use case). This is due to the fact that driver cards are removed from the slot while downloading, tachographs powering down while downloading (dangerous goods transports) etc. In short, i catch all incorrect files with the structural test that i have now.

This does not mean that it cannot be improved of course. If there is a "simple" method to check the signatures of the files, that would be great.

If you have any info on how to proceed, that would be very welcome :)

mpi-wl commented 4 years ago

Are you a telematics system provider? To prevent errors in files caused by removing the driver card from the tachograph, or turning off the power supply (ADR), the device that downloads data from the tachograph before sending should check whether all data has been read.

I saw damaged files that were downloaded directly from the tachograph (old tachographs).

Checking signatures is described in Regulation (EU) 2016/799 https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1468399756621&uri=CELEX:32016R0799

I don't know C # (I check ddd files in JS) but it seems to me that this project has code for checking signatures in 1G files. This is modular exponentiation and SHA-1.

In the case of 2G, this is ASN.1 coding, cryptography of elliptic curves.

Check the documentation and ask.

Dilemma725 commented 4 years ago

Are you a telematics system provider?

Yes indeed. In practice we only retrieve the ddd files from the OBU and deliver it to the customer. This explains why I'm not particularly interested in the content of the files. I just have to make sure we deliver all the files.

To prevent errors in files caused by removing the driver card from the tachograph, or turning off the power supply (ADR), the device that downloads data from the tachograph before sending should check whether all data has been read.

Very true. Work in progress :)

Checking signatures is described in Regulation (EU) 2016/799 https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1468399756621&uri=CELEX:32016R0799

I don't know C # (I check ddd files in JS) but it seems to me that this project has code for checking signatures in 1G files. This is modular exponentiation and SHA-1.

In the case of 2G, this is ASN.1 coding, cryptography of elliptic curves.

Check the documentation and ask.

Thanks for that. If have have some spare time i'll look into this. We'll see :)

You sound very expercienced in the certificate/checksum/math stuff. In the past i worked on engine management systems (ECU's) and I met some guys that dazzled me with their knowledge about this stuff. Most of the time, i understand what needs to be done and it can code it just fine, but my experience does not lie in this area...

mariandaniel24 commented 4 years ago

@mpi-wl Ddd file for the VU 76 01, 76 02, 76 03, 76 04, 76 05 - gen 1 76 21, 76 22, ​​76 23, 76 24, 76 25 - gen 2

The VU ddd file contains the data of gen 1 or gen 2

@mpi-wl Hello, I also jumped on this train, working in JS, I can successfully read the Data for driver cards, but I'm a bit stuck on the VU, I know that the 4th byte represents the length of the current EF data (for driver cards) , is there any byte that represents the length for VU Gen1, what about Gen2 ?

In my sample VU gen1 file, the 4th byte shows length 14595 dec, but the length should be 720, is there something that I'm missing ?
I'm assuming that the 4th byte in this case represents something else. How can I check the length ? Thanks in advance !

mpi-wl commented 4 years ago

In ddd from the VU there is no information about the length (this is only in the data from the driver card). In ddd from the VU immediately after the ID are individual data. The data length is fixed and you know it from the documentation. For example the length of block 76 01 = 194 + 194 + 17 + 1 + (...) + 18 + 36 + 1(1) + (x 98) + 1(2) + (y 31) + 128 (1) - byte noOfLocks x - the number of noOfLocks (2) - byte noOfControls y - the number of noOfControls

image

AskmeKolaric commented 4 years ago

@ramasofficial Can you please send couple more files gen 2?

ramasofficial commented 4 years ago

Sure. Gen2.zip

ramasofficial commented 4 years ago

@ramasofficial Can you please send couple more files gen 2?

See above.

AskmeKolaric commented 4 years ago

@ramasofficial Thank you so much:)

AskmeKolaric commented 4 years ago

@mpi-wl Which protocol do you use when you read files?
Can you please help me out where to find, which response I get if it's not "90 00"? So far we got also "67 00" and "6c 00". Friend of mine maid for my Android app ( from 30 cards, 4 we cannot read), and we have problem with some cards, when we try to download them because of this response? I am now learning all the processes and trying to figure out how to solve this problem.

mpi-wl commented 4 years ago

@AskmeKolaric Your question is not about reading the ddd file, only about reading the driver card. I hope that jugglingcats will not notice that we deviate from the main topic.

I learned to read the driver card a few years ago. I created a small C++ program using the winscard library. I don't choose the protocol - winscard does it for me:

image

Once I was able to communicate with the driver card for Android. This is difficult. I need to find this code and remember how I did it.

My guess is that you get the 6700 and 6C00 response for the READ BINARY command? Search for TCS_43 in the documentation.

image

AskmeKolaric commented 4 years ago

@mpi-wl You are here the only group that I find that know and understands this topic. Our Android app works goods. With 5 cards, I have more than 300 reds. I have already a DOS app, that reads all the Driver .ddd file. Work in progress is for Workshop cards, control cards, company cards. Thanks to your help, I will try to create if statement for all this response.

ghost commented 4 years ago

Do someone already have something for the Generation2-Files? I would be very thankful for an answer :)

ramasofficial commented 4 years ago

Do someone already have something for the Generation2-Files. I would be very thankful for an answer :)

Please see above, i attached 3 files.

RazviPatcas commented 4 years ago

This converter can be used for the new tachograph ? (SMART Tachograph)

RazviPatcas commented 4 years ago

@jugglingcats

Do you know if this converter works for the new tachograph files (SMART tachograph) ?

AskmeKolaric commented 4 years ago

@mpi-wl Hello, any chance that you can find your old java(android) code. I have small problems with "performHashFile".

mpi-wl commented 4 years ago

@mpi-wl Hello, any chance that you can find your old java(android) code. I have small problems with "performHashFile".

What is your problem?

RazviPatcas commented 4 years ago

@mpi-wl

Do you know if this converter works for the new tachograph files (SMART tachograph) ?

mpi-wl commented 4 years ago

@RazviPatcas This project doesn't support 2G structure.

@AskmeKolaric My android program is incomplete. When I was able to connect to the driver card and read one EF file, it stopped working. I read the entire driver card in my other program (c++). Describe exactly what your problem is - maybe I can help.

AskmeKolaric commented 4 years ago

@mpi-wl When the app red file on "05 04", when I have to make PERFORM HASH on that file, the app brake down on some driver cards. For some cards, work perfectly. I am trying to find a solution how to make an app to work on every card:) I am not exactly sure, what I have to test to find a solution for it. Thank you for your help.

mpi-wl commented 4 years ago

Do you read a 1G or 2G driver card? For 2G, an algorithm selection has been added (SHA-1, SHA-256, SHA-384 and SHA-512).

Do you do it in the following order?

  1. Select file
  2. Perform Hash of File
  3. Read Binary
  4. Compute Digital Signature

Are readable cards and non-readable cards issued by the same authority?

What is the response of the Perform Hash of File (SW1, SW2) command when the error occurs?

AskmeKolaric commented 4 years ago

@mpi-wl At the moment 1G. We use the same order in the app. Non-readable cards are not issued by the same authority. When I selcet file "0504" i get '9000' , after that goes, perfomHasha and i get "bulk response failed with status 80 and error: 1'

mpi-wl commented 4 years ago

Do you send commands to the card in a similar way?

image

I don't know if I can help you. Send me an email: mpi@vp.pl

jugglingcats commented 4 years ago

@mpi-wl, @AskmeKolaric, it seems you are having a conversation about different software? Let's keep the conversation here about tachograph-reader

mpi-wl commented 4 years ago

@jugglingcats You're right. The conversation is about reading a driver card to create a ddd file. This conversation will continue via email.

Sorry.

WallHackUp01 commented 4 years ago

Hi ! Did someone succeed to make a VehicleUnitData.config for G2 Vehicle file in here, or succeed to read a G2 vehicle file ? I tried, but impossible to have a decent documentation to clearly explain it, like the driver cards. Can someone send me this structure ? (or a perfect documentation) Thanks in advance !

martinlowinski commented 4 years ago

@WallHackUp01 a colleague and I did some experiments and could read gen2 vehicle unit files to some degree. But we used Kaitai Struct for that, it's a declarative language that compiles to source files of many programming languages (JavaScript, C#, Python etc.).

We have used the tachograph-reader for quite some time and are very happy about it (thanks @jugglingcats!). So we tried to add support for gen2, as many of you tried as well, but haven't been successful. This is why we took a different direction and Kaitai Struct seems promising so far.

rimutaka commented 4 years ago

@jugglingcats Alfie, would you allow a fork of this repo as AGPL3 for v.2 development? I will be adding v.2 support and want to make sure it stays in FOSS.

jugglingcats commented 4 years ago

Hi @rimutaka, I think the AGPL is same as or perhaps more restrictive than the vanilla GPL. I imagine that most people using tachograph-reader are static linking it with or compiling it into their proprietary code of some form. Wouldn't making it AGPL discouraged almost all potential developers from using it?

Having said that you are perfectly at liberty to fork the code if you wish. Or we could think about changing the licence on this repo to LGPL which is more permissive...

rimutaka commented 4 years ago

@jugglingcats , let me clarify. I want to change the license for the fork only. However, any code created before that will be under the original license. That's complicated. What I'm asking is your permission to replace the original license in my v.2 fork with (A)GPL while retaining the reference to the original project + your copyright. Then any changes made to the fork will be automatically under (A)GPL without having to delineate which part of the v.2 code is under which license.

No pressure. I totally understand if you want to keep it as-is.

jugglingcats commented 4 years ago

Understood. I'm exploring whether you'd be happy to have a more permissive licence on this repo and make your changes here, in the interests of keeping everything in one place and avoid a fork. I'd be happy to make you a contributor so you have the ability to merge pull requests, etc. I'm not quite clear what your rationale is for wanting your changes to be AGPL, given it will prevent most people from using the code...

rimutaka commented 4 years ago

@jugglingcats ,

  1. a fork is because you said in PR #58 you are no longer involved. More than happy to keep it all in one repo, though.

  2. AGPL is to make sure this type of conversation does not happen ...

image

What I see there is they piggy-backed on the open source and are not willing to contribute. Maybe I'm reading too much into it.

davispuh commented 4 years ago

I'm strongly against any *GPL and always prefer the least restrictive license as possible and MIT is very good :). That is true that some people may wish to build their own stuff on top of it and don't contribute anything back but that's their own choice maintaining their own fork which over time might be worse than this because it's a lot of work to merge/rebase changes especially if forks have diverged a lot. Also this library is very tiny part of whole stack - download .ddd file from vehicle, parse/process .ddd file, save to DB, analytics/user friendly graphs/UI. I can say that if this library was AGPL the company I worked, we just wouldn't have used it at all and there wouldn't be any contributions from our side

28 commits
26 files changed, 2562 insertions(+), 1405 deletions(-)

The problem with AGPL is that this is a library to be integrated with wider software stack and that would require open sourcing those other parts which is core business of company. I would say this library alone isn't that useful without rest of functionality. For example for .ddd file downloading from different vehicle manufacturers there weren't any library at all so we built everything on own from scratch and all proprietary. The default choice for most companies is implement from scratch themselves and keep it proprietary which is kinda stupid but that's how it usually is. So 10 different companies waste time on implementing same things for no reason. If there is some library there is chance some companies will use same library and some might even contribute back thus everyone benefits :P So anyway current license does allow you to fork with more restrictive license but then you'll be pretty much on your own as I don't think if any company will even touch it.

mpi-wl commented 4 years ago

What I see there is they piggy-backed on the open source and are not willing to contribute. Maybe I'm reading too much into it.

@rimutaka

That's not true. Read my post carefully. My code is in JavaScript and has nothing to do with this project. I didn't use this project when I was creating my application. Additionally, I support 2nd generation ddd files (also cryptography), and this project doesn't have that.

Even though I don't share my (and only mine) library as open source, I helped other people with their problems (2nd generation structure).

Your opinion is unfair and harmful.

jugglingcats commented 4 years ago

Let's calm this discussion down folks... ;)

@mpi-wl, I read your post and understood your code was not related to tachograph-reader in any way, and am sure people appreciate your contribution to the discussion / problem solving. @rimutaka can be forgiven for not picking up on this detail.

@rimutaka's concern is valid especially if he is planning to contribute significant enhancements.

@davispuh articulated my concern with (A)GPL perfectly. This code is going to end up embedded in bigger system and GPL licence doesn't work for any commercial company wanting to use this code in their closed source product.

Perhaps we could compromise on LGPL which would be more permissive. The company I work for is generally comfortable with LGPL code. Open to suggestions / further thoughts.

rimutaka commented 4 years ago

Thanks everyone for your input. I tend to agree that no license change is needed.

jugglingcats commented 4 years ago

Thanks @rimutaka if you are happy to contribute your changes to this repo I will give you contributor access. Perhaps you could work on a v2.x branch until it becomes stable

jugglingcats commented 4 years ago

@rimutaka I have made you collaborator - welcome!

I am going to close this issue because it's getting rather long and covers a few different threads of conversation. I'll create a new pinned issue giving high level on DTCO 4.0 support.

rimutaka commented 4 years ago

Thanks Alfie. I'll start by posting new issues for .net5 and other changes I made in that PR to make sure I'm not breaking it for everyone else. I won't be merging anything in a hurry.

Phoryn commented 2 years ago

Hello

I want to summarize and consult all information which I need to write update to gen2.

-The structures of bytes for the segments is the same like in gen1. -In the segment is only one change, in new generation type can be also 0x02(content) and 0x03(sygnature). -On the one card might be also types of gen1 and gen2

On the beggining I thought that I can compare length of data "application_Identification" becouse it is exacly 10 byte in gen1(pic.2) and 15 byte in gen2(pic.3) . After select generation, choosen config. But I found information that in the card can be mixed information (gen1 and gen2 segments).

*"segment" mean one block of data like "application_Identification"

*@Dilemma725 wrote a program which to be able to check wchich generation card is. In his code he checking first and second byte of all card byte data. -if first 0x76 and second 0x06 - G1 -if first 0x00 and second 0x02 - G2 Why? First block of data is the same for both generation(pic1)

My question is: Did the best way do update this application to generation 2 is getting each segment one by one and checking his type?

pic1 pic2 pic3

tcepxxx commented 1 year ago

@mpi-wl Is there somehow possible to contact you ? I want to ask few questions about project you were making,

MRDTONY commented 1 year ago

Hi!

I have some questions regarding the public key used internal authentication process.

1.How can I retrieve the public key from the card? 2.What is the relationship between the public key stored in the card and the RSA encryption algorithm?

I would greatly appreciate it if anyone could provide insights, guidance, or resources on these topics. image