Number type record have been cut after finishing exportToCSV

[redshim]

Hello, I want to put this issue again. I used last committed version of 'mdf reader', you made it.

#################################

Current working script

################################# import sys import os.path from mdfreader import mdfreader qt = sys.argv[1] filepath = '%s' %qt print(filepath) filepathToCsv = filepath+'.csv'

yop = mdfreader.mdf(filepath) yop.keys() yop.exportToCSV(filepathToCsv) ##################################
--> this works fine but number type data format have bee cut.

There is the key which name is 'VS' in my mdf(version3). When I check my 'VS' data using the below script. 'vs = yop.getChannelData('VS') --> 4.500000000000000000e+01 4.500000000000000000e+01 4.500000000000000000e+01 4.400000000000000000e+01 ... 'vs = yop.getChannelData('VS').astype(int) --> 45 45 45 44 ... After finished convert the mdf, I can only see the result '0.0' values for VS column in the CSV result. I think getChannelData works properly, but ExportToCSV Function has little bug. Could you please check this?

Best Regards,

MDF 3.20 VDMS2^@^@^@^@^@^@^@@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@HDÐ^@^P^A^@^@þ7^K^@8^K^@¤^@20:01:201714:36:48PentaSystems^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@HKMC Namyang Research Center^@^@^@^@HKMC VDMS 2nd Construction^@^@^@^@^@^@HKMC VDMS Test Vehicle Data^@^@^@^@^@^@^@¥º<9b><81><9b>^T ^@ ^@HKMC VDMS Device Reference Time^@DG^\^@/>^@^@,^A^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^@^@^@CG^Z^@^@^@^@^@£^A^@^@F^A^@^@^@^@^R^@^P^@^@^@^@^@TX]^@{"channel":2,"message_id":550,"message_name":"NAVI_STD_STUB_E","comment":"[E] On Event"}^@CNä^@Å^B^@^@87>^B^@^@^@^@^@^@^@^@^@^@^@^@^@^@^A^@TimeChannel^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@^@^P^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@CC^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@Seconds^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^B^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ð?CNä^@R^E^@^@ä^C^@^@^@^@^@^@^@^@^@^@©^C^@^@^@^@STUB_MessageType^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@This is Stub Message for Map Based ADAS.^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@^@^C^@^M^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@TX;^@{"comment":"This is Stub Message for Map Based ADAS."}^@CCn^A^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^\@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^K^@^H^@^@^@^@^@^@^@^@^@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ð?Position^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@Segment^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^H@Stub^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^P@Profile Short^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^T@Profile Long^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^X@Meta Data^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^\@Invalid^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@CNä^@Æ^F^@^@p^F^@^@^@^@^@^@^@^@^@^@6^F^@^@^@^@STUB_Offset^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@This is Stub Offset for Map Based ADAS.^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@C^@^M^@^M^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@TX:^@{"comment":"This is Stub Offset for Map Based ADAS."}^@CCV^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@ÿ¿@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^K^@^A^@^@^@^@^@^@ÿ¿@Invalid^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@CNä^@º^H^@^@ì^G^@^@^@^@^@^@^@^@^@^@ª^G^@^@^@^@STUB_CyclicCounter^@^@^@^@^@^@^@^@^@^@^@^@^@^@This is Stub Cyclic Counter for Map Based ADAS.^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@P^@^B^@^M^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@TXB^@{"comment":"This is Stub Cyclic Counter for Map Based ADAS."}^@CCÎ^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^H@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^K^@^D^@^@^@^@^@^@^@^@^@Cyclic Counter^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ð?Cyclic Counter^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@Cyclic Counter^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^H@Cyclic Counter^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@CNä^@^G^T^@^@Ù ^@^@^@^@^@^@^@^@^@^@<9e> ^@^@^@^@STUB_PathIndex^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@This is Stub Message for Map Based ADAS.^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@R^@^F^@^M^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@TX;^@{"comment":"This is Stub Message for Map Based ADAS."}^@CC. ^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@<80>O@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^K^@@^@^@^@^@^@^@^@^@^@first path in the Horizon^@^@^@^@^@^@^@^@^@^@^@^@^@ð?Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^H@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^P@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^T@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^X@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^\@Reserved^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ @General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@"@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@$@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@&@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@(@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@,@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@.@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@0@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@1@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@2@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@3@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@4@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@5@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@6@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@7@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@8@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@9@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@:@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@;@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@=@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@>@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@?@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@@@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>@@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@A@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>A@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@B@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>B@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@C@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>C@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@D@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>D@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@E@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>E@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@F@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>F@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@G@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>G@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@H@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>H@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@I@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>I@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@J@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>J@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@K@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>K@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@L@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>L@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@M@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>M@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@N@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>N@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@O@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<80>O@General Path Index^@^@^@^@^@^@^@^@^@^@^@^@^@^@CNä^@[^_^@^@-^U^@^@^@^@^@^@^@^@^@^@ë^T^@^@^@^@STUB_SubPathIndex^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@This is Stub Sub Path Index for Map Based ADAS.^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@X^@^F^@^M^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@TXB^@{"comment":"This is Stub Sub Path Index for Map Based ADAS."}^@CC.

[ratal]

Thanks for explaining more in detail. Could you send your file by mail instead on this platform ? Also, what version of python are you using ? 2.7, 3.x ? (csv module is coming from bare python) There is maybe an issue with the way your csv reader understand the generated file. With which tool are you reading it ? Line 617, the csv module is set to export the CSV in excel dialect, you could also try to change this with excel_tab or unix_dialect (depending also your platform). You could also play with the delimeter... Have a look at the documentation: https://docs.python.org/2/library

[redshim]

Could you tell me you personal email address which I can send you MDF file. I'm using python 2.7. The problem is that I can only see the value 0.0 after convert to csv format. But I can see right value using getChannelData('VS').int I will email you with attached. Thank you

[ratal]

my email is aymeric.rateau@gmail.com

[ratal]

I tried your file and I could not reproduce your issue (using python 2.7 on a debian, reading the csv with libreoffice calc --> what soft+OS do you use to open your csv ?). I could notice there might be an issue with the units encoding (what encoding did you use to record your file ? (maybe corean ?) issues starts with CF_Ems_ModeledAmbTemp, what is unit ?) but the data is matching what I see in ipython from your file after reading it. First, I tried on low spec PC and crashed it, not enough memory. Your csv file is 743Mo for a mdf of 87Mo. I let you imagine this is not really practical and you'd better use another format like hdf5 or mat (also hdf), or even remain in python by converting to pandas for your analysis.

[redshim]

Hi,

I would like to show you both my result and environment below. Could you try to resolve two different ways. 1st way - use getChannelData('VS') module 2nd way - use exportToCSV module There would be a different result.

I will also try to use hdf5 format as well, thank you for your support.

Mu current OS is Red Hat 6.5 [6546788@rnode02 script]$ grep . /etc/*-release /etc/lsb-release:LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch /etc/redhat-release:Red Hat Enterprise Linux Server release 6.5 (Santiago) /etc/system-release:Red Hat Enterprise Linux Server release 6.5 (Santiago)

Python version Python 2.7.12 |Anaconda 2.4.1 (64-bit)| (default, Jul 2 2016, 17:42:40) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://anaconda.org

mdfreader version [6546788@rnode02 bin]$ ./pip list | grep mdf DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning. mdfreader (0.2.4)

Test Result [6546788@rnode02 bin]$ ipython Python 2.7.12 |Anaconda 2.4.1 (64-bit)| (default, Jul 2 2016, 17:42:40) Type "copyright", "credits" or "license" for more information.

IPython 4.0.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]: import pandas as pd In [2]: import numpy as np In [3]: from mdfreader import mdfreader In [4]: filePath = "/home/6546788/MDF/FLEET/A2/RJ/170116/RJ_N16-12-363_BM-15C-0024_228_2_20170116094355_CAN_20170116093930_89612_mdf.dat" In [5]: fileExportPath = "/home/6546788/MDF/ImgAna/" In [6]: fileName = "proof_mdf.csv" In [7]: yop=mdfreader.mdf(filePath) /home/6546788/MDF/FLEET/A2/RJ/170116/RJ_N16-12-363_BM-15C-0024_228_2_20170116094355_CAN_20170116093930_89612_mdf.dat In [8]: vs = yop.getChannelData('VS') In [9]: len(vs) Out[9]: 271525 In [10]: vs[20000] Out[10]: 68 In [11]: vs[20001] Out[11]: 69 In [12]: vs[20002] Out[12]: 69 In [13]: vs[20003] Out[13]: 69

In [14]: yop.exportToCSV(fileExportPath+fileName)

Malfunction Step After I creating CSV file using 'exportToCSV module, the record was different which I can show on Ipython console using "yop.getChannelData('VS')

'VS' channel is locating at the 370 step of the CSV.

[6546788@rnode02 ImgAna]$ sed -e '1,20000d;20005,$d' proof_mdf.csv | awk -F"[,]" '$370 > 0 { print $370 } ' 4.0 4.0 4.0 4.0

---> 4.0 should be a 68 km/h . The value result is different.

CSV Header Information CF_Lvr_ShfErrInf,CF_Lvr_NFnStat,WHL_SPD_FL,CF_Fpcm_LPPumpDiscShort,CF_Ems_DesCurGr,WHL_DIR_FL,ESS_STAT,WHL_SPD_Checksum_LSB,WHL_SPD_FR,Alive_Counter,CF_Mdps_WLmp,CF_Tcu_SiCluster,ACC_ObjDist,TCU_TYPE,AVH_ALARM,MainMode_ACC,ACCFailInfo,WHL_DIR_FR,CF_Tcu_TqGrdLim,CR_Datc_SelfDiagCode,CF_Lkas_LdwsSysState,CF_Clu_Odometer,PID_0Ch,AliveCounterACC,_4WD_CLU_LIM,CF_Fatc_CtrInf,CF_Gway_PASDisplayFCTR,FUP_MES,CF_Aaf_OpenRqSysAct,CF_Gway_PassiveAccessLock,CF_Lvr_UlkButStat,CF_Lvr_BkeAct,_4WD_LIM_MODE,SWI_GS,DTP,FUEL_CUT_TCU,NR_DTC,ENG_STAT,CF_Ems_UpTarGr,CF_Gway_PASDisplayRCTR,FA_PV_CAN,CF_Gway_HeadLampLow,CF_Datc_ModDisp,PSB_LH_FAIL,CYL_PRES_STAT,DBC_CTL,CF_Ems_ModeledAmbTemp,TCU_OBD,WHL_PUL_RL,ACC_REQ,CRUISE_LAMP_M,CR_Tcu_IsgTgtRPM,CF_Gway_LightSwState,CRUISE_LAMP_S,WHL_PUL_RR,CF_CGW_ISGSW,BRAKE_ACT,CF_Gway_IGNSw,TEMP_FUEL,FTL,YAW_RATE_STAT,UNIT,HAC_DEF,CF_Tcu_SmartShift_Inf,CF_Tcu_NCStat,CF_Tcu_ITPhase,CR_Datc_PsTempDispF,CF_Tmu_VehImmo,CR_Datc_PsTempDispC,CF_Aaf_WrnLamp,CF_Fatc_Compload,CF_TCU_WarnImg,BAT_SNSR_Error,CF_Gway_WiperLowSw,SAS_Speed,TCS_PAS,VLSLB1S1,VLSLB1S2,CF_Clu_LowfuelWarn,VS_TCU_DECIMAL,FCO,CF_Tcu_Alive,YAW_RATE,PID_11h,DBC_PAS,CF_PasBkl_Stat,PID_05h,CTR_CDN_OBD,TauGapSet,CR_Fatc_TqAcnOut,ENG_CHR,CF_Ems_DecelReq,PUT,CR_Datc_RearDrTempDispC,CR_Datc_RearDrTempDispF,CF_Gway_HeadLampHigh,TQI_ACC,CF_DriBkl_FltStat,PID_23h,ACC_ObjRelSpd,BAT_SNSR_I,CF_TCU_WarnMsg,BAT_SNSR_V,DBC_DEF,BLECFail,CF_FATC_Iden,CF_Gway_TrunkTgSw,CF_Datc_TempDispUnit,DrivingModeState,_4WD_LIM_REQ,CF_Tmu_ReqRepCnd,CR_Wcs_ClassStat,CF_Fpcm_PreSnrShort,CF_Tcu_ChkSum3,ABS_DIAG,CF_Tcu_ChkSum1,WHL_SPD_Checksum_MSB,CF_Tcu_StRelStat,POS_FR_W_LAMP,PUC_STAT,TQI_MSR,CR_Photosensor_LH,PID_06h,CF_Clu_Vanz,CF_Ems_DownTarGr,BLA_CTL,CF_Lvr_AC,F_N_ENG,CF_Datc_AutoDisp,HU_PhoneActivity,QECACC,PID_09h,CF_Lkas_LdwsLHWarning,CUR_GR,ESP12_AliveCounter,TQI_MAX,CF_Gway_WiperHighSw,TQ_STND,MUL_CODE,CF_Gway_PASDisplayRRH,HU_TimeSetting,TPS,CF_Ems_FCopen,CF_Gway_WiperParkPosition,ESP12_Checksum,DBC_F_LAMP,CF_Aaf_ActFlapStatus,CF_Gway_RainSnsState,ObjValid,INH_DC_OBD,MAF_FAC_ALTI_MMV,TCU_RPM_Cluster,CF_Gway_PASDisplayRLH,STATE_DC_OBD,CF_Ems_PumpTPres,BAT_SNSR_Invalid,CF_Lvr_GearInf,CF_Fatc_MsgCnt,CF_Clu_DrivinglampStat,CF_Tmu_TempMd,DATC_SyncDisp,CR_Fpcm_LPActPre,CLU12_STS,CF_Acu_FltStat,R_PAcnC,master_0,CF_SWL_Ind,TCS_LAMP,CF_Esc_LimoInfo,CR_Ems_BstPre,ESP_CTL,EBD_DEF,CF_Gway_RRDrSw,CF_Clu_ParityBit1,LAT_ACCEL,CF_TCU_WarnSnd,CF_Tcu_BkeOnReq,ACCMode,CF_PasBkl_FltStat,C_STS_CheckSum,CF_Ems_BrkReq,CF_Datc_FrontBlwDisp_OSD,C_STS_AliveCounter,CF_Gway_TurnSigRh,DATC_AutoDefogSysOff_Disp,CF_Datc_AutoDefogBlink,CF_Tcu_SbwPInfo,CF_Tcu_TarGr_SS,CF_Datc_FrDefLed,CF_Aaf_ErrStatus,master,CF_Datc_ChgReqDisp,WHL_DIR_RR,CF_Gway_PASDisplayFLH,CR_Datc_DrTempDispC,CR_Datc_DrTempDispF,CF_Tcu_SptRdy,TQI_B,AliveCounterTCS,GEAR_TYPE,WHL_SPD_RR,INH_FUEL_CUT,SCC12_STS,TI_B2,TI_B1,CF_Fpcm_PreSnrHi,ROL_CNT_ESP,CR_Ems_EngOilTemp,CF_Lvr_ShtLkStat,CF_Clu_AltLStatus,CF_Tmu_AirconCtr,CF_Gway_DrvDrSw,WHL_SPD_RL,PreFill,TREAD_W_LAMP,LDWS11_STS,CF_Ems_EtcLimpMod,CF_Tcu_ChkSum,CF_Tcu_InhCda,CF_Ems_IdlStpFCO,CF_Aaf_MilOnReq,CF_Clu_DrivingModeSwi,F_TCU,ABS_W_LAMP,LONG_ACCEL_STAT,TCS_DEF,OBD_FRF_ACK,IM_AUTEHN,F_SUB_TQI,SOAK_TIME,CR_Fatc_OutTemp,PID_03h,CF_Datc_RearOffDisp,CF_Lvr_PosCpl,TQI_138,CF_Fatc_DefSw,CF_Tmu_VehSld,SwitchOffCondExt,CAN_VERS,CF_Fpcm_LPCtrCirFlt,PID_0Dh,DATC_SmartVentDisp,CF_Tmu_AliveCnt1,PBRAKE_ACT,HAC_PAS,CF_Lkas_LdwsRHWarning,CF_Tcu_ShfPatt,ETC_PWM,AliveCounter_TCS1,TQI,CF_Gway_AstSeatBeltSw,CF_Hoodsw_memory,STATUS_TPMS,CF_VSM_BeltCmd,N_TC_RAW,REQ_EPB_ACT,TEMP_AT,TEMP_ENG,CF_TCU_NCC_set_chk,CLU_ACK,TQI_MIN,WHL_DIR_RL,aReqMin,BrakeLight,MsgCount,IM_STAT,PRE_FUEL_CUT_IN,PV_AV_CAN,MSR_C_REQ,ENG_VOL,CF_VSM_Stat,CF_Clu_CruiseSwState,ESP_DEF,CF_Datc_RearBlwDisp,CR_Fatc_OutTempSns,POS_RL_W_LAMP,CF_Ems_AclAct,PID_07h,CF_Datc_OpSts,CR_Wcs_ErrStat,RLY_AC,PRESSURE_RL,CF_Ems_AliveCounterEMS9,PRESSURE_RR,TQI_TCU_INC,CF_Tcu_IsgInhib,ABS_DEF,VB_OFF_ACT,C_SunRoofOpenState,CF_Ems_IsgStat2,CR_VSM_ChkSum,CF_Acu_NumOfFlt,TCS_CTL,ACC_ObjStatus,MODE_4WD,BAT_SNSR_State,CF_Gway_ParkBrakeSw,N_TGT_LUP,CF_Ems_ActEcoAct,CF_Ems_ChecksumEMS9,POS_FL_W_LAMP,CF_Clu_ActiveEcoSW,CF_Lvr_PRelStat,CF_Gway_PASDisplayFRH,_4WD_TQC_LIM,TCS_MFRN,AVH_STAT,ACK_ES,CHECKSUM,CF_Tcu_PRelReq,CF_Tcu_Fstatic,HU_VRActivity,CF_Datc_PSModDisp,CF_Datc_RearModDisp,CLU13_STS,CR_VSM_Alive,TCS_REQ,CF_Tcu_InhVis,ECGPOvrd,TCS_OFF_LAMP,CPPWM,SAS_Stat,N_INC_TCU,WHL_PUL_FL,G_SEL_DISP,WHL_PUL_FR,POS_RR_W_LAMP,MODE_TM,CR_Datc_OutTempF,PSB_RH_FAIL,CR_Datc_OutTempC,C_MirOutTempSns,LKAS11_STS,CL,ISG_INHIBIT_LAMP_STAT,N,CF_Tcu_TarGr_Eng,BAT_LAMP_STAT,HU_Status,PID_08h,CheckSum,MAF_ADD_AD,WHL_SPD_AliveCounter_LSB,TQI_TCS,TQI_TCU,LAT_ACCEL_STAT,CF_Ems_SldPosAct,FUP_SP,CONF_MIL_FMY,AVH_I_LAMP,CF_Gway_RLDrSw,CF_Lvr_CS,CF_Gway_HazardSw,AliveCounter,PID_0Bh,CF_Tmu_DefrostCtr,_4WD_OPEN,ETL_TCU,CYL_PRES,CF_Gway_HoodSw,CF_Lvr_NWStatus,VLSLB2S2,VLSLB2S1,CF_Gway_TurnSigLh,VB,RATIO_TQI_BAS_MAX_STND,ECD_ACT,CF_Fatc_EngStartReq,CF_Fpcm_LPPumpOverCur,VS,TQI_ACOR,IDLE_UP_TCU,CF_Fpcm_LP_System_Error,CF_Datc_Option2,CF_Datc_Option1,CF_DATC_TempDisp_Sts,CF_Datc_Type,CF_Tcu_ActEcoRdy,CF_Ems_AAFOpenReq,aReqMax,SAS11_STS,CF_Esc_BrkCtl,EPB_DBF_STAT,CF_Gway_WiperAutoSw,ACK_TCS,TQFR,CF_Gway_RheostatLevel,CF_Gway_DrvSeatBeltSw,N_TC,AVH_CLU,CF_Fatc_AcnCltEnRq,LDM_STAT,HAC_CTL,EPB_DBF_REQ,ACC_ObjLatPos,ESC_Off_Step,LONG_ACCEL,DBC_W_LAMP,Checksum,TCU_RPM_Flag,ACC_ACT,CF_Datc_PSModDisp_OSD,CF_Ems_OilChg,EPB_STATUS,EBD_W_LAMP,CF_Ems_EngRunNorm,SPK_TIME_CUR,SLOPE_TCU,CONF_TCU,SAS_Angle,MODE_ENG,CF_Ems_DnShftReq,CF_Fatc_IsgStopReq,CF_Aaf_ModeStatus,CF_Ods_IDRcv,CF_Datc_FrontBlwDisp,CF_Gway_DetentOut,PID_04h,SLD_VS,ABS_ACT,CF_Acu_Dtc,OD_OFF_REQ,AVH_LAMP,CF_Clu_AliveCnt1,TCS_GSC,TQI_SLW_TCS,LONG_ACCEL_DIAG,CF_Gway_RKECmd,CR_Datc_DRSeatWheel_USM_State,VS_TCU,CF_Datc_RearAutoDisp,CF_Tcu_Alive1,CF_Tcu_Alive3,CF_Gway_AstDrSw,MDPS11_STS,CF_Gway_DefoggerRly,CF_Fatc_EcvFlt,CF_Clu_CruiseSwMain,CF_Tcu_DriWarn2,CF_Gway_InhibitRMT,CF_Tcu_DriWarn1,WHL_SPD_AliveCounter_MSB,L_MIL,REQ_EPB_STAT,BAT_SNSR_Temp,CheckSum_TCS3,CheckSum_TCS1,CF_Tcu_GSel_BlinkReq,WHL_PUL_AliveCounter,AMP_CAN,CF_Tcu_DrivingModeDisp,DATC_SmartVentOnOffStatus,CF_Tcu_DrivingModeReq,BAT_Alt_FR_Duty,CF_Aaf_PStatus,CF_Ems_OPSFail,CF_Datc_ModDisp_OSD,CF_Ems_SldAct,CF_Fatc_BlwrOn,R_NEngIdlTgC,CF_Lvr_PosInf,CTR_IG_CYC_OBD,CF_Gway_WiperIntT,ACC_EQUIP,CR_Photosensor_RH,SOAK_TIME_ERROR,CF_Clu_VanzDecimal,VSetDis,StopReq,CF_Datc_AcDisp,R_TqAcnApvC,CF_Clu_SPEED_UNIT,LAT_ACCEL_DIAG,CF_Fpcm_PreSnrSigErr,TQI_TARGET,TakeOverReq,CF_Lvr_WarnSnd,BAT_SOH,SWI_CC,CF_Mdps_ALTRequest,BAT_SOC,BAT_SOF,CF_Ems_IsgStat,CF_Fatc_BlwrMax,CF_Tmu_TempSet,CF_Lvr_PNStat,SMART_SHIFT_MODE,CF_DriBkl_Stat,IntAirTemp,CF_Fatc_ChkSum,PRESSURE_FL,CF_Fatc_AcnRqSwi,YAW_RATE_DIAG,TQ_COR_STAT,PRESSURE_FR,ESP_PAS,CF_CdaStat,CF_Gway_WiperIntSw,SWI_IGK,CF_Ems_HPresStat,TPMS_W_LAMP,CF_Clu_AliveCnt2,CYL_PRESS_DIAG,CF_Gway_HLpHighSw,CF_Ems_IsgBuz,ECFail,SPK_RTD_TCU,CF_Acu_ExtOfSab,CF_Fpcm_PreSnrDisc,CF_Fatc_PtcRlyStat,WHL_PUL_Chksum,CF_Tcu_TarGr

[ratal]

Comparatively, I use debian stretch (9.0), pytohn 2.7.13 and ipython 5.1.0 Good point, I can find the same data as you were doing: vs = yop['VS']['data'] vs[20000] --> 68 And also the issue inside the CSV file. Actually, CSV export itself is not the problem, it is resampling (done before exporting to CSV). yop.resample() yop.plot({'VS']) you will see it is almost constant at 4.0. New computed master channel is wrong, I am investigating.

[ratal]

Apparently resample() is not an issue, rather reading of data blocks. Some time channels are max in 10E304. Investigating

[ratal]

I implemented hidden bytes handling (unused bytes in records). You can have a look at last commit. There are still other mistakes in the reading but resampling is luckily going ok so csv export should also improve for most of the data. Continuing to improve.

[ratal]

Latest commit should have brought robustness on mdf3 against hidden bytes and not aligned bytes and therefore solved your issues.

[redshim]

Okay, Thanks. I will run and check it again with latest commit version.