Initialize a Project in DICE!

[Piluri]

Hello, I was trying to use co2dice and this Problem popped up:

C:\Users\G00550\Documents\CO2MPAS\co2mpas\co2mpas_AIO-v1.5.5\Demos>co2dice project init -i Input\co2mpas_demo-1.xlsx -o Output\20170628_172801-co2mpas_demo-1.xlsx --report 17:54:08 : INFO:InitCmd:Project 'IP-10-AAA-2017-1002' derived from 'inp' file: C:\Users\G00550\Documents\CO2MPAS\co2mpas\co2mpas_AIO-v1.5.5\Demos\Input\co2mpas_demo-1.xlsx 17:54:08 : INFO:traitlets:Creating project 'IP-10-AAA-2017-1002'... 17:54:08 :ERROR:co2dice:Project 'IP-10-AAA-2017-1002' already exists!

C:\Users\G00550\Documents\CO2MPAS\co2mpas\co2mpas_AIO-v1.5.5\Demos>co2dice project ls 17:54:44 : INFO:LsCmd:Listing all projects...

• IP-10-AAA-2017-1002: wltp_iof

C:\Users\G00550\Documents\CO2MPAS\co2mpas\co2mpas_AIO-v1.5.5\Demos>co2dice project tsend 17:59:40 :ERROR:co2dice:IP-10-AAA-2017-1002: Can't trigger event do_sendmail from state wltp_iof!

I cant send it to the email.

Any recomandations?

Thank you.

[ankostis]

Dear Piluri, the following line in the logs means that project init command failed:

17:54:08 :ERROR:co2dice:Project 'IP-10-AAA-2017-1002' already exists!

It's ok, because you have already created this project, and both input & out files have been imported into it, since it is in the wltp_i(nput)o(utput) state, as seen in the next ls command you pasted.

What you should do next is the following:

1. Check the imported files in the project are what they should.
   Assuming your are in your "HOME" folder CO2MPAS, type these commands to create copies of the imported files in a new testdir folder and launch Excel to inspect them (we do not want to inspect and modify the originals):

   mkdir testdir
   cp   .co2dice/repo/inp/*  .co2dice/repo/out/*  testdir
   start testdit/*

2. If they are ok, proceed the project to report state:

   co2dice project report

3. Send the dice-email (assuming your configurations are properly setup with your email-server):

   co2dice project tsend 

Tip: if a command fails, relaunch it with the -v flag and paste the results here.

[ankostis]

Correction: In step 1, wherever you see the asterisk(*), you have to press [Tab] key, to complete the name of the file that is located in this folder. Otherwise you have to do the "copy" and "open" action with file-explorer.

[Piluri]

Thank you for the Response,

I checked the Input and Output files and they are ok. No abnormalities! I still can’t initialize a Project.

This is what I did:

1. I loaded the Input file on Co2mpas and l let the simulation run. After 3-4 min the results were there. The Simulation runed with no errors.

co2mpas_version: 1.5.7.b3 co2mpas_rel_date: 2017-05-14 08:16:03

co2mpas_path: C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas python_version: 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)]

2. I opened DICE with Launch Console … and I loaded the Input and Output file from Simulation in DICE to initialize a Project:

C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\CO2MPAS>co2dice project init -i C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\ Input\co2mpas_input _Test1_R401_0016011_Test1.xlsx -o C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\ Output\20170629_120810-co2mpas_input xlsx --report –v

This is the answer:

12:24:24 : INFO:InitCmd:Project 'IP-XX-YYY-2017-XXXX' derived from 'inp' file: C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\ Input\co2mpas_in.xlsx 12:24:24 : INFO:traitlets:Creating project 'IP-XX-YYY-2017-XXXX'... 12:24:25 : INFO:traitlets:...failed opening repo 'C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\CO2MPAS.co2dice\repo', initializing a new repo 'C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\CO2MPAS\.co2dice\repo' instead... gpg2 --status-fd 2 --no-tty --debug ipc --use-agent --allow-weak-digest-algos --armor --keyid-format long --version gpg: Hinweis: Keine voreingestellte Optionendatei 'C:/Users/G00550/Documents/co2mpas_AIO-v1.5.5/Apps/GnuPG/home/gpg.conf' vorhanden gpg2 --status-fd 2 --no-tty --debug ipc --use-agent --allow-weak-digest-algos --armor --keyid-format long --list-secret-keys --fixed-list-mode --fingerprint --with-colons gpg: Hinweis: Keine voreingestellte Optionendatei 'C:/Users/G00550/Documents/co2mpas_AIO-v1.5.5/Apps/GnuPG/home/gpg.conf' vorhanden

sec::1024:1:B124C999CBBB52FF:1485434541:1500986541::::::::: fpr:::::::::5464E04EE547D1FEDCAC4342B124C999CBBB52FF: uid:::::::F3C8DBC15DD5EB340D03F3FD5F0F79753115FACD::CO2MPAS Test sampling@co2mpas.jrc.ec.europa.eu: 12:24:25 :WARNI:traitlets:Overwritten 8 out of 8 missmatched value in git-settings('C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\CO2MPAS.co2dice\repo\ : core.filemode: [true, false, ] core.ignorecase: [true, false, ] user.email: [, co2mpas@yahoo.com, ] user.name: [, co2mpas, ] gc.auto: [, 0, ] alias.lg: [, 'log --graph --abbrev-commit --decorate --date=relative --format=format:''%C(bold blue)%h%C(reset) - %C(bold green)(%ar)%C(reset) %C(white)%s%C(reset) %C(dim white)- %an%C(reset)%C(bold yellow)%d%C(reset)'' --all', ] gpg.program: [, /cygdrive/c/Users/G00550/Documents/co2mpas_AIO-v1.5.5/Apps/GnuPG/pub/gpg2.EXE,

] user.signingkey: [, 5464E04EE547D1FEDCAC4342B124C999CBBB52FF, ] 12:24:25 : INFO:transitions.core:IP-XX-YYY-2017-XXXX: Exited state BORN 12:24:25 : INFO:transitions.core:IP-XX-YYY-2017-XXXX: Entered state empty 12:24:25 : INFO:Project:Committing Project(IP-XX-YYY-2017-XXXX: empty): new project 12:24:25 : INFO:transitions.core:IP-XX-YYY-2017-XXXX: Exited state empty 12:24:25 : INFO:Project:Importing files: PFiles(inp=['C:\\Users\\G00550\\Documents\\co2mpas_AIO-v1.5.5\\ \Input\\co2mpas_input 1_0016011_Test1.xlsx'], out=['C:\\Users\\G00550\\Documents\\co2mpas_AIO-v1.5.5\\ \Output\\20170629_120810-co2mpas_input_JD11 t1.xlsx'], other=())... 12:24:36 : INFO:transitions.core:IP-XX-YYY-2017-XXXX: Entered state wltp_iof 12:24:36 : INFO:Project:Committing Project(IP-XX-YYY-2017-XXXX: wltp_iof): add 2 files 12:24:36 : INFO:transitions.core:IP-XX-YYY-2017-XXXX: Exited state wltp_iof 12:24:36 : INFO:Project:Preparing report: {}... 12:24:47 : INFO:transitions.core:IP-XX-YYY-2017-XXXX: Entered state tagged 12:24:47 : INFO:Project:Committing Project(IP-XX-YYY-2017-XXXX: tagged): drep 2 files 12:24:47 : INFO:Project:Tagging Project(IP-XX-YYY-2017-XXXX: tagged): dices/IP-XX-YYY-2017-XXXX/0 12:24:47 :WARNI:Project:New status('tagged') failed, REVERTING to prev-status('wltp_iof'). 12:24:47 :ERROR:co2dice:Launch failed due to: Cmd('git') failed due to: exit code(128) cmdline: git tag --local-user=None -m - {v: 1.0.0, a: drep 2 files, p: IP-XX-YYY-2017-XXXX, s: tagged} - file: co2mpas_input iokind: inp report: {report_type: input_report, vehicle_family_id: IP-XX-YYY-2017-XXXX} - file: 20170629_120810-co2mpas_input.xlsx iokind: out report: 0.vehicle_family_id: [IP-XX-YYY-2017-XXXX, IP-XX-YYY-2017-XXXX] 1.CO2MPAS_version: [1.5.7.b3, 1.5.7.b3] 2.report_type: [dice_report, dice_report] 3.datetime: ['2017/06/29-12:08:18', '2017/06/29-12:08:18'] 4.TA_mode: ['True', 'True'] 12:24:47.283843 run-command.c:336 trace: run_command: '/cygdrive/c/Users/G00550/Documents/co2mpas_AIO-v1.5.5/Apps/GnuPG/pub/gpg2.EXE' '-bsau' 'None' gpg: skipped "None": No secret key gpg: signing failed: No secret key error: gpg failed to sign the data error: unable to sign the tag' Traceback (most recent call last): File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\dice.py", line 173, in main baseapp.consume_cmd(cmd.start()) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\baseapp.py", line 1110, in start return self.subapp.start() File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\baseapp.py", line 1110, in start return self.subapp.start() File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\baseapp.py", line 1099, in start res = self.run(*self.extra_args) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\project.py", line 1725, in run return self.append_and_report(pfiles) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\project.py", line 1658, in append_a ok = proj.do_report() File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\transitions\core.py", line 282, in trigger return self.machine._process(f) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\transitions\core.py", line 831, in _process return trigger() File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\transitions\core.py", line 312, in _trigger if t.execute(event_data): File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\transitions\core.py", line 201, in execute machine._callback(func, event_data) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\transitions\core.py", line 812, in _callback func(event_data) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\co2mpas\sampling\project.py", line 539, in _cb_commi sign=True, local_user=git_auth.master_key) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\git\repo\base.py", line 338, in create_tag return TagReference.create(self, path, ref, message, force, **kwargs) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\git\refs\tag.py", line 80, in create repo.git.tag(*args, **kwargs) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\git\cmd.py", line 424, in return lambda *args, **kwargs: self._call_process(name, *args, **kwargs) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\git\cmd.py", line 873, in _call_process return self.execute(call, **_kwargs) File "C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\Apps\WinPython\python-3.5.2.amd64\lib\site-packages\git\cmd.py", line 687, in execute raise GitCommandError(command, status, stderr_value, stdout_value) git.exc.GitCommandError: Cmd('git') failed due to: exit code(128) cmdline: git tag --local-user=None -m - {v: 1.0.0, a: drep 2 files, p: IP-XX-YYY-2017-XXXX, s: tagged} - file: co2mpas_input_JD11_R405_0016219_Test1_R401_0016011_Test1.xlsx iokind: inp report: {report_type: input_report, vehicle_family_id: IP-XX-YYY-2017-XXXX} - file: 20170629_120810-co2mpas_input.xlsx iokind: out report: 0.vehicle_family_id: [IP-XX-YYY-2017-XXXX, IP-XX-YYY-2017-XXXX] 1.CO2MPAS_version: [1.5.7.b3, 1.5.7.b3] 2.report_type: [dice_report, dice_report] 3.datetime: ['2017/06/29-12:08:18', '2017/06/29-12:08:18'] 4.TA_mode: ['True', 'True'] 3. C:\Users\G00550\Documents\co2mpas_AIO-v1.5.5\CO2MPAS>co2dice project ls 12:44:21 : INFO:LsCmd:Listing all projects... * IP-XX-YYY-2017-XXXX: wltp_iof the Project is still not TAGGED.

[Piluri]

Note: 'C:/Users/G00550/Documents/co2mpas_AIO-v1.5.5/Apps/GnuPG/home/gpg.conf' - gpg.conf - is missing tells the DICE. Thank you!

[dimitriskomnos]

Dear Mr Piluri,

In your message log, we see this:

12:24:47.283843 run-command.c:336 trace: run_command: '/cygdrive/c/Users/G00550/Documents/co2mpas_AIO-v1.5.5/Apps/GnuPG/pub/gpg2.EXE' '-bsau' 'None'
gpg: skipped "None": No secret key
gpg: signing failed: No secret key
error: gpg failed to sign the data
error: unable to sign the tag'

This message pops up because you do not have added the master_key in your configuration file.

Please, add the following in your _co2diceconfig.py file:

## The key-id (or recipient) of a *secret* PGP key to use for various crytpo
#  operations.
#  For CO2MPAS testing-period (till ~July 2017) leave it as it is.
c.GpgSpec.master_key = 'CBBB52FF'

(This key is for the purposes of testing. )

In order to produce the report you need the master_key, but not before. Since you have already appended the files input and output , and you received the message:

IP-XX-YYY-2017-XXXX: wltp_iof

Continue with the workflow, starting with:

co2dice project report

Let us know if you still face problems with the DICE.

Regards, Dimitris

[Piluri]

Hello Dimitris,

Thank you for your help, you are right the master key was missing. Now after adding it every thing was fine.

---

Now I have a problem with:

$ co2dice project tparse. Copy from the above log message (from -----BEGIN PGP SIGNED MESSAGE----- to the end) and paste it in the console by pressing [Shift+INSERT], and send the end-stream character after a new-line, 4 that is to say: press [Enter], [Ctrl+Z], [Enter], in a row.

This is what I receive:

-----END PGP SIGNATURE----- ^Z 15:13:00 :ERROR:co2dice:Cannot verify timestamp-response's signature due to: signature bad

[dimitriskomnos]

Dear Mr Piluri,

If I am correct, we received the time stamp response, too, since you, correctly, added us in c.TstampSender.tstamp_recipients of your configuration file.

We managed to parse the signature and get the SAMPLE/NO SAMPLE decision. If you sent the report through the yahoo public e-mail provider , please try the following:

1. Open the e-mail that you received from the time stamp server. Find the option to see the raw message. Maybe the picture bellow helps:

2. Copy the message from there and paste it, as you did before.

This procedure should work. Let us know if everything is fine and if you managed to get the decision.

Regards, Dimitris.

[Piluri]

Thank you!!! Every Thing works fine now.

Thank you for your Support.

[Piluri]

Hello Everybody, The last point of the DICE instructions is: Co2dice project export ….. What is included in this .zip file? I have uploaded my export file as example and I was asking myself why there are not any Input or output files. Is it ok so?

CO2MPAS_projects-20170630-175833.zip

[ankostis]

The files are indeed included in the zip archive.
The zip archive is actually a "bare" git repository, so the files are all stored "encoded" under the repo/.objects dirs. You can "checkout" and view them using git commands.

In practice, exploring the contents of the archive is not required by the legislation, and they are send to the TAA for tracing the procedure in case of future problems. But for completeness, I present a console session with the git commands that checkout the contents (assuming your archive is located in your CO2MPAS home dir):

> mkdir project_testdir
> cd project_testdir
> unzip CO2MPAS_projects-20170630-175833.zip
Archive:  /d/Downloads/CO2MPAS_projects-20170630-175833.zip
   creating: repo/
        ...: ...
  inflating: repo/refs/tags/dices/IP-10-AAA-2017-1002/0

> mv repo .git
> git config core.bare false
> git branch
  projects/IP-10-AAA-2017-1002

> git checkout projects/IP-10-AAA-2017-1002
Switched to branch 'projects/IP-10-AAA-2017-1002'
> ls inp out
inp:
co2mpas_demo-1.xlsx

out:
20170630_173257-co2mpas_demo-1.xlsx

> start inp/co2mpas_demo-1.xlsx
> start out/20170630_173257-co2mpas_demo-1.xlsx

The last 2 commands open the co2mpas input/output files in Excel.

[Piluri]

I was looking in .zip file the DICE decision (dice hexnum and the random number in percent). Is it possible to export this decision from DICE? Not only with copy-paste.

[ankostis]

I'm not sure I understand your question?

But for sure you can view the "full" decision for projects with --verbose, for instance:

> co2dice ls  -v
IP-10-AAA-2017-1009:
  msg.s: nosample
  msg.a: diced as OK
  last_dice: 'dices/IP-10-AAA-2017-1009/0: 2bc0977ba7f2ead4e9abfc4e0ab800984cf6dc6b'
  last_commit: a6c59130172b5b04ce61bb023361fe6c004ede05
  last_tree: b41487f0feba935e3d00908b4f3c3b935f6bf59e
  files:
  - CO2MPAS
  - tstamp.txt
  - inp/co2mpas_demo-8.xlsx
  - out/20170302_141422-co2mpas_demo-8.xlsx
  dices:
  - 'dices/IP-10-AAA-2017-1009/0: 2bc0977ba7f2ead4e9abfc4e0ab800984cf6dc6b'
  dices_count: 1
  revs_count: 4
  files_count: 4
  last_cdate: '2017-06-23 15:01:59+02:00'
  author: Dimitrios Komnos <dimitris.komnos@outlook.com>
  is_current: false
RL-99-BM3-2017-0001:
  msg.s: wltp_iof
  msg.a: imp 3 files
  last_dice: null
  last_commit: 6e5d63300b011c5aee68cc82072b19d201fd249b
  last_tree: 4a6ed28a981348c9281627edf619d982be10aa43
  files:
  - CO2MPAS
  - inp/input.xlsx
  - other/LICENSE.txt
  - out/output.xlsx
  dices: []
  dices_count: 0
  revs_count: 1
  files_count: 4
  last_cdate: '2017-02-06 19:00:36+00:00'
  author: Kostis Anagnostopoulos <ankostis@gmail.com>
  is_current: true

Is that what you asked?

[Piluri]

My question is, is it possible to export the dicision:

0967354.a@stamper.itconsult.co.uk: To: company@mail.com Subject: 'Proof of Posting Certificate 0967354:[co2dice.test]: dices/IP-10-AAA-2017-1002/0' Date: Fri, 30 Jun 2017 16:55:05 +0100 project: IP-10-AAA-2017-1002 dice: decision: OK hexnum: ADD1BCE040765BA582701FA622CF99AD82191AE0 percent: 88

TAA need this confirmation.

[ankostis]

A message formatted like your example is not stored anywhere, per se, because you combined 2 disparate elements: the tstamped email-response + the decision.

The tstamped reply email does not contain the decision - just the tstamper's (random) signature, as explained in this diagram:

It is co2dice application that parses that signature and derives the decision, which is then stored in the internal git repo. The following image depicts the whole timestamping procedure:

You may read any old decisions with the commands shown in my previous comment. Since those decisions contain also the cryptographic hashes, they can be sent as a proof (I've updated the console printout with a sampled project for you to see).

Finally, a programmer, or someone comfortable with git commands can retrieve the actual parsed response emails and other artifacts (inp, out, or other files). For the rest, "opening" the project and visiting .co2dice/repo folder is a viable solution, as long as the user is careful not modify these files.

Do the above answer your questions? In any case, if you have some specific use case in mind, we would happily discuss it.