⚠️ WORK IN PROGRESS ⚠️
This repository contains a complete test environment for the Exchange System (ES, Italian: SDI) for Electronic Invoices, including implementations for the ES itself and for the other participants.
The test environment can be used to:
simulate the complete process of invoice issue, transmission and receipt, including all notifications and handling of anomalous situations
during the development of SDICoop compliant services, simulate, debug and test locally their interaction with the ES and other actors
develop SDICoop-compliant services, forking the IR (Issuer/Recipient) implementation
develop higher-level applications that interact with a SDICoop compliant service, i.e. user interfaces, invoice/notification archiving ...
At this stage the testsdi is WIP and not fully implemented, most importantly these features are missing:
Some functionalities are also excluded from the initial design:
FatturaElettronicaBody
elements ("multi-invoices") (see issue #22)Index:
The ES system run by the Italian government (SDICoop Service - Transmit / Receive) is used to transmit and receive invoices, receipts and notifications.
There are three actors:
issuer (Italian: trasmittente), implements 1 SOAP Web Service (WS):
and uses the ES SDICoop Transmit / SdIRiceviFile endpoint to issue invoices
ES (forwards invoice from issuer to recipient Italian: SDI), implements 2 SOAP WS:
and uses:
and recipient (Italian: destinatario), implements 1 SOAP WS:
and uses the ES SdIRiceviNotifica WS to send notifications
The following animation shows the minimal workflow from invoice issue to receipt of acceptance:
This can also be seen grouping separately the two services:
SDICoop Transmit
SDICoop Receive
There is some English documentation but it's outdated. The Italian documentation is more up-to-date.
The testsdi is monolithic but modular, so that specific functionalities can be easily extracted.
A distinctive design choice has been to use the same database schema, API and structure for all actors. Rather than breaking it down in components based on the actors, it has been broken down in layers:
The SOAP server component exposes the interfaces required to communicate in accordance with the FatturaPA specification; it uses the fatturapa-core classes
The core component (fatturapa-core), has:
Base
, Issuer
, Exchange
and Recipient
classes.The control component (fatturapa-control), also uses fatturapa-core, and exposes a Remote Procedure Calls (RPC) API over the HTTP protocol. This API can be used to control the simulation / tests or to show status information in user interfaces.
The ui component fatturapa-ui provides a basic User Interface to interact with the test environment.
This picture shows how the 4 layers stack up:
This screencast demonstrates the complete workflow (see Demo section below) as seen through the UI, i.e. how you can send an invoice from I/R 0000001 to I/R 0000002, and make sure that the various notifications are sent back and forth between the three involved actors until the invoice acceptance is confirmed for all the parties:
The invoices change state during the workflow; certain state changes trigger notifications that have to be sent to specific actors.
The states are represented with strings starting with E_
for exchanger states, I_
for issuer states, R_
for recipient states and N_
for notification states.
The possible states, state changes and triggers are shown in the following state diagrams.
Legend for all state diagrams:
Status | Description |
---|---|
I_UPLOADED | ready to be transmitted |
I_TRANSMITTED | transmitted to ES |
I_DELIVERED | ES notified that it was delivered to Recipient |
I_FAILED_DELIVERY | failed delivery within 48 hours |
I_INVALID | ES notified it was invalid |
I_IMPOSSIBLE_DELIVERY | ES notified that it was not delivered to the recipient within 48 hours + 10 days |
I_ACCEPTED | ES notified that it was accepted by the recipient |
I_REFUSED | ES notified that it was refused by the recipient |
I_EXPIRED | ES notified that it was not accepted / refused by the recipient for more than 15 days |
Status | Description |
---|---|
E_RECEIVED = I_TRANSMITTED | received from transmitter |
E_VALID | passed checks |
E_FAILED_DELIVERY | failed delivery within 48 hours |
E_DELIVERED | delivered to recipient |
E_INVALID | did not pass test |
E_IMPOSSIBLE_DELIVERY | failed delivery within 48 hours + 10 days |
E_ACCEPTED | Recipient notified that it accepted the invoice |
E_REFUSED | Recipient notified that it refused the invoice |
E_EXPIRED | not accepted / refused by the recipient for more than 15 days |
Status | Description |
---|---|
R_RECEIVED = E_DELIVERED | received from ES |
R_ACCEPTED | Accepted |
R_REFUSED | Refused |
R_EXPIRED | ES notified that it was not accepted / refused by the recipient for more than 15 days |
Status | Description |
---|---|
N_RECEIVED | inbound notification has been received |
N_PENDING | outbound notification has been generated and must be processed |
N_OBSOLETE | outbound notification has been generated but must not be processed because another notification has been generated that makes notifcation of this one useless |
N_DELIVERED | outbound notification has beensuccessfully delivered |
There is a common database for all actors, consisting in three tables:
invoices:
notifications:
channels (lookup table between cedente
and issuer
):
For each of the four SOAP Web Services, we started from the Web Services Description Language, (WSDL) and XML Schema Definition, (XSD) files from fatturapa.gov.it, fed them to wsdl2phpgenerator which generated types and boilerplate for the endpoint in a directory named as the endpoint.
This code generation step has been performed once and for all by the soap/bin/generate.php script.
In each of the four resulting directory matching the endpoints, we place a index.php
file similar to (this one is for the SdIRiceviFile
endpoint):
require_once("SdIRiceviFileHandler.php");
$srv = new \SoapServer('SdIRiceviFile_v1.0.wsdl');
$srv->setClass("SdIRiceviFileHandler");
$srv->handle();
which leverages the PHP SoapServer class and delegates the implementation to a handler class SdIRiceviFileHandler
.
The handler class is implemented in a file with the same name SdIRiceviFileHandler.php
in the endpoint directory, and uses robust type cheching thanks to type hinting and the type declarations obtained from wsdl2phpgenerator.
Tested on: amd64 Debian 9.5 (stretch, current stable) with PHP 7.0 and Laravel 5.5.44.
Install prerequisites:
sudo apt install php-cli php-fpm nginx php-soap php-mbstring php-dom php-zip composer nginx postgresql php-pgsql php-curl php-xml
Clone the repo into the /var/www/html
directory on your webserver.
cd /var/www/html
git clone https://github.com/italia/fatturapa-testsdi .
Install prerequisites with composer:
cd /var/www/html
composer install
Configure the database:
in /etc/postgresql/9.6/main/pg_hba.conf
add this line:
local testsdi www-data md5
before this one:
# "local" is for Unix domain socket connections only
local all all peer
restart postgresql with: sudo systemctl restart postgresql
Create the database:
sudo su - postgres
psql
CREATE USER "www-data" WITH PASSWORD 'www-data';
CREATE DATABASE testsdi OWNER "www-data";
^d
^d
You'll be able to access the database with:
PGPASSWORD="www-data" psql -U www-data testsdi
Configure database credentials in core/config.php
and in rpc/config/database.php
.
Configure HOSTNAME
in soap/config.php
and in core/config.php
.
Set up Laravel:
cd /var/www/html
sudo chown www-data core/storage/time_travel.json
cd rpc
touch storage/logs/laravel.log
sudo chown -R www-data storage/logs
sudo chmod g+w storage/logs/laravel.log
sudo chown -R www-data storage/framework
sudo chown -R www-data bootstrap/cache
cp .env.example .env
php artisan key:generate
^d
Configure nginx:
sudo rm /etc/nginx/sites-enabled/*
sudo vi /etc/nginx/sites-enabled/fatturapa
Set the contents of the /etc/nginx/sites-enabled/fatturapa
file to something like:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name testsdi.simevo.com;
root /var/www/html;
index index.html index.htm index.php;
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
fastcgi_read_timeout 300;
}
location ^~ /sdi/rpc/js/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/js/;
}
location ^~ /sdi/rpc/css/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/css/;
}
location ^~ /sdi/rpc/webfonts/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/webfonts/;
}
location ^~ /sdi/rpc/font/ {
alias /var/www/html/rpc/packages/fatturapa/ui/src/public/font/;
}
location ~ /.*/rpc {
try_files $uri $uri/ /rpc/index.php?$query_string;
}
location ~ /.*/soap {
try_files $uri $uri/ /soap/index.php;
}
}
Finally check the configuration and restart nginx:
sudo nginx -t
sudo systemctl restart nginx
At this point you should be able to access the UI at: https://testsdi.example.com/sdi/rpc/dashboard
Dynamic routing makes sure that the RPC endpoints for the actors will be reachable at:
/sdi
- the Exchange System (there's only one)/tdxxxxxxx
, /tdyyyyyyy
, ... - where td stands for trasmittente/destinatario (T/D), Italian for issuer/receiver (I/R) and xxxxxxx
, yyyyyyy
are the 7-characters I/R identification codes.The number of simulated I/R (T/D) actors are autoconfigured based on the actors that appear in the channels
table.
For example if you set the channels table like this so that invoices can be sent (needed for the tests):
INSERT INTO channels(cedente, issuer) VALUES ('IT-01234567890', '0000001');
INSERT INTO channels(cedente, issuer) VALUES ('IT-12345678901', '0000002');
INSERT INTO channels(cedente, issuer) VALUES ('IT-23456789012', '0000003');
your SOAP endpoints will be at:
Sample manual session to demonstrate the flow of one invoice from issuer 0000001 to recipient 0000002, and acceptance:
clear status
POST https://www.example.com/sdi/rpc/clear
POST https://www.example.com/td0000001/rpc/clear
POST https://www.example.com/td0000002/rpc/clear
create a valid sample invoice for TD 0000002 (FatturaElettronica.FatturaElettronicaHeader.DatiTrasmissione.CodiceDestinatario
should be set to 0000002
) and upload it to TD 0000001, then check it is in the right queue
POST https://www.example.com/td0000001/rpc/upload {file XML}
GET https://www.example.com/td0000001/rpc/invoices?status=I_UPLOADED
force transmission to ES and check status:
POST https://www.example.com/td0000001/rpc/transmit
Check status with ES (the invoice should be in the E_RECEIVED queue):
GET https://www.example.com/sdi/rpc/invoices?status=E_RECEIVED
Check status with TD 0000001 (the invoice should be in the I_TRANSMITTED queue):
GET https://www.example.com/td0000001/rpc/invoices?status=I_TRANSMITTED
force validation by ES and check status:
POST https://www.example.com/sdi/rpc/checkValidity
GET https://www.example.com/sdi/rpc/invoices?status=E_VALID
force transmission from ES to recipient and check status:
POST https://www.example.com/sdi/rpc/deliver
GET https://www.example.com/sdi/rpc/invoices?status=E_DELIVERED
GET https://www.example.com/sdi/td0000002/invoices?status=R_RECEIVED
GET https://www.example.com/td0000001/rpc/invoices?status=I_DELIVERED (no response yet because ES has not notified to issuer)
force ES to dispatch back the notification to the issuer:
POST https://www.example.com/sdi/rpc/dispatch
check notification and status, now for the issuer TD 0000001 the invoice should be in the I_DELIVERED queue:
GET https://www.example.com/td0000001/rpc/notifications/id
GET https://www.example.com/td0000001/rpc/invoices?status=I_DELIVERED
make recipient accept invoice and check status:
POST https://www.example.com/td0000002/rpc/accept/id
GET https://www.example.com/td0000002/rpc/invoices?status=R_ACCEPTED
GET https://www.example.com/sdi/rpc/invoices?status=E_ACCEPTED (no response yet)
force receiver to dispatch back the notification to the ES:
POST https://www.example.com/td0000002/rpc/dispatch
check notification and status:
GET https://www.example.com/sdi/rpc/notifications/id
GET https://www.example.com/sdi/rpc/invoices?status=E_ACCEPTED
GET https://www.example.com/td0000002/rpc/invoices?status=I_ACCEPTED (no response yet)
force ES to dispatch back the acceptance notification to the issuer:
POST https://www.example.com/sdi/rpc/dispatch
check notification and status:
GET https://www.example.com/td0000001/rpc/notifications/id
GET https://www.example.com/td0000002/rpc/invoices?status=I_ACCEPTED
You can test manually by making SOAP requests using Postman.
You can import this collection into Postman, to test the AttestazioneTrasmissioneFattura
operation of the TrasmissioneFatture
web service (change the url to match that of your test server !):
{
"variables": [],
"info": {
"name": "SOAP",
"_postman_id": "0ee991f3-5203-a8ac-6b38-32c8bfabc05e",
"description": "",
"schema": "https://schema.getpostman.com/json/collection/v2.0.0/collection.json"
},
"item": [
{
"name": "SDICoop Transmit / TrasmissioneFatture service, AttestazioneTrasmissioneFattura operation",
"request": {
"url": "http://testsdi.simevo.com/td0000001/soap/TrasmissioneFatture/",
"method": "POST",
"header": [
{
"key": "Content-Type",
"value": "text/xml;charset=\"utf-8\"",
"description": ""
},
{
"key": "SOAPAction",
"value": "http://www.fatturapa.it/TrasmissioneFatture/AttestazioneTrasmissioneFattura",
"description": ""
}
],
"body": {
"mode": "raw",
"raw": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<SOAP-ENV:Envelope\n xmlns:SOAP-ENV=\"http://schemas.xmlsoap.org/soap/envelope/\"\n xmlns:ns1=\"http://www.fatturapa.gov.it/sdi/ws/trasmissione/v1.0/types\">\n\t<SOAP-ENV:Body>\n\t\t<ns1:attestazioneTrasmissioneFattura>\n\t\t\t<IdentificativoSdI>104</IdentificativoSdI>\n\t\t\t<NomeFile>IT01234567890_11111_AT_001.xml</NomeFile>\n\t\t\t<File>UEQ5NGJXd2dkbVZ5YzJsdmJqMGlNUzR3SWlCbGJtTnZaR2x1WnowaVZWUkdMVGdpUHo0S1BEOTRiV3d0YzNSNWJHVnphR1ZsZENCMGVYQmxQU0owWlhoMEwzaHpiQ0lnYUhKbFpqMGlRVlJmZGpFdU1TNTRjMndpUHo0S1BIUjVjR1Z6T2tGMGRHVnpkR0Y2YVc5dVpWUnlZWE50YVhOemFXOXVaVVpoZEhSMWNtRWdlRzFzYm5NNmRIbHdaWE05SW1oMGRIQTZMeTkzZDNjdVptRjBkSFZ5WVhCaExtZHZkaTVwZEM5elpHa3ZiV1Z6YzJGbloya3ZkakV1TUNJZ2VHMXNibk02ZUhOcFBTSm9kSFJ3T2k4dmQzZDNMbmN6TG05eVp5OHlNREF4TDFoTlRGTmphR1Z0WVMxcGJuTjBZVzVqWlNJZ2RtVnljMmx2Ym1VOUlqRXVNQ0lnZUhOcE9uTmphR1Z0WVV4dlkyRjBhVzl1UFNKb2RIUndPaTh2ZDNkM0xtWmhkSFIxY21Gd1lTNW5iM1l1YVhRdmMyUnBMMjFsYzNOaFoyZHBMM1l4TGpBZ1RXVnpjMkZuWjJsVWVYQmxjMTkyTVM0eExuaHpaQ0FpUGdvOFNXUmxiblJwWm1sallYUnBkbTlUWkVrK01qRTBQQzlKWkdWdWRHbG1hV05oZEdsMmIxTmtTVDRLUEU1dmJXVkdhV3hsUGtsVU1ERXlNelExTmpjNE9UQmZNVEV4TVRFdWVHMXNMbkEzYlR3dlRtOXRaVVpwYkdVK0NqeEVZWFJoVDNKaFVtbGpaWHBwYjI1bFBqSXdNVFF0TURRdE1ERlVNVEk2TURBNk1EQThMMFJoZEdGUGNtRlNhV05sZW1sdmJtVStDanhFWlhOMGFXNWhkR0Z5YVc4K0NpQWdJQ0E4UTI5a2FXTmxQa0ZCUVVGQlFUd3ZRMjlrYVdObFBnb2dJQ0FnUEVSbGMyTnlhWHBwYjI1bFBsQjFZbUpzYVdOaElFRnRiV2x1YVhOMGNtRjZhVzl1WlNCa2FTQndjbTkyWVR3dlJHVnpZM0pwZW1sdmJtVStDand2UkdWemRHbHVZWFJoY21sdlBnbzhUV1Z6YzJGblpVbGtQakV5TXpRMU5qd3ZUV1Z6YzJGblpVbGtQZ284VG05MFpUNUJkSFJsYzNSaGVtbHZibVVnVkhKaGMyMXBjM05wYjI1bElFWmhkSFIxY21FZ1pHa2djSEp2ZG1FOEwwNXZkR1UrQ2p4SVlYTm9SbWxzWlU5eWFXZHBibUZzWlQ0eVl6Rm1NMkV5TkRCaE1EVTJaRGsxTXpkaE9EWXdPR1psWkRNeE1EZ3hNbVZtTjJJeFlqZGhOREV3WkRBeE5USm1OV001WXpsbE9UTTBPRFpoWlRRMFBDOUlZWE5vUm1sc1pVOXlhV2RwYm1Gc1pUNEtQQzkwZVhCbGN6cEJkSFJsYzNSaGVtbHZibVZVY21GemJXbHpjMmx2Ym1WR1lYUjBkWEpoUGc9PQ==</File>\n\t\t</ns1:attestazioneTrasmissioneFattura>\n\t</SOAP-ENV:Body>\n</SOAP-ENV:Envelope>"
},
"description": ""
},
"response": []
}
]
}
Test the PHP classes with:
phpunit --testdox tests
This project complies with the PSR-2: Coding Style Guide.
Lint the code with:
./vendor/bin/phpcs --standard=PSR2 xxx.php
SOAP client/server interactions can be tricky to debug.
The issue is even more complicated when you perform a RPC calls (such as POST /sdi/rpc/transmit
) which has to perform internally a SOAP call.
You'll then have: client -> 1st level server (RPC) | SOAP client -> SOAP server | 2nd level server.
It is easy to trace and debug the 1st level server:
echo
is sent back to clienterror_log
statements get written to /var/log/nginx/error.log
.For the 2nd level server it's more complicated:
echo
because the body is used to return XML payloadserror_log
statements get lost.To make sure you get to see all messages written to log at the 2nd level server (SOAP server) edit /etc/php/7.0/fpm/pool.d/www.conf
adding these lines at the end:
catch_workers_output = yes
php_flag[display_errors] = on
php_admin_value[error_log] = /var/log/fpm-php.www.log
php_admin_flag[log_errors] = on
then create the new log file and make sure it can be written by the webserver:
sudo touch /var/log/fpm-php.www.log
sudo chown www-data:www-data /var/log/fpm-php.www.log
and finally restart the servers:
systemctl restart nginx && systemctl restart php7.0-fpm
Another option you have is to use instrumented versions of the PHP SoapClient
and SoapServer
builtins.
To instrument a SOAP client, use SoapClientDebug
instead of SoapClient
, for example for TrasmissioneFatture
add this to soap/TrasmissioneFatture/autoload.php
:
'SoapClientDebug' => __DIR__ .'/../SoapClientDebug.php',
then modify TrasmissioneFatture/TrasmissioneFatture_service.php
like this:
- class TrasmissioneFatture_service extends \SoapClient
+ class TrasmissioneFatture_service extends \SoapClientDebug
To instrument a SOAP server, use SoapServerDebug
instead of SoapServer
, for example for TrasmissioneFatture
make sure soap/index.php
has:
require_once("SoapServerDebug.php");
then modify TrasmissioneFatture/index.php
like this:
-$srv = new \SoapServer(dirname(__FILE__) . '/TrasmissioneFatture_v1.1.wsdl');
+$srv = new SoapServerDebug(dirname(__FILE__) . '/TrasmissioneFatture_v1.1.wsdl');
$srv->setClass("TrasmissioneFattureHandler");
$srv->handle();
+foreach ($srv->getAllDebugValues() as $value) {
+ error_log('==== '. print_r($value, true));
+}
For your contributions please use the git-flow workflow.
Emanuele Aina, Riccardo Mariani, Marco Peca and Paolo Greppi.
Copyright (c) 2018, simevo s.r.l.
License: AGPL 3, see LICENSE file.