Implement FacturX/XRechnung/ZUGFeRD for upcoming obligation to issue and receive electronic invoices

bastolino commented 5 months ago

Feature Request

As already asked by https://github.com/Dolibarr/dolibarr/issues/8836 FacturX/XRechnung is a european standard (EU-Richtlinie 2014/55/EU / EN 16931) and being enforced between companies from 2025 on, it is growing a basic feature that'd rule out the usage of Dolibarr in Germany from 2027 on. It is expectable that it is going to be mandatory for other european countries aswell.

It should be possible to import invoices from FacturX aswell as import format for supplier invoices.

Use case

all companies in germany need this from 2027 on.

Suggested implementation

some hints have already been made regarding libraries.

Suggested steps

xml embedded in pdf for export, xml aswell as pdf w/ embedded xml for import

rycks commented 5 months ago

@bastolino and all others, please note that there is a module for that : https://www.dolistore.com/fr/crm-gestion-relation-client/1511-FacturX---Facture---lectronique--Chorus----.html

i'm a the official developer of that module and i explain that this module will become free (gratuit / gratis / 0€) when that format will be mandatory (please read that for example: https://www.dolibarr.fr/forum/t/est-ce-que-dolibarr-pourra-lire-les-facture-x-en-2024/43169/9 in french but please use online translators if you can't read french).

All users who pay for that module make a direct sponsoring action because it's a hard job to make that sort of module (please note i do 6 "major" release and near that 100 intermediate releases).

So don't waste your time and let me know if you want to contribute to that module (source code is GNU/GPL).

Basti-Fantasti commented 4 months ago

@rycks Thanks for your response. This is great news that you will make this module freely available

As it will be mandatory for all companies in the EU it would be good, if this could be added as an inbuilt / default module in dolibarr.

rycks commented 4 months ago

@Basti-Fantasti maybe you have an information i don't but for me FacturX/XRechnung is only for Germany and France, not the others countries in europe: they use PeppolXML files / UBL

And even if that file format was for all european people i think it's a bad idea to put it into main dolibarr code because of all the rest of the world don't care about that specific file format.

For example do we have to put into dolibarr main code specific file format for each country / region ? We are working on a separate repository with external modules with community support which seems to be more flexible (and modules could be more "up to date" / "updates released faster" than the core).

Basti-Fantasti commented 4 months ago

@rycks oh I see. I thought that all European countries would need to have the FacturX/XRechnung implemented.

You are also right that it would not make much sense to add it to the default dolibarr installation from that point of view. But how would the implementation then work? Would everyone need to download the module from dolistore or would there be another way of distributing this module?

bastolino commented 4 months ago

Correct me but isn't this all very similiar with some changes to suit each countries flavour? FacturX/XRechnung is basically planned to be used within Peppol (Pan-European Public Procurement Online) Network for delivery. As far as I understand it, Peppol is the data exchange platform and FacturX/XRechnung is the format.

There is a (german) technology blog by SAP which brings all that into context: https://community.sap.com/t5/technology-blogs-by-sap/xrechnung-zugferd-und-peppol-alles-was-sie-%C3%BCber-die-elektronische-rechnung/ba-p/13427891

RetroYogi commented 4 months ago

I often read in Dolibarr forums that electronic invoicing will become mandatory in 2026/27. This is the date for France! Other EU countries have different deadlines. In Luxembourg, where I am, many types of companies (not all yet) are already obliged to use the Peppol network to send and receive invoices since March 2023. Therefore, please keep in mind that there are Dolibarr users for whom the e-invoicing obligation, and the obligation to use Peppol are already in place.

@bastolino, you're correct: Peppol is the network, and Factur-X is an e-invoice format. The used formats vary by country. For example, France accepts Factur-X and UBL (among others). In Luxembourg, we use XML, UBL, or something called CII, I think. The Peppol access points convert the sent and received information (invoices and other documnets) from one format to another. Peppol access points communicate using "messages" in a common format.

bastolino commented 4 months ago

Yes. In Germany, being able to receive electronic invoices is mandatory from January 1st 2025 on for each and every company. The need to be able to send electronic invoices is mandatory depending on company size.

The big point is that - for Germany - from 2025 on the XML part of the invoice is the leading one if there are differences between the (human readable) PDF and the XML part. While there is already a (buyable) Module for exporting FacturX, I don't see the possibility of parsing the XML part of it.

So is it possible to create an import tool to parse the FacturX and simply use the XML part of it to create a supplier invoice?

rycks commented 4 months ago

@bastolino import invoices is the job of ScanInvoices (see on dolistore) :-) and in case of factur-x import is done full automatic :-)

yes dolibarr is ready (and some of us are doing with that) !

bastolino commented 4 months ago

@rycks yes, but to use that you need an abo with your company and each and every invoice is sent to your webservers. importing pdfs with xmls shouldn't make OCR necessary, especially not sending them to any cloud service hosted by a different company.

spoonerarthur commented 2 months ago

01.01.2025 is getting closer and closer. Every company must guarantee the receipt of an XML invoice and also the automatic processing in the accounting department. If I have understood correctly, a special e-mail address must be set up for this, which Dolibarr must then retrieve and process. At the moment I don't know how it works in Dolibarr.

spoonerarthur commented 2 months ago

I've now done some more research on the internet. I think companies will have to fulfil both requirements, receiving and sending e-invoices

bastolino commented 2 months ago

@spoonerarthur as far as I understand: it's not yet implemented at all. The rules are: from Jan 1st 2025 on every company in Germany is required to be able to receive e-invoices. Depending on their size they will need to send e-invoices by 2027. For sending e-invoices, the FacturX-module is capable, with the latest updates a log of bugs and translational issues are solved. Regarding the receival process: I don't see any steps forward. The ocr stuff from Eric is OCR only and therefor not sufficient for the german legislature on XRechnung regarding xml first. (that means: if there are differences between pdf viewable and its corresponding xml, xml has priority for the fiscal authorities)

hregis commented 2 months ago

@spoonerarthur @Basti-Fantasti and Germany will not be conciliatory if a company is not yet in order by January 1, 2025? You're all going to end up in prison on January 1, do you think? I pity you! ;-) In my opinion, you have to stay calm, @rycks will sort out the problem, I trust him!

hregis commented 2 months ago

You have to tell your German government and all the governments of Europe that it is all well and good to be part of Europe, but if each country makes its own soup, Europe makes no sense!

hregis commented 2 months ago

To put it simply, for our society which is becoming stupid... it's like a football team where each player has their own rules of the game... it's a mess! And Europe is rubbish!

rycks commented 2 months ago

Hello @bastolino

Regarding the receival process: I don't see any steps forward. The ocr stuff from Eric is OCR only and therefor not sufficient for the german legislature on XRechnung regarding xml first. (that means: if there are differences between pdf viewable and its corresponding xml, xml has priority for the fiscal authorities)

please have a look at https://www.dolistore.com/7137-thickbox_default/OCR---ScanInvoices.jpg

ScanInvoices process is:

try to find xml data, if present use it, if conform to factur-x import that
fallback to data extraction if no xml
fallback to OCR if nothing works before

So you can import factur-x invoices sinces 2 years :-) if you want to check it i could make you an invoice just let me know the amount you want :-)

rycks commented 2 months ago

@spoonerarthur

If I have understood correctly, a special e-mail address must be set up for this, which Dolibarr must then retrieve and process. At the moment I don't know how it works in Dolibarr.

here is the answer (in french) but let me know if somebody want to translate it in other languages, i will copy all of that on dolibarr wiki to make translations easy !

https://doc.cap-rel.fr/projet_scaninvoices/import_mail

bastolino commented 2 months ago

Hello @rycks I have tried out your service with some dolibarr test instance and some real invoices I received containing xml. It did not work and the amounts were just clearly wrong. Also textfields were cut off and some other minor errors that happen when using OCR. Also, I don't want to upload all of my invoices to a third party and pay per invoice. Then I could go ahead and use DATEV aswell. But I want it on my own premise with as much open source as possible. I totally agree that it must not be for free but not that way and not to have such errors when trying just 3 invoices.

rycks commented 2 months ago

@bastolino so that's bugs ... did you open a ticket to send us that invoices to make analyze and correct bugs ? please do it

ScanInvoices is AGPL

Server code name is "docwizon" (it convert documents to json data) and docwizon could be self hosted like some of our partners does, i'm a fan of self hosted systems (and i make only free software since 1998 ...) so no problem with that.

sonikf commented 2 months ago

Hi everyone @eldy @rycks @hregis and Dolibarr team

This is primarily related to #16898

I think we are in a miss the forest for the trees situation..

In simple words Peppol is the network you have to connect regardless of the format all through an access point to send and receive invoices and possibly other documents automatically(this is the magic word)

As @RetroYogi(only one got it right so far) correctly stated Electronic invoicing is already implemented to various degrees all over the world with Peppol being the dominant standard so far. Countries outside Europe such as Australia, Canada, China, India, Japan, USA(that's about half the world population)...all using Peppol. You can see the full list here

All major commercial and open source ERP's have already implemented it as a core feature because simply Electronic invoicing is the new invoicing.

Please see Odoo implementation offered since October 2021

@rycks To start thank you for your work and contributions. In light of the above please be kind and answer some questions

Will you also give for free https://www.dolistore.com/en/modules/1585-Peppol-XML---facture-electronique.html ?
If yes my understanding is that in contrast with the FacturX module you only create the xml in Peppol module, what about sending/transmission? (In Dolistore you provide changelog only in French but in English, German info is scarce and points to only xml generation please fix this, you maybe losing customers!) Bear in mind that vendor lock-ins have already been imposed for access points providers. For example in Greece where Electronic invoicing is mandatory for B2G transactions only a Greek ministry licenced authorized access point provider can be used and a third country provider can either register as a Greek company and receive a licence or connect through a Greek access point provider(i hope this change for B2B or a platform like chorus will be created) So software bridges will be needed and this is also a new stream of revenue for module developers but also extra costs for customers..
Your module https://www.dolistore.com/7137-thickbox_default/OCR---ScanInvoices.jpg is only necessary for FacturX because in your module you only use chorus if i got that right by reading your changelogs. What about Peppol formats where you can request data the same way you transmit and automatically import vendor invoices that way?
You wrote "We are working on a separate repository with external modules with community support which seems to be more flexible (and modules could be more "up to date" / "updates released faster" than the core)." A NEXTCLOUD like implementation with module auto update would be perfect! Where is this repo at (i can't find it ) and how can i or anyone else willing can help?
Can you please provide a timeline/roadmap?

To address @hregis comments: Also thanks for your work and i agree with you that Europe is a mess but in this case France and Germany changed the recipe reinventing the wheel with FacturX/XRechnung/ZUGFeRD No one will go to prison and i also trust @rycks will figure this out but this is business software. Governments already offer incentives for early adoption and the fact is that unfortunately Dolibarr can't reliably provide right now!

eldy commented 2 months ago

There is a wiki page to describe the situation for each country here

https://wiki.dolibarr.org/index.php?title=Electronic_Billing

If you have information about format, dates of expectation for ebilling, thanks to fill this page...

sonikf commented 2 months ago

Hi @eldy Thanks for creating the wiki page! I have amended the already created entries for cohesion and structure. Also I added e-reporting which is also coming your way(France) in same timeline as e-invoicing. This is the same as MyData module used by the Greek community to transmit xml invoice data via an API. Still work to be done.. After speaking to an invoicing expert, extensively reading French legislation and to some extent German and visiting accredited French PDP's websites i verified that both countries officially support Peppol if not for other reasons for cross border exchanges. So is there a plan for Dolibarr to support Peppol?

eldy commented 2 months ago

Hi @eldy Thanks for creating the wiki page! I have amended the already created entries for cohesion and structure. Also I added e-reporting which is also coming your way(France) in same timeline as e-invoicing. This is the same as MyData module used by the Greek community to transmit xml invoice data via an API. Still work to be done.. After speaking to an invoicing expert, extensively reading French legislation and to some extent German and visiting accredited French PDP's websites i verified that both countries officially support Peppol if not for other reasons for cross border exchanges. So is there a plan for Dolibarr to support Peppol?

Thanks. What you mean with the text "e-reporting" is related to the transmission or declaration of the invoice (sending or transfer), right ?

I agree with you that the faster we have support natively or from a centralized community repository (not yet available) instead of support by an non official module, the better it is. But roadmap is above all handled by submitted contributions of developers. So I can't say myse if there is plan or not. Just that there is hope or intent by some people. In all cases, I will discuss about Peppol or other support during next devcamp in november to raise awareness to progress on this topics ...

sonikf commented 2 months ago

Thanks. What you mean with the text "e-reporting" is related to the transmission or declaration of the invoice (sending or transfer), right ?

No this is mainly for tax evasion and vat fraud checks from central authorities, please read section "Digital reporting requirements" from https://ec.europa.eu/digital-building-blocks/sites/display/DIGITAL/eInvoicing+in+France See our rudimentary but working implementation https://github.com/evansgl/dolibarr-mydata/tree/dolibarr-19/mydata/mydatalist

API easy translation with firefox https://www.aade.gr/sites/default/files/2024-07/myDATA%20API%20Documentation%20v1.0.9_official_erp.pdf

I agree with you that the faster we have support natively or from a centralized community repository (not yet available) instead of support by an non official module, the better it is. But roadmap is above all handled by submitted contributions of developers. So I can't say myse if there is plan or not. Just that there is hope or intent by some people. In all cases, I will discuss about Peppol or other support during next devcamp in november to raise awareness to progress on this topics ...

Thank you I'll open another issue about roadmap because i see evolution of parts but no communication is done before, so willing Dolibarrians can't offer insight or help. I think by being open and communicating issues and plans more people will help. I saw this issue by chance searching for sth else by recently updated but not everyone does that... In this spirit a request if possible, to open polls in forums

ultrasites commented 4 days ago

@eldy Maybe we could implement a PR with this dependency and some developers? https://github.com/horstoeko/zugferd

rycks commented 4 days ago

@ultrasites of course, that repo is already included into the facturx module and horstoeko do a incredible work but please read what we explain before, facturx is only for german and french people, then we did not include 20Mb of libs into dolibarr core...

@sonikf yes we are working on the community-official repository, we speaked about it during last week (devcamp in nancy) with @eldy and eldy asked me to manage (or be a part of managers of course) for that repository, nice i will do my best and it will take time (not paid)

@sonikf roadmap without $$$ is a little bit hard to plan no ? writing free (gpl) software is my job then i have to find solution to pay my time, dolistore is the way we use up to now but what for the future ?

Next step (yes a roadmap !!!!!) for me seems to be:

a) set up dolibarr-community repository
b) make a proposition on the wiki linked to that repository for the best way to put data (one folder per module, then one branch per dolibarr main version, for example FacturX branch 18.0 ... FacturX branch 20.0 but facturx or xrechnung please do not make a german/french fight then facturx-xrechnung folder... ?)
c) put code into it and all developper documentation (how to build that module with external depends and dolibarr "composer" point of view)
d) make release process
e) add dolibarr core auto-connexion to it
f) tests & validate (with dolibarr experiment tag)
g) put into dolibarr main without experiment tag

Then ... during that time you can buy the module (and sponsoring the job with that, please note that 20% of the amount is to dolibarr association) and contribute to "my" git repository (all of "my" customers has a access to that repository and can contribute, some of them do it like you can see on the changelog of facturx module for example).

ultrasites commented 4 days ago

@ultrasites of course, that repo is already included into the facturx module and horstoeko do a incredible work but please read what we explain before, facturx is only for german and french people, then we did not include 20Mb of libs into dolibarr core...

@sonikf yes we are working on the community-official repository, we speaked about it during last week (devcamp in nancy) with @eldy and eldy asked me to manage (or be a part of managers of course) for that repository, nice i will do my best and it will take time (not paid)

@sonikf roadmap without $$$ is a little bit hard to plan no ? writing free (gpl) software is my job then i have to find solution to pay my time, dolistore is the way we use up to now but what for the future ?

Next step (yes a roadmap !!!!!) for me seems to be:
* a) set up dolibarr-community repository

* b) make a proposition on the wiki linked to that repository for the best way to put data (one folder per module, then one branch per dolibarr main version, for example FacturX branch 18.0 ... FacturX branch 20.0 but facturx or xrechnung please do not make a german/french fight then facturx-xrechnung folder... ?)

* c) put code into it and all developper documentation (how to build that module with external depends and dolibarr "composer" point of view)

* d) make release process

* e) add dolibarr core auto-connexion to it

* f) tests & validate (with dolibarr experiment tag)

* g) put into dolibarr main without experiment tag
Then ... during that time you can buy the module (and sponsoring the job with that, please note that 20% of the amount is to dolibarr association) and contribute to "my" git repository (all of "my" customers has a access to that repository and can contribute, some of them do it like you can see on the changelog of facturx module for example).

Nice news!

When you will start?

Dolibarr / dolibarr