DE Fiskaly: Split TAR export to avoid "E_TOO_MANY_RECORDS" exception

[saschaknapp]

Issue

Fiskaly has a export limit of 1000000 signatures. Exceeding this limit results in the TAR export failing with the exception E_TOO_MANY_RECORDS. We have customers who reach this limit.

Possible fix

Fiskaly shared the following information with us:

We do recommend that you split your request to avoid triggering too much data at once.

E.g. if tss.signature_counter is equal to 2500000, then a complete export of all TSS records should be done in 3 batches:

1. First export with end_signature_counter=1000000
2. Second export with start_signature_counter=1000001 and end_signature_counter=2000000
3. Third export with start_signature_counter=2000001

Further information

If you need to know which customer is affected on our side, then please just send me a message on slack to keep this private.

Tasks

• [x] Create a test TSS with many transactions
• [x] Before running a fiskaly export, check how many signatures will be exported (by comparing the current signature counter with the previous one from the metadata property)
  • This will return the transaction number difference, which is not the same as the signature difference that we need here
  • To get the amount of exported signatures from the number of transactions, you can do this (simplified): Number of signatures = Number of Transactions * 2.5
• [x] If the export would include more than 800k signatures, split the export to multiple requests, and combine the resulting TAR file (like we do in the DeutscheFiskal SCU here)
• [x] Test the split & merge functionality by temporary setting the split range to e.g. 100 (to avoid creating a super large TSE)