Taric Importer refactoring. It used to rely on hashes and proc's and they are hard to test and override if needed. Now it relies on class naming convention and duck typing instead. This makes specs easier to write (and read) and code to understand. Added some additional integration specs with Measure creation to ensure that we are really working.
Fixes an false positive conformance error we witnessed in Taric update for 31st of July (2013-07-31_TGB13144.xml) which said:
FO4
At least one description record is mandatory.
The start date of the first description period must be equal to the start date of the footnote. No two associated description periods may have the same start date.
The start date must be less than or equal to the end date of the footnote.
Why is this happening? Take a look at related long-ish Taric update excerpt which creates footnote, its description period and description:
<env:transaction id="13924647">
<env:app.message id="6">
<oub:transmission xmlns:oub="urn:publicid:-:DGTAXUD:TARIC:MESSAGE:1.0" xmlns:env="urn:publicid:-:DGTAXUD:GENERAL:ENVELOPE:1.0">
<oub:record>
<oub:transaction.id>13924647</oub:transaction.id>
<oub:record.code>200</oub:record.code>
<oub:subrecord.code>00</oub:subrecord.code>
<oub:record.sequence.number>3</oub:record.sequence.number>
<oub:update.type>3</oub:update.type>
<oub:footnote>
<oub:footnote.type.id>TM</oub:footnote.type.id>
<oub:footnote.id>835</oub:footnote.id>
<oub:validity.start.date>2013-08-01</oub:validity.start.date>
</oub:footnote>
</oub:record>
</oub:transmission>
</env:app.message>
<env:app.message id="8">
<oub:transmission xmlns:oub="urn:publicid:-:DGTAXUD:TARIC:MESSAGE:1.0" xmlns:env="urn:publicid:-:DGTAXUD:GENERAL:ENVELOPE:1.0">
<oub:record>
<oub:transaction.id>13924647</oub:transaction.id>
<oub:record.code>200</oub:record.code>
<oub:subrecord.code>05</oub:subrecord.code>
<oub:record.sequence.number>4</oub:record.sequence.number>
<oub:update.type>3</oub:update.type>
<oub:footnote.description.period>
<oub:footnote.description.period.sid>96485</oub:footnote.description.period.sid>
<oub:footnote.type.id>TM</oub:footnote.type.id>
<oub:footnote.id>835</oub:footnote.id>
<oub:validity.start.date>2013-08-01</oub:validity.start.date>
</oub:footnote.description.period>
</oub:record>
</oub:transmission>
</env:app.message>
<env:app.message id="15">
<oub:transmission xmlns:oub="urn:publicid:-:DGTAXUD:TARIC:MESSAGE:1.0" xmlns:env="urn:publicid:-:DGTAXUD:GENERAL:ENVELOPE:1.0">
<oub:record>
<oub:transaction.id>13924647</oub:transaction.id>
<oub:record.code>200</oub:record.code>
<oub:subrecord.code>10</oub:subrecord.code>
<oub:record.sequence.number>5</oub:record.sequence.number>
<oub:update.type>3</oub:update.type>
<oub:footnote.description>
<oub:footnote.description.period.sid>96485</oub:footnote.description.period.sid>
<oub:language.id>EN</oub:language.id>
<oub:footnote.type.id>TM</oub:footnote.type.id>
<oub:footnote.id>835</oub:footnote.id>
<oub:description>The frequency of physical checks carried out by the Member States shall be significantly reduced provided that the consignments are accompanied by the following documents:(a) the results of sampling and analysis performed by a USDA approved laboratory carried out in accordance with or equivalent to the provisions of Commission Regulation (EC) No 401/2006 of 23 February 2006 laying down the methods of sampling and analysis for the official control of mycotoxins in foodstuffs (TARIC certificate code C692); (b) a certificate in accordance with the model set out in the Annex of Decision 47/2008, completed, signed and verified by an authorised representative of the USDA for foodstuffs from the United States of America (TARIC certificate code C691).</oub:description>
</oub:footnote.description>
</oub:record>
</oub:transmission>
</env:app.message>
</env:transaction>
Since we process these records one by one we created and validated footnote first and found no associated description period. This change fixes that by enhancing TaricImporter::Transaction which now keeps records inside FIFO structure (Stack) and validates them in reverse order.
As an aside in the next monthly support, we should start to add documentation to the classes, explaining how they work and interact with each other, also explaining the weirdness we had to implement with CHIEF
There are two main changes here:
Fixes an false positive conformance error we witnessed in Taric update for 31st of July (
2013-07-31_TGB13144.xml
) which said:Why is this happening? Take a look at related long-ish Taric update excerpt which creates footnote, its description period and description:
Since we process these records one by one we created and validated footnote first and found no associated description period. This change fixes that by enhancing
TaricImporter::Transaction
which now keeps records inside FIFO structure (Stack) and validates them in reverse order.