[16.0][IMP] edi_oca: introduce a deduplication mechanism at the sending step

[QuocDuong1306]

Rebased on:

• 63

Example problematic scenario:

• User creates a new customer
• User goes through n rounds of (Edits, Saves)
• This generates n edi.exchange.record
• Given that we always send a full representation of the customer to WAMAS (and not incremental changes), we only need the last exchange record to be sent
• Sending more than the last echange record can create trouble:
  • if for some reason, WAMAS does not process the records in their filename order, it could save an outdated representation of the customer
  • too many records to process might clog queues and/or create delays/high load

Solution:

• Introduce a deduplication mechanism in edi_oca at the sending step

Pros:

• If needed for debugging , we can keep track of all the superseded records
• If last change on Customer ends up with a failure of action_exchange_generate, the previous representation will be sent
• The Output Sync cron controls the buffering window: if run every 5 minutes, we'll have at max one WAMAS file every 5 minutes (per customer)

Cons:

• We still generate a lot of queue jobs (generate, send),

[QuocDuong1306]

I have rebased on #63

[simahawk]

@QuocDuong1306 can you rebase?

[QuocDuong1306]

Hello @simahawk , I have rebased

[QuocDuong1306]

Hello, I updated this commit: https://github.com/OCA/edi-framework/pull/61/commits/0d4b9bdd1b180b06e0eab7b61aeee4d7bc3dc40e

Reason:

• Adding exchange_file in the domain is slow when files are stored in object storage
• we also remove useless domain ("type_id.direction", "=", "output"), it is not actually needed, as the only criteria that matters is the is_obsolete flag

[santostelmo]

@QuocDuong1306 still a draft PR ?

[QuocDuong1306]

Hello @simahawk @santostelmo , sorry for the delay, I will update it ASAP...

[QuocDuong1306]

I would move this feature to a new module edi_exchange_deduplicate_oca and apply the changes suggested

Hi @simahawk @santostelmo , I moved this one to the new PR, could you have a look? After that, maybe this one can be closed