Need to handle German Sonderwurst

[antrik]

So you thought that with the SEPA file format being an European standard, it would work across Europe?

Well, guess again. German banks decided the standard format ain't good enough for them, and came up with an own subset. This would be fine if it was a recommendation of what is most likely to work reliably -- but it's not. According to our initial testing, at least some German banks actually refuse to accept standard SEPA files, and will only take those using the German variant. Hurray for standards...

In other words, we will need some conditionals in the SDD extension, so it can also generate German SEPA files instead of standard SEPA files. Or we could perhaps introduce some hook(s) so an extra "German SEPA" extension could modify the behaviour of the standard SEPA extension as needed... Though the latter would bear a distinct tinge of over-engineering in my opinion :-)

[tttp]

I can't say I'm really surprised.

Let's see if I'm smart enough to avoid the german bank trap:

The xml is generated using a smarty template. if you create another german sepa extension and put the modified template into that, you should be good.

X+

On 30 October 2013 16:50, Olaf Buddenhagen notifications@github.com wrote:

So you thought that with the SEPA file format being an European standard, it would work across Europe?

Well, guess again. German banks decided the standard format ain't good enough for them, and came up with an own subset. This would be fine if it was a recommendation of what is most likely to work reliably -- but it's not. According to our initial testing, at least some German banks actually refuse to accept standard SEPA files, and will only take those using the German variant. Hurray for standards...

In other words, we will need some conditionals in the SDD extension, so it can also generate German SEPA files instead of standard SEPA files. Or we could perhaps introduce some hook(s) so an extra "German SEPA" extension could modify the behaviour of the standard SEPA extension as needed... Though the latter would bear a distinct tinge of over-engineering in my opinion :-)

— Reply to this email directly or view it on GitHubhttps://github.com/Project60/sepa_dd/issues/48 .

[antrik]

From a cursory glance, this should be trivial for two of the necessary changes: the different schema definition; and not including the optional field in , which doesn't exist in the German variant.

Also, the templates should be static enough that this won't become a maintainance nightmare I hope...

There is one issue though: the German variant restricts allowed characters in -- and '_' is not among them. Are you willing to merge a patch that will change the default file references to use '-' instead, so we don't need special handling? Otherwise, we will need a hook or something...

BTW, I hope this is not a silly question -- but how exactly do I go about reliably overriding the template in another extension?

[tttp]

MsgId is the name of the group, happy to change it. Don't have any problem we change it to something that makes everyone happy, as long as it indicates the type and the date

eg: "2013-11-21 RCUR" or whatever the format that works for your bank, don't care ;)

Reliably: not silly question at all. there isn't a "weight" on extension. we need to have them loading in the right order. Can you check with @totten how to be sure that in the include path we have the german extension that comes before the sepa one?

X+

On 30 October 2013 18:13, Olaf Buddenhagen notifications@github.com wrote:

From a cursory glance, this should be trivial for two of the necessary changes: the different schema definition; and not including the optional

field in , which doesn't exist in the German variant. Also, the templates should be static enough that this won't become a maintainance nightmare I hope... There is one issue though: the German variant restricts allowed characters in -- and '_' is not among them. Are you willing to merge a patch that will change the default file references to use '-' instead, so we don't need special handling? Otherwise, we will need a hook or something... BTW, I hope this is not a silly question -- but how exactly do I go about reliably overriding the template in another extension? — Reply to this email directly or view it on GitHubhttps://github.com/Project60/sepa_dd/issues/48#issuecomment-27414119 .

[totten]

I don't fully understand the context, but to address the question about include-paths and ordering:

• Standard warning: One should avoid designs where it's necessary to worry about the order -- e.g. overriding files provided by someone else is a tricky proposition. It's hard to maintain/upgrade, and you can get boxed into a corner when there are multiple modules with multiples overrides and conflicting priorities.
• Strictly speaking, each extension has some control over its placement in the include path, so one can hack around a little. In civix-based modules, hook_civicrmconfig delegates to *_civix_civicrm_config which includes this:

  $include_path = $extRoot . PATH_SEPARATOR . get_include_path();
  set_include_path($include_path);

If you want to manipulate the order, then you can edit hook_civicrmconfig, disable the call to *_civix_civicrm_config, and put your own initialization code directly in hook_civicrm_config. For example, in the lower-priority extension, you might say:

  $include_path =  get_include_path() . PATH_SEPARATOR . $extRoot;
  set_include_path($include_path);

It won't matter which extension initializes first -- the lower-priority extension will always come later in the include-path than the higher-priority extension.

[tttp]

Hi Tim. the context is having an extension (sepa germany) that depends on the main sepa. eg. to implement custom hooks or change the template to alter the default xml file exported. If you think "simply" overriding the default template is going to be an issue (I expected to be changed to handle national specific needs), we might add a hook that sets the template file to use.

Still the need of being able to have an extension that extends the extension is very much a need ;)

On 30 October 2013 19:28, Tim Otten notifications@github.com wrote:

I don't fully understand the context, but to address the question about include-paths and ordering:

• Standard warning: One should avoid designs where it's necessary to worry about the order -- e.g. overriding files provided by someone else is a tricky proposition. It's hard to maintain/upgrade, and you can get boxed into a corner when there are multiple modules with multiples overrides and conflicting priorities.

• Strictly speaking, each extension has some control over its placement in the include path, so one can hack around a little. In civix-based modules, hook_civicrm_config delegates to ***civix_civicrm_config which includes this:

  $include_path = $extRoot . PATH_SEPARATOR . get_include_path(); set_include_path($include_path);

If you want to manipulate the order, then you can edit hook_civicrm_config, disable the call to ***civix_civicrm_config, and put your own initialization code directly in hook_civicrm_config. For example, in the lower-priority extension, you might say:

$include_path = get_include_path() . PATH_SEPARATOR . $extRoot; set_include_path($include_path);

It won't matter which extension initializes first -- the lower-priority extension will always come later in the include-path than the higher-priority extension.

— Reply to this email directly or view it on GitHubhttps://github.com/Project60/sepa_dd/issues/48#issuecomment-27423693 .

[antrik]

After some serious consideration, I'd prefer to go with a single extension after all.

As fun as the "extension to an extension" concept would be :-) -- I fear it would be too much hassle both for us and for our users.

Also, it would be less flexible: it seems perfectly possible that one CiviCRM instance might serve both cerditors using normal banks, and creditors using German banks. Thus, I'm now thinking of an option in the PP configuration:

SEPA File format: (x) pain.008.001.02 (ISO 20022/official SEPA guidelines) ( ) pain.008.003.02 (German Sonderwurst)

Also, rather than using completely separate templates, I think it would be much easier to maintain conditionals in the standard template for the two things that are actually different.

[tttp]

It would be more flexible, as your extension might decide if they want to replace or not the template and do that with more criteria.

Anyway, I think your idea is better in that case indeed and add on the creditor what xml format they want.

Tip: ing provides an online tool to validate sepa file that everyone can use and that returns error messages that are quite understandable. I like to hate banks as much as the next guy, but they did good on that one.

On 1 November 2013 04:22, Olaf Buddenhagen notifications@github.com wrote:

After some serious consideration, I'd prefer to go with a single extension after all.

As fun as the "extension to an extension" concept would be :-) -- I fear it would be too much hassle both for us and for our users.

Also, it would be less flexible: it seems perfectly possible that one CiviCRM instance might serve both cerditors using normal banks, and creditors using German banks. Thus, I'm now thinking of an option in the PP configuration:

SEPA File format: (x) pain.008.001.02 (ISO 20022/official SEPA guidelines) ( ) pain.008.003.02 (German Sonderwurst)

Also, rather than using completely separate templates, I think it would be much easier to maintain conditionals in the standard template for the two things that are actually different.

— Reply to this email directly or view it on GitHubhttps://github.com/Project60/sepa_dd/issues/48#issuecomment-27545611 .

[tttp]

Clarification:

We might have other "country specific" rules than the xml format (eg. different rules to generate the group name or whatever) and could imagine that we have different set of rules within the same country (eg. the french part of belgium having whatever different than the flemish one)

can you put "localisation" as the name of the field and store as a string(10)? I don't think we need to make it configurable via option, we can php hardcode the list (right now "DEFAULT" and "DE").

am I clear? ping me on irc after my first coffee ;)

On 1 November 2013 07:14, Xavier Dutoit xavier@sydesy.com wrote:

It would be more flexible, as your extension might decide if they want to replace or not the template and do that with more criteria.

Anyway, I think your idea is better in that case indeed and add on the creditor what xml format they want.

Tip: ing provides an online tool to validate sepa file that everyone can use and that returns error messages that are quite understandable. I like to hate banks as much as the next guy, but they did good on that one.

On 1 November 2013 04:22, Olaf Buddenhagen notifications@github.comwrote:

After some serious consideration, I'd prefer to go with a single extension after all.

As fun as the "extension to an extension" concept would be :-) -- I fear it would be too much hassle both for us and for our users.

Also, it would be less flexible: it seems perfectly possible that one CiviCRM instance might serve both cerditors using normal banks, and creditors using German banks. Thus, I'm now thinking of an option in the PP configuration:

SEPA File format: (x) pain.008.001.02 (ISO 20022/official SEPA guidelines) ( ) pain.008.003.02 (German Sonderwurst)

Also, rather than using completely separate templates, I think it would be much easier to maintain conditionals in the standard template for the two things that are actually different.

— Reply to this email directly or view it on GitHubhttps://github.com/Project60/sepa_dd/issues/48#issuecomment-27545611 .

[antrik]

Fixed by #81.