Respect allowed character restrictions in SEPA XML

[tttp]

So remember that utf8 thing? turns out that the french banks don't like it, they want ascii.

So if you have émeric that wants to contributes, too bad, he needs to change his name (or move to a country that have technologies that dare being less than 20 years obsolete)

Anyway, it means that we need iconv to make sepa dd working.

@pdelbar, you have another suggestion?

[pdelbar]

Not sure they can do this ... utf8 is kinda standard. I'll check ... not that it will do us any good.

---

Paul Delbar office: +32 9277 9177 direct: +32 9277 9171 mobile: +32 468 330 314

advice and solutions for online communication and fundraising delius bvba, Henri Farmanstraat 40, 9000 Gent

On 30 Jul 2013, at 20:13, xavier dutoit notifications@github.com wrote:

So remember that utf8 thing? turns out that the french banks don't like it, they want ascii.

So if you have émeric that wants to contributes, too bad, he needs to change his name (or move to a country that have technologies that dare being less than 20 years obsolete)

Anyway, it means that we need iconv to make sepa dd working.

@pdelbar, you have another suggestion?

— Reply to this email directly or view it on GitHub.

[tttp]

Hi,

So the problem is that the french banks accepts utf8, as long as their isn't any éàè.... so as long as you limit yourself to ascii 7 bits utf8 is fine.

Anyway, what I did after my beloved bank rejected it with a braindead msg "too many errors, roughly somewhere in the file" (or something as descriptive)

$iconv -futf8 -tascii//TRANSLIT xml.xml > xml2.xml

and then the file worked fine (what it does is replace é by e à by a and so on)

Wanted to add that when generating the file not in the contrib page: there is no reason because banks are stupid and impose nonsense rules that civi should do the same and prevent the people that donate to use their real name.

On 30 July 2013 21:46, pdelbar notifications@github.com wrote:

Not sure they can do this ... utf8 is kinda standard. I'll check ... not that it will do us any good.

---

Paul Delbar office: +32 9277 9177 direct: +32 9277 9171 mobile: +32 468 330 314

advice and solutions for online communication and fundraising delius bvba, Henri Farmanstraat 40, 9000 Gent

On 30 Jul 2013, at 20:13, xavier dutoit notifications@github.com wrote:

So remember that utf8 thing? turns out that the french banks don't like it, they want ascii.

So if you have émeric that wants to contributes, too bad, he needs to change his name (or move to a country that have technologies that dare being less than 20 years obsolete)

Anyway, it means that we need iconv to make sepa dd working.

@pdelbar, you have another suggestion?

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/Project60/sepa_dd/issues/22#issuecomment-21817440 .

[antrik]

This is not at all a "French bank" thing. It is very explicitly described in the SEPA rulebooks.

What's worse, it's not ASCII either. (ASCII is always 7 Bit BTW. What is commonly referred to as "8 Bit ASCII" can be IBM 437 or Windows 1250 or ISO 8859-1 or something else, but none of them are ASCII.) The SEPA charset is much more limited than ASCII. (As you found out by trial&error by now, the ASCII characters '&' and '!' are also not supported, along with various others...) So just transliterating to ASCII is not sufficient. We have to do further replacements according to EPC recommendations. (Don't remember whether these are part of the SEPA rulebook, or a separate document...)

So this problem is not really solved yet. Not sure whether to reopen this issue, or create a new one...

BTW, the rulebook mentions that banks may optionally support extended charsets. The implementation guidelines of the German banking council have a statement saying that German banks have "commited" to accept all characters supported in the German legacy DD scheme -- which are a few more than the SEPA charset -- and convert them as needed. Don't know though whether this really works in practice... If so, we might need some country-specific handling in the SDD extension.

[tttp]

I have added a few extra conversions and haven't had problems since the past two months. If you have the rulebook, would be good you check, and see if the setlocale trick works for german ü eszett...

X+

[antrik]

Fixed in sfe branch.

(Well, except that it would still be nice to support optional extended charsets at national level...)

[bjendres]

This is implemented by the now merged SYSTOPIA version