Use Forbidden API and explicit Locale and Charset

[GoogleCodeExporter]

As explained in [https://code.google.com/p/crawler-commons/issues/detail?id=63] 
we currently rely on the default locale and charset in many places in the code 
which is error prone.

Using [https://code.google.com/p/forbidden-apis/] will help discover such cases 
and fix them.

Patch attached includes the Forbidden API in our pom. Run 'mvn clean test' to 
see the results.  

Original issue reported on code.google.com by digitalpebble on 23 Jan 2015 at 9:19

Attachments:

• 66-forbidden.patch

[GoogleCodeExporter]

See [http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html] 
for a good explanation

Original comment by digitalpebble on 23 Jan 2015 at 11:26

[GoogleCodeExporter]

New patch which explicits the Locale to Locale.ROOT. It could make sense to use 
Locale.getDefault() in some places e.g. logs, but at least this would make 
things more explicit. It also uses Charset.defaultCharset() explicitly, we'll 
improve the detection of the charset later as appropriate, for instance when 
parsing the sitemaps.

Still need to fix the test classes which currently give 63 errors.

Making the use of the defaults explicit in our code has only a relatively minor 
advantage but it means that we can use the Forbidden API by default and be able 
to detect more serious issues later on.

Original comment by digitalpebble on 23 Jan 2015 at 12:15

[GoogleCodeExporter]

Hi Julien - the patch looks like it's just the change to the pom.xml, not the 
changes to the code to fix up reported problems.

Original comment by kkrugler...@transpac.com on 26 Jan 2015 at 1:24

[GoogleCodeExporter]

indeed. I must have attached the wrong file. Will fix that on Wednesday, thanks 
for pointing this out.

Original comment by digitalpebble on 26 Jan 2015 at 5:55