Obtain the language files from the l10n and generate language files

[marcoscaceres]

For initial comment

[marcoscaceres]

@oyiptong, this is the first step in obtaining and parsing the .properties and .dtd files into JSON. Would like your feeback on bin/generatel10n.js.

We have a few options now:

• we can move those files into the src directory, and then consume them using fetch() as needed.
• we directly write the strings into the HTML or JS to keep everything static.

There are obvious pros and cons to each approach. My preference is that we just write them directly into the HTML files tho.

@everyone, thoughts?

[marcoscaceres]

Added throughput control, as MXR was crapping out (sockets were closing if too many simultanious connections were made... limited to 3 at a time). Works much better now, with fancy error recovery, etc. Also prints pretty colors.

More seriously tho, the l10n situation is quite sad - there are a lot of gaps. We will need to work with localizers to get those fixed.

@oyiptong r?

[oyiptong]

After a conversation on l10n:

We need to fetch the file shipped-locales instead of all-locales.

Instead of: https://raw.githubusercontent.com/mozilla/remote-newtab/l10n_generator/l10n/all-locales

We need to get:

https://hg.mozilla.org/releases/mozilla-aurora/raw-file/tip/browser/locales/shipped-locales

There are differences with what we have in remote-newtab vs what's in aurora/shipped-locales:

$ diff shipped-locales RNT-all-locales

3d2
< an
7d5
< az
19d16
< dsb
22d18
< en-US
43d38
< hsb
49,50c44
< ja linux win32
< ja-JP-mac osx
---
> ja-JP-mac
86d79
< uz

Here are example URLs for fetches:

https://hg.mozilla.org/releases/l10n/mozilla-aurora/az/raw-file/tip/browser/chrome/browser/newTab.dtd https://hg.mozilla.org/releases/l10n/mozilla-aurora/az/raw-file/tip/browser/chrome/browser/newTab.properties

[oyiptong]

more details on #126

[marcoscaceres]

Ok, I think I got this all working... but it needs to be committed in stages.