Import JSON File is importing fine but does not save on OK

[GoogleCodeExporter]

Recently I just finished writing an import script that will convert all 
Compound Kanji to their hiragana equivalent. It consists of a MASSIVE amount of 
Kanji combinations (roughly 79,000).

It loads into the FoxReplace options window properly.

But the problem is, that when I press OK, the FoxReplace options window 
immediately closes and when I open it to check it again, the newly imported 
replacement sections are no longer there.

I already have 37 groups containing more than 7000 substitutions in all (amoung 
other groups, I have groups that convert Jōyō Kanji, as well as Hiragana and 
Katakana, to their romanji equivalents. My intention was to be able to convert 
Japanese (Kanji, Hiragana and Katakana) into it's Romanji equivalent.

I tried the following approaches to solve the problem but none of it worked:
1. At first I put all the Compound Kanji replacements into one group and then 
importing them but it was no good.

2. Then I tried splitting it into many groups with roughly 1000 replacements 
per group but that didn't work either.

3. I then tried importing one group at a time but that only worked until Part 
07. Part 08 onwards had the same problem.

4. I even tried clearing all the groups and only importing the Compound Kanji 
groups but that didn't work either.

So here is my Question. Is there a limit to the amount of replacement 
groups/entries that FoxReplace can remember? Or is there a problem in the 
script?

Please help me out. You have no idea how important this script can be. It can 
revolutionize fan translation of Japanese Light Novels. Making it far more 
easier to translate between the two languages since more fans know how to 
understand romanji than written Japanese (kanji, hiragana, katakana).

If you want the file in question please tell me. I'd rather send it to you 
directly than risk exposing it publicly before the script is complete. I don't 
particularly care what happens after the process of conversion is completed.

Original issue reported on code.google.com by jedi200...@gmail.com on 9 Aug 2014 at 2:38

[GoogleCodeExporter]

Ah. I forgot to mention but, I'm using Firefox 31.0 with FoxReplace 0.16.1.

Original comment by jedi200...@gmail.com on 9 Aug 2014 at 2:41

[GoogleCodeExporter]

Hi, as I understand, you have created a JSON file to be imported to Firefox, 
right? If it fails, the most likely cause is that there's a syntax error in the 
file, which should be captured and shown but it isn't, as reported in issue 
117. From you third approach, it seems that the error may be in part 08.

There's a way to check if the cause is a syntax error:

1. Open the browser console (in the web developer menu, or with Ctrl+Shift+J)
2. Uncheck all buttons/tabs except the JS one
3. Clear the console
4. Import the JSON file
5. Look in the error console if there is an error

This should also work to know if it's another kind of error. Let me know if 
something gets written in the error console with this test.

Original comment by marc.r...@gmail.com on 10 Aug 2014 at 10:48

• Changed title: Import JSON File is importing fine but does not save on OK

[GoogleCodeExporter]

It's giving the following error:

The character encoding of the plain text document was not declared. The 
document will render with garbled text in some browser configurations if the 
document contains characters from outside the US-ASCII range. The character 
encoding of the file needs to be declared in the transfer protocol or file 
needs to use a byte order mark as an encoding signature. 
FoxReplace_Compound_Kanji_to_Romanji.json

I checked the groups that were imported in FoxReplace upon importing of the 
JSON file. There was no garbled text.
So I doubt this is what's causing the problem of the groups not remaining after 
OK is clicked (Remember, the file imports properly. The groups just don't stay 
when OK in the fox replace options dialog box is clicked).
I'll send you the file in question through e-mail. I'll have to send it in two 
pieces though. Do you have 7-zip. I suggest you download it if you don't (it's 
freeware).

Original comment by jedi200...@gmail.com on 10 Aug 2014 at 6:34

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Ok, send me the file, then. Yes, I have 7-zip.

Original comment by marc.r...@gmail.com on 10 Aug 2014 at 6:39

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I'd need your email adress for that. For some reson when I clcik on your email 
adress it redirects me to a captcha and then it doesn't give me the full 
address.

Original comment by jedi200...@gmail.com on 10 Aug 2014 at 6:41

[GoogleCodeExporter]

Ah...

Only project members can see each others full email address.
It's a security feature on your googlecode profile.

So that's why.

Original comment by jedi200...@gmail.com on 10 Aug 2014 at 6:49

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Haha. I totally get what you mean. I'm sending the file in two pieces via two 
e-mails.

Original comment by jedi200...@gmail.com on 10 Aug 2014 at 7:06

[GoogleCodeExporter]

You can delete your e-mail post now.

Original comment by jedi200...@gmail.com on 10 Aug 2014 at 7:07

[GoogleCodeExporter]

Annnnd sent. You should have the file in a few seconds, if not already.

Original comment by jedi200...@gmail.com on 10 Aug 2014 at 7:24

[GoogleCodeExporter]

I have received them. I will tell you when I discover something.

Original comment by marc.r...@gmail.com on 10 Aug 2014 at 11:13

[GoogleCodeExporter]

Ok, it looks like a problem of size indeed: the JSON string is too large to be 
stored to a preference. I get this error on the console:

NS_ERROR_ILLEGAL_VALUE: Component returned failure code: 0x80070057 
(NS_ERROR_ILLEGAL_VALUE) [nsIPrefBranch.setComplexValue]

I suppose you didn't see it because I have some Firefox debug preferences 
enabled. Reducing dramatically the size, I get a warning:

Warning: attempting to write 736072 bytes to preference 
extensions.foxreplace.substitutionListJSON. This is bad for general performance 
and memory usage. Such an amount of data should rather be written to an 
external file.

And with this it works. I had seen this warning before a few weeks ago and I'm 
currently working on storing the substitution list to a file, but now I see it 
must be a top priority. I will release a new version when I have this change 
ready.

Original comment by marc.r...@gmail.com on 10 Aug 2014 at 11:41

• Changed state: Accepted
• Added labels: Component-Persistence, OpSys-All, Priority-Critical, Type-Defect
• Removed labels: Priority-Medium, Type-Other

[GoogleCodeExporter]

Understood. Thanks. Keep us informed of the progess.

Original comment by jedi200...@gmail.com on 12 Aug 2014 at 6:00

[GoogleCodeExporter]

So... How's it going? Any luck?

Original comment by jedi200...@gmail.com on 6 Sep 2014 at 8:04

[GoogleCodeExporter]

Hi, I haven't worked on it in the last three weeks because I was on holidays, 
but work is in progress.

Original comment by marc.r...@gmail.com on 6 Sep 2014 at 12:09

• Changed state: Started

[GoogleCodeExporter]

How's it going?

Original comment by jedi200...@gmail.com on 13 Nov 2014 at 4:46

[GoogleCodeExporter]

I was hoping to have it ready by the end of october, but I will need a few more 
weeks. But it will be out before the end of the year, I promise!

Original comment by marc.r...@gmail.com on 13 Nov 2014 at 9:16

[GoogleCodeExporter]

Fixed in r245. I will release a beta soon so you can try it if you want.

Original comment by marc.r...@gmail.com on 6 Dec 2014 at 11:53

• Changed state: Fixed

[GoogleCodeExporter]

I have uploaded the beta to addons.mozilla.org. If you want to try it you will 
find it at https://addons.mozilla.org/firefox/addon/foxreplace/#beta-channel at 
the bottom of the page, under "Development channel".

I have tried the file you sent me and it can be imported, although it takes a 
few seconds and there's no progress indicator (I should implement it before the 
final release). Please let me know if you find any issue.

Original comment by marc.r...@gmail.com on 12 Dec 2014 at 10:35

[GoogleCodeExporter]

Seems to be working fine. Thanks.

The progress indicator idea seems like a good one.

Now all I need is a "Move to Top" and "Move to Bottom" button in the Options 
window. Since the order of the groups and substitutions is important it would 
be good idea to implement that before rolling out the official update.

Original comment by jedi200...@gmail.com on 13 Dec 2014 at 8:35