RyanMarcus / dirty-json

A parser for invalid JSON
GNU Affero General Public License v3.0
307 stars 30 forks source link

I have a very badly broken text, can the library be improved to handle it? #18

Closed RezaRahmati closed 4 years ago

RezaRahmati commented 5 years ago

Hi

I already checked with the demo, it doesn't convert it, but after few round of cleaning it start working, so wondering if it's possible to handle this worst case scenario?

id: \"55b3a5c5-16e7-4ae6-91bc-7f08fb152dde-ee1dc704\"\nlang: \"en\"\nsession_id: \"6b1f9ba2-7c79-47bc-aadb-2c600b111836\"\ntimestamp: \"2019-09-04T17:13:43.374Z\"\nresult {\n source: \"agent\"\n resolved_query: \"welcome\"\n action: \"customSettingsAnswer\"\n score: 1.0\n parameters {\n fields {\n key: \"key\"\n value {\n string_value: \"welcome\"\n }\n }\n fields {\n key: \"default\"\n value {\n string_value: \"Hello, you called condo bot. Your virtual concierge. How can I help you today?\"\n }\n }\n }\n metadata {\n intent_id: \"040b9e41-d20e-4da8-9fff-d2c1f1f5812e\"\n webhook_response_time: 4992\n intent_name: \"Default Welcome Intent - custom\"\n webhook_used: \"true\"\n webhook_for_slot_filling_used: \"false\"\n is_fallback_intent: \"false\"\n }\n fulfillment {\n speech: \"Hello, you called condo bot. Your virtual concierge. How can I help you today?\"\n messages {\n lang: \"en\"\n type {\n number_value: 0.0\n }\n speech {\n string_value: \"Hello, you called condo bot. Your virtual concierge. How can I help you today?\"\n }\n }\n }\n}\nstatus {\n code: 200\n error_type: \"success\"\n}\n

This is my string, what I did to make it work

  1. added { and } to start and end of string
  2. convert all \n to ,
  3. find all properties which has an immediate { and add a : between
  4. find all } , } and replace it with } }
  5. find all {, and replace it with {

after all of that modifications it starts working

RezaRahmati commented 5 years ago

actually if I only do this, it will convert this monster

        let str = input.trim();

        if (!(str.startsWith('[') || str.startsWith('{'))) {
            str = `{${str}}`;
        }

        str = str.replace(/(\w+\s*){/g, '$1: {');

        return JSON.stringify(dirtyJson.parse(str));
RyanMarcus commented 4 years ago

Yes, sadly this is probably a case that is too far gone... the problem is that there's no way to fix the \" and \n without breaking the parser on valid JSON... I'm glad you found a workaround!