TIMEN dies silently on some input

[GoogleCodeExporter]

What steps will reproduce the problem?

    TIMEN timen = new TIMEN();
    String normalized = timen.normalize("almost five month now", "2012-12-12");
    System.out.println(normalized);

What is the expected output? What do you see instead?

I expect to see something like "2013-05-12". Instead, the entire Java program 
exits shortly after printing:

line 1:0 no viable alternative at input 'ADD'
Errors found (TIMEN):
    null

Please use labels and text to provide additional information.

There's no stack trace or anything. It looks like TIMEN is calling System.exit 
all over the place, something you should pretty much never do in Java code.

Bad error handling aside, I still have no idea what's the problem with the 
input "almost five month now" or why it produces a null result. My only 
workaround at the moment is to create my TIMEN instances like:

  new TIMEN(Locale.ENGLISH) {

    @Override
    public String apply_rules(
        ArrayList<Rule> rules_found,
        String pattern,
        TIMEX_Instance timex_object) {
      // override to avoid NullPointerException in TIMEN.normalize
      String norm_value = super.apply_rules(rules_found, pattern, timex_object);
      return norm_value == null ? "" : norm_value;
    }
  };

Original issue reported on code.google.com by steven.b...@gmail.com on 6 Mar 2013 at 2:41

[GoogleCodeExporter]

Hi!

Very good catch. I have no time now to solve it properly but I will.

You can easily fix it by editing timen rules (SQLite DB)

rule for TUnit_Num_now must be ADD(DCT,PAT(1),NEGATIVEINT(PAT(0)))
same happens with TUnit_Num_today that must be ADD(DCT,PAT(1),INT(PAT(0)))

It seems that errors in grammar parsing go to some system.exit... will work on 
that and please tell me if you find the location of the bug before I do.

Thanks!
Hector

Original comment by hectorlm...@gmail.com on 11 Mar 2013 at 11:10

[GoogleCodeExporter]

A new ruledb has been committed. Java code still needs to improve.

Original comment by hectorlm...@gmail.com on 11 Mar 2013 at 11:16

[GoogleCodeExporter]

TIMEN.java has been updated and committed to handle this kind of errors.

    public String apply_rules(ArrayList<Rule> rules_found, String pattern, TIMEX_Instance timex_object) {
        String norm_value = "default_norm";
        try {
            if (rules_found.size() == 0) {
                // debugg (return default)
                if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                    System.err.println("\tNo rules found for: " + pattern);
                }
            } else {
                // debugg
                for (Rule rule : rules_found) {
                    if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                        System.err.println("\t" + rule.get_id() + " " + rule.get_pattern() + " rule to apply: " + rule.get_rule());
                    }
                }

                if (rules_found.size() == 1) {
                    //apply
                    norm_value = Rule_Engine.apply(rules_found.get(0), this, timex_object);
                    if (norm_value == null) {
                        throw new Exception("[ERROR] Malformed rule in the used rules_xx.db: " + rules_found.get(0).get_rule());
                    }
                    if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                        System.err.println("result: " + norm_value);
                    }
                } else {
                    // choose one:
                    // Disambiguation: select the first that matches all conditions
                    boolean select;
                    if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                        System.err.println("Disambiguation needed...");
                    }
                    for (Rule rule : rules_found) {
                        select = true;
                        if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                            System.err.println("\t" + rule.get_id() + " rule to apply: " + rule.get_rule() + " conditions: " + rule.get_rule_condition());
                        }
                        if (rule.get_rule_condition() != null) {
                            // TODO: WE SHOULD CREATE A CONDITION GRAMMAR TOO...
                            //BUT FOR THE MOMENT WE CAN ALLOW ONLY ONLY ONE CONDITION
                            String condition = rule.get_rule_condition().replaceAll("\\s+", "");
                            String operator = condition.replaceAll("[^!<=>]", "");
                            String[] condition_args = rule.get_rule_condition().split("(<=|>=|==|!=)");
                            if (condition_args.length < 2) {
                                condition_args = rule.get_rule_condition().split("(<|>)");
                            }
                            if (condition_args.length != 2 || operator.length() < 1) {
                                throw new Exception("Malformed condition in rule: " + rule.get_id() + " condition: " + condition);
                            }
                            // get PAT values
                            if (condition_args[0].matches("PAT\\([^)]*\\)")) {
                                condition_args[0] = timex_object.getNormTextArr()[Integer.parseInt(condition_args[0].replaceFirst("PAT\\(([^)]*)\\)", "$1"))];
                            }
                            if (condition_args[1].matches("PAT\\([^)]*\\)")) {
                                condition_args[1] = timex_object.getNormTextArr()[Integer.parseInt(condition_args[1].replaceFirst("PAT\\(([^)]*)\\)", "$1"))];
                            }
                            // get tense values
                            if (condition_args[0].equalsIgnoreCase("tense")) {
                                condition_args[0] = timex_object.getTense();
                            }
                            if (condition_args[1].matches("tense")) {
                                condition_args[1] = timex_object.getTense();
                            }

                            /*if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                            System.err.println(operator+"("+condition_args[0]+","+condition_args[1]+")");
                            } */
                            // handle operators
                            if (operator.equals("==") && !condition_args[0].equals(condition_args[1])) {
                                select = false;
                            }
                            if (operator.equals("!=") && condition_args[0].equals(condition_args[1])) {
                                select = false;
                            }
                            if (operator.equals("<=") && !(Integer.parseInt(condition_args[0]) <= Integer.parseInt(condition_args[1]))) {
                                select = false;
                            }
                            if (operator.equals(">=") && !(Integer.parseInt(condition_args[0]) >= Integer.parseInt(condition_args[1]))) {
                                select = false;
                            }
                            if (operator.equals("<") && !(Integer.parseInt(condition_args[0]) < Integer.parseInt(condition_args[1]))) {
                                select = false;
                            }
                            if (operator.equals(">") && !(Integer.parseInt(condition_args[0]) > Integer.parseInt(condition_args[1]))) {
                                select = false;
                            }
                        }
                        if (select) {
                            //apply
                            norm_value = Rule_Engine.apply(rule, this, timex_object);
                            if (norm_value == null) {
                                throw new Exception("[ERROR] Malformed rule in the used rules_xx.db: " + rules_found.get(0).get_rule());
                            }
                            if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                                System.err.println("result (rule " + rule.get_id() + "): " + norm_value);
                            }
                            break;
                        } else {
                            if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                                System.err.println("\tUnmached condition");
                            }
                        }
                    }
                }

            }
        } catch (Exception e) {
            System.err.println("Errors found (" + this.getClass().getSimpleName() + "):\n\t" + e.getMessage() + "\n");
            if (System.getProperty("DEBUG") != null && System.getProperty("DEBUG").equalsIgnoreCase("true")) {
                e.printStackTrace(System.err);
                System.exit(1);
            }
            return "default_norm";
        }
        return norm_value;

    }

Will update Maven asaic,
Hector

Original comment by hectorlm...@gmail.com on 12 Mar 2013 at 7:50

[GoogleCodeExporter]

Fixed

Original comment by hectorlm...@gmail.com on 12 Mar 2013 at 7:57

• Changed state: Fixed