autoFormalization and autoVocabulary is empty

[lex-lex]

See title, example: https://nai.uni.lu/theory/5ce43ea028332221b630ea8c

Or is the annotation somehow wrong?

<h2>Article 3 - Freedom of choice</h2>
<p><br></p>
<ol>
  <li>
    <span class="connective-depth-1 annotator-connective" id="2a7dc4b6-8b46-42d9-8959-7bdb7e010d12" data-connective="obonif">
      <span class="annotator-term" id="7e74072b-b8fd-4cf0-8dc9-387767de1013" data-term="contract(Law,Part)">A contract</span>
      shall be governed by
      <span class="annotator-term" id="1a3346fb-2d3f-43d1-be2a-dd588c6ad3fd" data-term="valid_choice(Law,Part)">the law chosen by the parties</span>.
    </span>
    The choice shall be made expressly or clearly demonstrated by the terms of the contract or the circumstances of the case. By their choice the parties can select the law applicable to the whole or to part only of the contract.
  </li>
  [...]
</ol>

[lex-lex]

@shaolintl : Also, I just checked the test cases for the "autoFormalization" property of the theory. As far as I understand, it will return something like (say result.autoFormalization[0] is the following)

{
  "text": "A contract shall be governed by the law chosen by the parties.",
  "connective": {
    "name": "Obligation Only If",
    "code": "obonif",
    "formulas": [
      {
        "text": "A contract",
        "term": {
          "name": "contract(Law,Part)"
        }
      },
      {
        "text": "the law chosen by the parties",
        "term": {
          "name": "validChoice(Law,Part)"
        }
      }
    ]
  }
}

assuming that the content of the theory contained these annotations. Now, this output seems irrelevant to me (at least at the current state of the front end); I will display a table of formulas consisting of (1) their original string from the text and (2) the formula that was created automatically for it. However, the return result.autoFormalization[0] does not contain the complete formula so I would need to reconstruct it. Personally, I'd favor a simple response object for autoFormalization just like for formalization, such as

autoFormalization: [
  {
    "_id" : "<something>"
    "original" : "A contract shall be governed by the law chosen by the parties.",
    "formula" : "<....something....>"
  },
  ....
]

This way, autoFormalization and formalization can be treated the same in the front end. Do you see any points why I would need the more verbose version?

[shaolintl]

@lex-lex For the first issue, I had a problem parsing two class values which I have just solved but according to my tests, the error is supposed to be error code 400 and message 'Cannot parse XML. Unknown annotator value: -depth-1 annotator-connective'. I guess you are hiding these particular messages? Anyway, now the tests succeed with this particular content but the error message is the same. If you are hiding the particular error message from the server, can you send me the correct one?

[shaolintl]

@lex-lex For the second issue. In the test case I do check the json object but it is stored in the property json of the autoFormalization[i]. autoFormalization[i] contains the fields: original (text), json and formula.

[lex-lex]

I had a problem parsing two class values [...]

Right, I see. It might be a good thing to ignore all other HTML attributes that are inside the theory's annotation spans, as their might be some of them appearing and/or disappearing over the course of development, e.g. when new features are introduced to the front end or so. This is also why I figured that adding additional classes shouldn't do any harm. What do you think?

[...] I guess you are hiding these particular messages?

Strange, in my tests the back end – also before your recent commit – gives a 200 OK with { message: "Theory updated" }. So there is no error message sent back by the back end during a theory update.

[...] autoFormalization[i] contains the fields: original (text), json and formula. Perfect!

I will try the new back end version now.

[lex-lex]

Hmm, autoFormalization and autoVocabulary is still empty for my test theory on nai.uni.lu, see https://nai.uni.lu/theory/5ce43ea028332221b630ea8c

@shaolintl Do you have any idea what might be the cause?

[shaolintl]

I had a problem parsing two class values [...]

Right, I see. It might be a good thing to ignore all other HTML attributes that are inside the theory's annotation spans, as their might be some of them appearing and/or disappearing over the course of development, e.g. when new features are introduced to the front end or so. This is also why I figured that adding additional classes shouldn't do any harm. What do you think?

I completely agree. I am not using html that much and forgot a class can have more than one value. You can use any property now. I have fixed it and it doesnt looks for the right value.

[...] I guess you are hiding these particular messages?

Strange, in my tests the back end – also before your recent commit – gives a 200 OK with { message: "Theory updated" }. So there is no error message sent back by the back end during a theory update.

My mistake, I was logged in with another account and this is th reason it was not saved. I should probably prevent GET requests as well if the wrong user is logged in. Right now I just search for the id

[shaolintl]

Hmm, autoFormalization and autoVocabulary is still empty for my test theory on nai.uni.lu, see https://nai.uni.lu/theory/5ce43ea028332221b630ea8c

@shaolintl Do you have any idea what might be the cause?

@lex-lex Yep, I assumed the content contains an html tag in order to parse it and it does not. Now I only check if it is not null. So indeed the parser was not executed. Now I can see vocabulary but not formalization. Are you sure you display there theory.autoFormalization?

[shaolintl]

Anyway, I added a debug printout and I see that the html is being converted to autoFormalizations. I also have tests which load the updated theory and check that the formalizations were saved and retrieved properly.

"Converted content <h2>Article 3 - Freedom of choice</h2><p><br></p><p><span class=\"connective-depth-1 annotator-connective\" id=\"02d85b71-28a3-4026-a5ab-1ae89d70360d\" data-connective=\"obonif\"><span class=\"annotator-term\" id=\"aed75366-c18d-4dc3-8448-2c1f40318c49\" data-term=\"a\">A contract </span>shall be governed by the law chosen <span class=\"annotator-term\" id=\"1e673319-fc48-4a68-a54b-c25268c58003\" data-term=\"b\">by the parties</span></span>. The choice shall be made expressly or clearly demonstrated by the terms of the contract or the circumstances of the case. By their choice the parties can select the law applicable to the whole or to part only of the contract.</p><p>The parties may at any time agree to subject the contract to a law other than that which previously governed it, whether as a result of an earlier choice made under this Article or of other provisions of this Regulation. Any change in the law to be applied that is made after the conclusion of the contract shall not prejudice its formal validity under Article 11 or adversely affect the rights of third parties.</p><ol><li>Where all other elements relevant to the situation at the time of the choice are located in a country other than the country whose law has been chosen, the choice of the parties shall not prejudice the application of provisions of the law of that other country which cannot be derogated from by agreement.</li><li>Where all other elements relevant to the situation at the time of the choice are located in one or more Member States, the parties' choice of applicable law other than that of a Member State shall not prejudice the application of provisions of Community law, where appropriate as implemented in the Member State of the forum, which cannot be derogated from by agreement.</li><li>The existence and validity of the consent of the parties as to the choice of the applicable law shall be determined in accordance with the provisions of Articles 10, 11 and 13.</li></ol> 

into formalization

 [{\"original\":\"A contract shall be governed by the law chosen by the parties\",\"json\":{\"text\":\"A contract shall be governed by the law chosen by the parties\",\"connective\":{\"name\":\"Obligation Only If\",\"code\":\"obonif\",\"formulas\":[{\"text\":\"A contract \",\"term\":{\"name\":\"a\"}},{\"text\":\"by the parties\",\"term\":{\"name\":\"b\"}}]}},\"formula\":\"(b O> a)\"}]."

[shaolintl]

Also, when I try to enter illegal annotations (i.e. implication with only one argument), I get "Theory not saved, an error occurred: Error: Request failed with status code 400". I am sending more informative errors. Maybe the error message can be printed as well?

[lex-lex]

Thanks for the update. I can see the autoformalization stuff now, great!

Now I can see vocabulary but not formalization. Are you sure you display there theory.autoFormalization?

Nope, I never said I did ;-)

I am sending more informative errors. Maybe the error message can be printed as well?

Sure, that comes on the list.

I guess the matter of this ticket is resolved so I'll close it.

Best Alex