Make Generics(.php) platform independent

[Michiel-s]

This issue describes a proposal to change the way by which the generated info from ampersand is provided to the backend framework. Currently all information is generated to Generics.php, a php file containing several php arrays. I propose to change this to multiple JSON files. JSON because it makes the generated information platform independent. Multiple files because that is more convenient to manually browse through, to e.g. bug fix.

The files are:

• [x] mysql-installer.json
• [x] concepts.json
• [x] relations.json
• [x] rules.json
• [x] interfaces.json
• [x] settings.json
• [x] conjuncts.json
• [x] tables.json can be removed. not used.
• services.json do not implement. old experiment, not used anymore
• [x] views.json
• [x] roles.json

Implement in backend (Michiel):

• [x] mysql-installer.json
• [x] concepts.json
• [x] relations.json
• [x] rules.json
• [x] interfaces.json
• [x] settings.json
• [x] conjuncts.json
• tables.json
• services.json
• [x] views.json
• [x] roles.json

All files should satisfy the JSON specification (see http://www.json.org for an introduction) php arrays can be easily converted into json arrays and objects.

mysql-installer.json

{ "allDBstructQueries" : 
  [ "<query1>"
  , "<query2>"
  , "<etc>"
  ]
, "allDefPopQueries" :
  [ "<query1>"
  , "<query2>"
  , "<etc>
  ]
}

concepts.json

• interfaces should contain the IDs of the toplevel interfaces that have the concept (or its generalizations) specified as SrcConcept.
• defaultViewId should specify the default view using the following reasoning: 1: If there is a view defined for this concept that is marked DEFAULT => use that one 2: If there is a view defined for a generalization concept that is marked DEFAULT => use the smallest one (if there are more than 1 smallest, just pick one!) 3: If there is no DEFAULT view defined => null (do not use a view here that is not marked default)

[ { name : "ConceptA"
  , type : "OBJECT"
  , generalizations : [ "ConceptX", ConceptY, "etc" ]
  , specializations : [ "ConceptB", ConceptC, "etc" ]
  , affectedConjuncts : [ "conj1", "conj2", "etc" ]
  , interfaces : [ "ifcA", "ifcB", "etc" ]
  , defaultViewId : "<name of default view>"
  , conceptTable : { name : "<table name>"
                            , cols : [ "<col1>", "<col2>", etc ]
                            }
}
, { another concept }
]

relations.json tableOf specifies if relation is administrated in table of srcConcept, tgtConcept or its own (n-n table)

[ { name : "rel"
  , signature : "rel[SrcConcept*TgtConcept]"
  , srcConcept : "SrcConcept"
  , tgtConcept : "TgtConcept"
  , uni : true|false
  , tot: true|false
  , inj : true|false
  , sur : true|false
  , affectedConjuncts : ["conj1", "conj2", "etc"]
  , mysqlTable : { name : "<table name>"
                         , tableOf : "src"|"tgt"|null
                         , srcCol : {name : "<col name>", null : true|false, unique : true|false}
                         , tgtCol : {name : "<col name>", null : true|false, unique : true|false}
                         }
  }
, { second relation }
, { etc }
]

roles.json maintains = rule names/ids of rules that are maintained by this role interfaces = interface ids of interfaces that are available to this role (including the interfaces for every role)

[ { id : <increasing nr>
  , name : "<role name>"
  , maintains : ["<rule1>", "<rule2>", "<etc>"]
  , interfaces : ["<interface1>", "<interface2>", "<etc>"]
  }
, { second role }
, { etc }
]

rules.json

interfaces.json

settings.json generateConstants from Generate.hs (isDev and autoRefreshInterval are obsolete) mysql settings from ObjBinGen.hs

{ "versionInfo" : "<versionInfo>"
, "contextName" : "<contextName>"
, "mysql-settings" : 
  { "dbHost" : "<dbHost>"
  , "dbName" : "<dbName>"
  , "dbUser" : "<dbUser>"
  , "dbPass" : "<dbPass>"
  , "signalTableName" : "<signalTableName>"
  }
}

[hanjoosten]

I think this is a good idea. The way to go would be to create these files beside the current .php files. Only when we all are happy with the contents in the .json files, and the frontend fully uses the .json files, the .php files will become obsolete.

[hanjoosten]

The approach to this (from a Haskell pov, would be to create a haskell data structure for each of these files, populate them using fspec. Then, by using the Aeson package to convert each of them to a json file. This should be pretty straitforward.

[hanjoosten]

Hi Michiel, I have built relations.json in a slightly different way. I have put all conjuncts of that relation in one array. That is closer to the Ampersand structure. A conjunct itself isn't signal or invariant. However, the rule where the conjunct is from has that property. It is possible to denormalize the property of the rule to a property of a conjunct, but that might have nasty consequences later on. I think that in the end you'd have nicer code when we keep it this way.

However, if you have a good reason to split at this level, then it can be done.

[hanjoosten]

I also added prettyprinting of the .json files

[hanjoosten]

Without any specification of Michiel, I made a rules.json. @Michiel-s , Let me know what you think about this!

[Michiel-s]

@hanjoosten

Take a look at the output of mysql-installer.json:

{
    "allDBstructQueries": [
        "$allDBstructQueries =",
        "  array ( 'CREATE TABLE \"__SessionTimeout__\"",
        "            ( \"SESSION\" VARCHAR(255) UNIQUE NOT NULL",
        "            , \"lastAccess\" BIGINT NOT NULL",
        "            ) ENGINE=InnoDB DEFAULT CHARACTER SET UTF8 COLLATE UTF8_BIN'",
        "        , 'CREATE TABLE \"__History__\"",
...

A couple of things need fixing:

• The $allDBstructQueries = is part of the php definition. This should not be here
• This also applies to the array (
• The 'CREATE TABLE .... COLLATE UTF8_BIN' is one query and should be 1 item in the json array.
• The single quotes around a query are not needed anymore. This was due to the php string. In JSON this is replaced by the single quotes.
• The order of the json array is very important. In other generated json files I saw that the order was different than the specification above.
• Double quotes need to be escaped.

[Michiel-s]

@hanjoosten

The rules.json looks good, except for the 'expSQL' field of a pairView item. It contains '\n' and space characters. These should be removed.

Furthermore, I saw that you've added a 'segmentNr'. Can you guarantee that these are put in the right order in the pairView array?

[Michiel-s]

@hanjoosten

The relations.json is ok, except for the 'signature' field of a relations. It now contains [Src*Tgt]. However, the full signature should also contain the relation name: rel[Src*Tgt]

Could you also add the multiplicities. UNI, TOT, INJ, SUR:

[ { name : "rel"
  , signature : "rel[SrcConcept*TgtConcept]"
  , srcConcept : "SrcConcept"
  , tgtConcept : "TgtConcept"
  , srcMin : 0 or 1
  , srcMax : 1 or null
  , tgtMin : 0 or 1
  , tgtMax : 1 or null
  , affectedConjuncts : ["conj1", "conj2", "etc"]
  }
, { second relation }
, { etc }
]

[Michiel-s]

I've updated the initial post and added specification for concepts.json

[hanjoosten]

On your comments about mysql-installer.json: Hmm. You are right. I went too fast. I could have noticed most of your issues myself. Sorry. About order in Json: There isn't any! So I added a seqNr to the queries. It looks however that the array is kept in order anyway. Not sure if that is by accident or if you may trust that ordering.

[hanjoosten]

Also fixed the other issues unto this comment.

[hanjoosten]

Michiel, I am not sure the segments of the viewdefinitions in concepts.json are generated correctly. Do you have a nice test-model somewhere?

[hanjoosten]

@Michiel-s more wishes for the .json files? let me know!

[Michiel-s]

@hanjoosten

For some reason I don't see roles.json yet. Could you:

• [x] han - add the roles.json output as specified below
• [x] michiel - process in backend

roles.json maintains = rule names/ids of rules that are maintained by this role interfaces = interface ids of interfaces that are available to this role (including the interfaces for every role)

[ { id : <increasing nr>
  , name : "<role name>"
  , maintains : ["<rule1>", "<rule2>", "<etc>"]
  , interfaces : ["<interface1>", "<interface2>", "<etc>"]
  }
, { second role }
, { etc }
]

[Michiel-s]

Implemented views.json in 0c4fad46353a4732753c0dacb66f37c534c49cd1

[Michiel-s]

@hanjoosten

I've updated the specs for the relations.json. Could you:

• [x] han - add the mysql table information exactly as specified below.
• [x] han - The tables.json file can be removed.
• [x] michiel - process in backend

relations.json tableOf specifies if relation is administrated in table of srcConcept (i.e. "src"), tgtConcept (i.e. "tgt") or its own n-n table (i.e. null).

[ { name : "rel"
  , signature : "rel[SrcConcept*TgtConcept]"
  , srcConcept : "SrcConcept"
  , tgtConcept : "TgtConcept"
  , uni : true|false
  , tot: true|false
  , inj : true|false
  , sur : true|false
  , affectedConjuncts : ["conj1", "conj2", "etc"]
  , mysqlTable : { name : "<table name>"
                         , tableOf : "src"|"tgt"|null
                         , srcCol : {name : "<col name>", null : true|false, unique : true|false}
                         , tgtCol : {name : "<col name>", null : true|false, unique : true|false}
                         }
  }
, { second relation }
, { etc }
]

[Michiel-s]

Hi @hanjoosten

I've updated the concept.json specs. Please:

• [x] han - add the defaultViewId attribute.
• [x] han - The old array with all view definitions (views attribute) can be removed here.
• [x] michiel - process in backend

concepts.json

• interfaces should contain the IDs of the toplevel interfaces that have the concept (or its generalizations) specified as SrcConcept.
• defaultViewId should specify the default view using the following reasoning: 1: If there is a view defined for this concept that is marked DEFAULT => use that one 2: If there is a view defined for a generalization concept that is marked DEFAULT => use the smallest one (if there are more than 1 smallest, just pick one!) 3: If there is no DEFAULT view defined => null (do not use a view here that is not marked default)

[ { name : "ConceptA"
  , type : "OBJECT"
  , generalizations : [ "ConceptX", ConceptY, "etc" ]
  , specializations : [ "ConceptB", ConceptC, "etc" ]
  , affectedConjuncts : [ "conj1", "conj2", "etc" ]
  , interfaces : [ "ifcA", "ifcB", "etc" ]
  , defaultViewId : "<name of default view>"
  , conceptTable : { name : "<table name>"
                            , cols : [ "<col1>", "<col2>", etc ]
                            }
}
, { another concept }
]

[Michiel-s]

Implemented roles.json in 72458e7b5634446c3cac15c2374e25a20a8c296b

[Michiel-s]

All json files are implemented in the backend.

[hanjoosten]

I cleaned up conform issue #198. I also modified the Cruds a little in the interface.json file, as we have spoken of before. The cruds are now in one object together, at all places. It might be that some of the connections with the backend is broken by that action. @Michiel-s, please check this, then you can close this issue again.

[Michiel-s]

All ok.