Add support for array type in tabular manifests

[sirex]

When property has prop[] name, then it should be implied, that prop is of array type.

When prop[] ref Model is given, then a many to many relationship between models should be created.

When Model.prop array Other is given, then a many to many relationship between models should be created though Other table instead of using Model/:list/prop hidden table. In this case prop[] type must be ref and must be given. By default, many to many relationship will be created though properties, that matches current and prop[] ref models.

When prop array Model[prop1, prop2] is given, then many to many relationship must be created through prop1 and prop2 properties in Model. This must be used, when several refs are used to the same models used in many to many relationship.

• https://docs.sqlalchemy.org/en/14/core/constraints.html#defining-foreign-keys
• https://docs.djangoproject.com/en/4.0/ref/models/fields/#django.db.models.ManyToManyField.through

Examples:

d | r | b | m | property    | type    | ref
example                     |         |
                            |         |
  |   |   | Language        |         |
  |   |   |   | name@en     | text    |
                            |         |
  |   |   | Country         |         |
  |   |   |   | name@en     | text    |
  |   |   |   | languages[] | ref     | Language

Here a hidden table example/Country/:list/languages will be created and used as intermediate table for many-to-many relationship with Language.

A custom intermediate table can be specified like this:

d | r | b | m | property    | type    | ref
example                     |         |
                            |         |
  |   |   | Language        |         |
  |   |   |   | name@en     | text    |
                            |         |
  |   |   | Country         |         |
  |   |   |   | name@en     | text    |
  |   |   |   | languages   | array   | CountryLanguage
  |   |   |   | languages[] | ref     | Language
                            |         |
  |   |   | CountryLanguage |         |
  |   |   |   | country     | ref     | Country
  |   |   |   | language    | ref     | Language

Now intermediate table is no longer hidden and CountryLanguage is used instead. CountryLanguage must have two ref type properties pointing to both many-to-many ends.

If there are multiple rel properties pointing to same models and in many-to-many relation, then which properties to be used for many-to-many relation can be specified like this:

d | r | b | m | property    | type    | ref
example                     |         |
                            |         |
  |   |   | Language        |         |
  |   |   |   | name@en     | text    |
  |   |   |   | countries | backref | Country
                            |         |
  |   |   | Country         |         |
  |   |   |   | name@en     | text    |
  |   |   |   | languages   | array   | CountryLanguage[country, language]
  |   |   |   | languages[] | ref     | Language
                            |         |
  |   |   | CountryLanguage |         |
  |   |   |   | country     | ref     | Country
  |   |   |   | language    | ref     | Language

By default data from array properties are not show, for example:

http get /example/Country

Will return following result:

{
    "_data": [
        {
            "name": "Lithuania",
            "languages": []
        }
    ]
}

In order to also get data from array, you need to explicitly requrest data with expand() function, like this:

http get /example/Country?expand()

Then you will get this:

{
    "_data": [
        {
            "name": "Lithuania",
            "languages": [
                {"_id": "..."},
                {"_id": "..."}
            ]
        }
    ]
}

By default expand() will expand all array data, but use should be able to tell which properties to expand like this:

http get /example/Country?expand(languages)

For tabular formats, expanded array data are shown liek this:

_id  name        languages[]._id
---  ----------  ---------------
001  Lithuania   251...
001  Lithuania   252...

Data from main table are repeated for each item in array.

Tasks

• [x] Add array expressions to tabular manifests
• [x] Write data
• [x] Read data
• [x] expand()
• [x] expand(prop)
• [ ] https://github.com/atviriduomenys/spinta/issues/663
• [ ] https://github.com/atviriduomenys/spinta/issues/760
• [ ] https://github.com/atviriduomenys/spinta/issues/761

Related issues

• [x] https://github.com/atviriduomenys/spinta/issues/96
• [x] https://github.com/atviriduomenys/spinta/issues/651

[sirex]

Opening this task, because exposed intermediate tables are not yet implemented.

[JuliusLADP]

Reikia atskirti į atsikiras užduotįs jas detalizuojant

1. Exposed intermediate table
2. RDF support https://github.com/atviriduomenys/spinta/pull/356#pullrequestreview-1663892816
3. ir tikriausiai Push data

[ATEAanalyst]

Reikalingi detalesni užduoties aprašymai. Šiuo metu nėra aišku, ką norite, kad atliktume.

[JuliusLADP]

Kad Justinui viskas kaip ir aišku

[JustinasKen]

Reikia tas 3 dalis išskaidyti į atskiras užduotis ir detalizuoti kiekvieną, pateikiant kaip bus naudojama, kokie turės būti rezultatai, kaip buvo įsivaizduota kad veiks.

[sirex]

Sukurta atskira užduotis:

• https://github.com/atviriduomenys/spinta/issues/663

[JuliusLADP]

Iškelta į atskiras užuotis: Push data #760 RDF support #761