Elastic search failing to index one of two collection

[jfrancarioInera]

@dimitrov-adrian Hi sorry for bothering you again. Reinstalled the extension following the new readme, but i get a strange behavior. Changed the host to http://localhost:9200/directus but only one collection gets saved in elasticsearch index.

directus-extension-searchsync Cannot update viaggi/1. Error: Request failed with status code 400
directus-extension-searchsync Cannot update viaggi/2. Error: Request failed with status code 400
directus-extension-searchsync Cannot update viaggi/3. Error: Request failed with status code 400
directus-extension-searchsync Cannot update viaggi/4. Error: Request failed with status code 400
directus-extension-searchsync Cannot update viaggi/5. Error: Request failed with status code 400
directus-extension-searchsync Cannot update viaggi/6. Error: Request failed with status code 400

This is what I get if I query elasticsearch: GET http://localhost:9200/directus/_search

Answer

``` { "took": 37, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 6, "relation": "eq" }, "max_score": 1.0, "hits": [ { "_index": "directus", "_type": "tappe", "_id": "1", "_score": 1.0, "_source": { "nome": "Cavalieri", "tipo": "piazza" } }, { "_index": "directus", "_type": "tappe", "_id": "2", "_score": 1.0, "_source": { "nome": "Massaciuccoli", "tipo": "lago" } }, { "_index": "directus", "_type": "tappe", "_id": "3", "_score": 1.0, "_source": { "nome": "Villa Flora", "tipo": "piazza" } }, { "_index": "directus", "_type": "tappe", "_id": "4", "_score": 1.0, "_source": { "nome": "test1", "tipo": "Monumento" } }, { "_index": "directus", "_type": "tappe", "_id": "5", "_score": 1.0, "_source": { "nome": "fewad", "tipo": "Monumento" } }, { "_index": "directus", "_type": "tappe", "_id": "6", "_score": 1.0, "_source": { "nome": "tappaprova", "tipo": "Parco" } } ] } } ```

Only the collection 'tappe' gets saved. The collection 'viaggi' has a one to many relationship with 'tappe'. Anyway, it happened the opposite aswell, the 'viaggi' collection got saved, but not 'tappe.

the configuration file is the same except the host.

Configuration

```javascript module.exports = { server: { type: "elasticsearch", host: "http://localhost:9200/directus", }, reindexOnStart: true, collections: { viaggi: { filter: { }, fields: [ "nome", "durata_h", "difficolta", ], transform: formatter, }, tappe: { filter: { }, fields: ["nome", "tipo"], transform: formatter, }, }, }; function formatter(value, { flattenObject }) { value = flattenObject(value); return value; } ```

Originally posted by @jfrancarioInera in https://github.com/dimitrov-adrian/directus-extension-searchsync/issues/7#issuecomment-886523293

[dimitrov-adrian]

I cannot reproduce with my setup. Could you provide sample dump with those 2 collections.

[jfrancarioInera]

dump.zip is sql format alright?

[dimitrov-adrian]

Should works, thanks

[jfrancarioInera]

@dimitrov-adrian After some research I may have found the problem. Since elasticsearch 6.0 you can't have more than one _type for index, that's why I couldn't index more than one collection. I changed the updateItem function of the elasticsearch indexer so it would create one index per collection, using one type "item" for index.

    async function updateItem(collection, id, data) {
        return await axios.post(
            `${config.host}-${collection}/item/${id}`,
            data,
            axiosConfig
        );
    }
};

let me know if it checks out

[dimitrov-adrian]

@jfrancarioInera yes, you are right, from 7.0 types are removed. It seems that this will happen also for 6.8. So I think will be good to move the current behaviour in elasticsearch_legacy as a type and use typeless in elasticsearch type

What I am thinking is how to persist the type actually. It seems there is a couple of options like:

1. To use it as a index and having query like <server>/<collection>/_doc/<id>
2. Use in doc id like <server>/<index_from_config>/_doc/<collection>-<id> and add as a custom property in the document

[dimitrov-adrian]

Fixed in v1.0.0-rc.88

[keniobats]

@dimitrov-adrian I'm having this issue with your plugin v1.0.3. My setup is Directus 9.23.1 and ELK 8.4.2.

Is there a way of solving this? Thanks!