Question: How to deal with relationships?

[clemblanco]

Hi,

A bit stupid and random question: "How are we supposed to save our relationships within our models as ElasticSearch is not a relational database?"

What we used to so far with our own code was to serialise things into comma separated values an save this onto ElasticSearch. The downside of this was that we had to parse and transform comma separated values into proper relationships between model classes upon retrieval (and to it the other way around for saving). If there is a good way to deal with it, it would be really interesting to expose it from the readme.

Thanks!

[galexth]

Model::with(['relation1', 'relation2'])->get()->addToIndex()

If this is what you mean.

[clemblanco]

Mmokay thanks but how this is going to be stored on ElasticSearch? Is it automatically converted? What the data is going to look like?

[galexth]

Json and it will be dynamically mapped, as inner docs, unless you create an index first thru createIndex() method. Here is my example:

{
  "_index": "testindex",
  "_type": "contacts",
  "_id": "1",
  "_version": 1,
  "found": true,
  "_source": {
    "id": 1,
    "first_name": "Don",
    "last_name": "Mann",
    "image_url": "http://lorempixel.com/640/480/?64642",l,
    "created_at": "2016-06-16 10:47:28",
    "updated_at": "2016-06-16 10:47:28",
    "created_iso": "2016-06-16T10:47:28+0000",
    "updated_iso": "2016-06-16T10:47:28+0000",
    "full_name": "Don Mann",
    "main_email": null,
    "company": {
      "id": 3,
      "name": "Napoleon",
      "token": null,
      "created_at": "2016-06-16 10:47:27",
      "updated_at": "2016-06-16 10:47:27"
    },
    "emails": []
  }
}

[richdynamix]

I am trying to do this myself with very little success. I have created two new index commands with the following -

Tags command Tag::all()->addToIndex();

Posts command Post::with(['tags'])->get()->addToIndex();

The tags command completes successfully but the post command fails with the following

[InvalidArgumentException]
  Unexpected data found.

Looking in the logs it shows the following -

[2016-06-25 21:33:38] testing.ERROR: InvalidArgumentException: Unexpected data found. in /home/vagrant/techup/vendor/nesbot/carbon/src/Carbon/Carbon.php:425
Stack trace:
#0 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2969): Carbon\Carbon::createFromFormat('Y-m-d H:i:s', '-0001-11-30 00:...')
#1 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2462): Illuminate\Database\Eloquent\Model->asDateTime('-0001-11-30 00:...')
#2 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2439): Illuminate\Database\Eloquent\Model->attributesToArray()
#3 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2549): Illuminate\Database\Eloquent\Model->toArray()
#4 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2441): Illuminate\Database\Eloquent\Model->relationsToArray()
#5 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Support/Collection.php(1022): Illuminate\Database\Eloquent\Model->toArray()
#6 [internal function]: Illuminate\Support\Collection->Illuminate\Support\{closure}(Object(TechUp\Model\Tag))
#7 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Support/Collection.php(1023): array_map(Object(Closure), Array)
#8 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2549): Illuminate\Support\Collection->toArray()
#9 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2441): Illuminate\Database\Eloquent\Model->relationsToArray()
#10 /home/vagrant/techup/vendor/elasticquent/elasticquent/src/ElasticquentTrait.php(177): Illuminate\Database\Eloquent\Model->toArray()
#11 /home/vagrant/techup/vendor/elasticquent/elasticquent/src/ElasticquentCollectionTrait.php(37): TechUp\Model\Post->getIndexDocumentData()
#12 /home/vagrant/techup/app/Model/PostIndex.php(19): Elasticquent\ElasticquentCollection->addToIndex()
#13 /home/vagrant/techup/app/Console/Commands/Index/Posts.php(44): TechUp\Model\PostIndex->index()
#14 [internal function]: TechUp\Console\Commands\Index\Posts->handle()
#15 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Container/Container.php(507): call_user_func_array(Array, Array)
#16 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#17 /home/vagrant/techup/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /home/vagrant/techup/vendor/symfony/console/Application.php(791): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /home/vagrant/techup/vendor/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(TechUp\Console\Commands\Index\Posts), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /home/vagrant/techup/vendor/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /home/vagrant/techup/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(107): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /home/vagrant/techup/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 {main}  

The only way I have been able to add the the posts to the index is as follows -

$post = Post::with(['tags'])->first()->get()->addToIndex();

However when I query Elasticsearch I get the following -

{
  "_index" : "techup",
  "_type" : "posts",
  "_id" : "2",
  "_version" : 1,
  "found" : true,
  "_source" : {
    "id" : 2,
    "user_id" : 2,
    "post_type" : "projects",
    "title" : "Qui occaecati deleniti explicabo quibusdam optio.",
    "link" : "https://www.mertz.com/quasi-temporibus-voluptas-voluptate-itaque-laboriosam",
    "description" : "Ipsa asperiores sed et et suscipit. Eos inventore recusandae exercitationem beatae. Autem magnam voluptas suscipit atque exercitationem.",
    "slack_id" : "aliquam",
    "created_at" : "1991-12-29 02:29:49",
    "updated_at" : "1979-03-10 17:44:48",
    "exported" : 0
  }
}

Notice there is no relationships however if I dump($post->getRelations()); on the $post = Post::with(['tags'])->first()->get()->addToIndex(); I get the following -

 [Symfony\Component\Debug\Exception\FatalThrowableError]
  Call to a member function getRelations() on array

If I call $post = Post::with(['tags'])->first(); then dump($post->getRelations()); then I get -

array:1 [
  "tags" => Elasticquent\ElasticquentCollection {#625
    #items: array:6 [
      0 => TechUp\Model\Tag {#619
        #fillable: array:1 [
          0 => "name"
        ]
        #mappingProperties: array:1 [
          "name" => array:2 [
            "type" => "string"
            "analyzer" => "standard"
          ]
        ]
        #connection: null
        #table: null
        #primaryKey: "id"
        #keyType: "int"
        #perPage: 15
        +incrementing: true
        +timestamps: true
        #attributes: array:2 [
          "id" => 5
          "name" => "inventore"
        ]
        #original: array:6 [
          "id" => 5
          "name" => "inventore"
          "pivot_post_id" => 1
          "pivot_tag_id" => 5
          "pivot_created_at" => "0000-00-00 00:00:00"
          "pivot_updated_at" => "0000-00-00 00:00:00"
        ]
        #relations: array:1 [
          "pivot" => Illuminate\Database\Eloquent\Relations\Pivot {#618
            #parent: TechUp\Model\Post {#22
              #fillable: array:6 [
                0 => "title"
                1 => "link"
                2 => "description"
                3 => "slack_id"
                4 => "user_id"
                5 => "post_type"
              ]
              #mappingProperties: array:2 [
                "title" => array:2 [
                  "type" => "string"
                  "analyzer" => "standard"
                ]
                "description" => array:2 [
                  "type" => "string"
                  "analyzer" => "standard"
                ]
              ]
              #connection: null
              #table: null
              #primaryKey: "id"
              #keyType: "int"
              #perPage: 15
              +incrementing: true
              +timestamps: true
              #attributes: []
              #original: []
              #relations: []
              #hidden: []
              #visible: []
              #appends: []
              #guarded: array:1 [
                0 => "*"
              ]
              #dates: []
              #dateFormat: null
              #casts: []
              #touches: []
              #observables: []
              #with: []
              #morphClass: null
              +exists: false
              +wasRecentlyCreated: false
              #usesTimestampsInIndex: true
              #isDocument: false
              #documentScore: null
              #documentVersion: null
            }
            #foreignKey: "post_id"
            #otherKey: "tag_id"
            #guarded: []
            #connection: null
            #table: "post_tag"
            #primaryKey: "id"
            #keyType: "int"
            #perPage: 15
            +incrementing: true
            +timestamps: true
            #attributes: array:4 [
              "post_id" => 1
              "tag_id" => 5
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #original: array:4 [
              "post_id" => 1
              "tag_id" => 5
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #relations: []
            #hidden: []
            #visible: []
            #appends: []
            #fillable: []
            #dates: []
            #dateFormat: null
            #casts: []
            #touches: []
            #observables: []
            #with: []
            #morphClass: null
            +exists: true
            +wasRecentlyCreated: false
          }
        ]
        #hidden: []
        #visible: []
        #appends: []
        #guarded: array:1 [
          0 => "*"
        ]
        #dates: []
        #dateFormat: null
        #casts: []
        #touches: []
        #observables: []
        #with: []
        #morphClass: null
        +exists: true
        +wasRecentlyCreated: false
        #usesTimestampsInIndex: true
        #isDocument: false
        #documentScore: null
        #documentVersion: null
      }
      1 => TechUp\Model\Tag {#620
        #fillable: array:1 [
          0 => "name"
        ]
        #mappingProperties: array:1 [
          "name" => array:2 [
            "type" => "string"
            "analyzer" => "standard"
          ]
        ]
        #connection: null
        #table: null
        #primaryKey: "id"
        #keyType: "int"
        #perPage: 15
        +incrementing: true
        +timestamps: true
        #attributes: array:2 [
          "id" => 4
          "name" => "quod"
        ]
        #original: array:6 [
          "id" => 4
          "name" => "quod"
          "pivot_post_id" => 1
          "pivot_tag_id" => 4
          "pivot_created_at" => "0000-00-00 00:00:00"
          "pivot_updated_at" => "0000-00-00 00:00:00"
        ]
        #relations: array:1 [
          "pivot" => Illuminate\Database\Eloquent\Relations\Pivot {#617
            #parent: TechUp\Model\Post {#22}
            #foreignKey: "post_id"
            #otherKey: "tag_id"
            #guarded: []
            #connection: null
            #table: "post_tag"
            #primaryKey: "id"
            #keyType: "int"
            #perPage: 15
            +incrementing: true
            +timestamps: true
            #attributes: array:4 [
              "post_id" => 1
              "tag_id" => 4
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #original: array:4 [
              "post_id" => 1
              "tag_id" => 4
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #relations: []
            #hidden: []
            #visible: []
            #appends: []
            #fillable: []
            #dates: []
            #dateFormat: null
            #casts: []
            #touches: []
            #observables: []
            #with: []
            #morphClass: null
            +exists: true
            +wasRecentlyCreated: false
          }
        ]
        #hidden: []
        #visible: []
        #appends: []
        #guarded: array:1 [
          0 => "*"
        ]
        #dates: []
        #dateFormat: null
        #casts: []
        #touches: []
        #observables: []
        #with: []
        #morphClass: null
        +exists: true
        +wasRecentlyCreated: false
        #usesTimestampsInIndex: true
        #isDocument: false
        #documentScore: null
        #documentVersion: null
      }
      2 => TechUp\Model\Tag {#621
        #fillable: array:1 [
          0 => "name"
        ]
        #mappingProperties: array:1 [
          "name" => array:2 [
            "type" => "string"
            "analyzer" => "standard"
          ]
        ]
        #connection: null
        #table: null
        #primaryKey: "id"
        #keyType: "int"
        #perPage: 15
        +incrementing: true
        +timestamps: true
        #attributes: array:2 [
          "id" => 2
          "name" => "glasgow"
        ]
        #original: array:6 [
          "id" => 2
          "name" => "glasgow"
          "pivot_post_id" => 1
          "pivot_tag_id" => 2
          "pivot_created_at" => "0000-00-00 00:00:00"
          "pivot_updated_at" => "0000-00-00 00:00:00"
        ]
        #relations: array:1 [
          "pivot" => Illuminate\Database\Eloquent\Relations\Pivot {#616
            #parent: TechUp\Model\Post {#22}
            #foreignKey: "post_id"
            #otherKey: "tag_id"
            #guarded: []
            #connection: null
            #table: "post_tag"
            #primaryKey: "id"
            #keyType: "int"
            #perPage: 15
            +incrementing: true
            +timestamps: true
            #attributes: array:4 [
              "post_id" => 1
              "tag_id" => 2
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #original: array:4 [
              "post_id" => 1
              "tag_id" => 2
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #relations: []
            #hidden: []
            #visible: []
            #appends: []
            #fillable: []
            #dates: []
            #dateFormat: null
            #casts: []
            #touches: []
            #observables: []
            #with: []
            #morphClass: null
            +exists: true
            +wasRecentlyCreated: false
          }
        ]
        #hidden: []
        #visible: []
        #appends: []
        #guarded: array:1 [
          0 => "*"
        ]
        #dates: []
        #dateFormat: null
        #casts: []
        #touches: []
        #observables: []
        #with: []
        #morphClass: null
        +exists: true
        +wasRecentlyCreated: false
        #usesTimestampsInIndex: true
        #isDocument: false
        #documentScore: null
        #documentVersion: null
      }
      3 => TechUp\Model\Tag {#622
        #fillable: array:1 [
          0 => "name"
        ]
        #mappingProperties: array:1 [
          "name" => array:2 [
            "type" => "string"
            "analyzer" => "standard"
          ]
        ]
        #connection: null
        #table: null
        #primaryKey: "id"
        #keyType: "int"
        #perPage: 15
        +incrementing: true
        +timestamps: true
        #attributes: array:2 [
          "id" => 3
          "name" => "london"
        ]
        #original: array:6 [
          "id" => 3
          "name" => "london"
          "pivot_post_id" => 1
          "pivot_tag_id" => 3
          "pivot_created_at" => "0000-00-00 00:00:00"
          "pivot_updated_at" => "0000-00-00 00:00:00"
        ]
        #relations: array:1 [
          "pivot" => Illuminate\Database\Eloquent\Relations\Pivot {#615
            #parent: TechUp\Model\Post {#22}
            #foreignKey: "post_id"
            #otherKey: "tag_id"
            #guarded: []
            #connection: null
            #table: "post_tag"
            #primaryKey: "id"
            #keyType: "int"
            #perPage: 15
            +incrementing: true
            +timestamps: true
            #attributes: array:4 [
              "post_id" => 1
              "tag_id" => 3
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #original: array:4 [
              "post_id" => 1
              "tag_id" => 3
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #relations: []
            #hidden: []
            #visible: []
            #appends: []
            #fillable: []
            #dates: []
            #dateFormat: null
            #casts: []
            #touches: []
            #observables: []
            #with: []
            #morphClass: null
            +exists: true
            +wasRecentlyCreated: false
          }
        ]
        #hidden: []
        #visible: []
        #appends: []
        #guarded: array:1 [
          0 => "*"
        ]
        #dates: []
        #dateFormat: null
        #casts: []
        #touches: []
        #observables: []
        #with: []
        #morphClass: null
        +exists: true
        +wasRecentlyCreated: false
        #usesTimestampsInIndex: true
        #isDocument: false
        #documentScore: null
        #documentVersion: null
      }
      4 => TechUp\Model\Tag {#623
        #fillable: array:1 [
          0 => "name"
        ]
        #mappingProperties: array:1 [
          "name" => array:2 [
            "type" => "string"
            "analyzer" => "standard"
          ]
        ]
        #connection: null
        #table: null
        #primaryKey: "id"
        #keyType: "int"
        #perPage: 15
        +incrementing: true
        +timestamps: true
        #attributes: array:2 [
          "id" => 6
          "name" => "quidem"
        ]
        #original: array:6 [
          "id" => 6
          "name" => "quidem"
          "pivot_post_id" => 1
          "pivot_tag_id" => 6
          "pivot_created_at" => "0000-00-00 00:00:00"
          "pivot_updated_at" => "0000-00-00 00:00:00"
        ]
        #relations: array:1 [
          "pivot" => Illuminate\Database\Eloquent\Relations\Pivot {#614
            #parent: TechUp\Model\Post {#22}
            #foreignKey: "post_id"
            #otherKey: "tag_id"
            #guarded: []
            #connection: null
            #table: "post_tag"
            #primaryKey: "id"
            #keyType: "int"
            #perPage: 15
            +incrementing: true
            +timestamps: true
            #attributes: array:4 [
              "post_id" => 1
              "tag_id" => 6
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #original: array:4 [
              "post_id" => 1
              "tag_id" => 6
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #relations: []
            #hidden: []
            #visible: []
            #appends: []
            #fillable: []
            #dates: []
            #dateFormat: null
            #casts: []
            #touches: []
            #observables: []
            #with: []
            #morphClass: null
            +exists: true
            +wasRecentlyCreated: false
          }
        ]
        #hidden: []
        #visible: []
        #appends: []
        #guarded: array:1 [
          0 => "*"
        ]
        #dates: []
        #dateFormat: null
        #casts: []
        #touches: []
        #observables: []
        #with: []
        #morphClass: null
        +exists: true
        +wasRecentlyCreated: false
        #usesTimestampsInIndex: true
        #isDocument: false
        #documentScore: null
        #documentVersion: null
      }
      5 => TechUp\Model\Tag {#624
        #fillable: array:1 [
          0 => "name"
        ]
        #mappingProperties: array:1 [
          "name" => array:2 [
            "type" => "string"
            "analyzer" => "standard"
          ]
        ]
        #connection: null
        #table: null
        #primaryKey: "id"
        #keyType: "int"
        #perPage: 15
        +incrementing: true
        +timestamps: true
        #attributes: array:2 [
          "id" => 1
          "name" => "paris"
        ]
        #original: array:6 [
          "id" => 1
          "name" => "paris"
          "pivot_post_id" => 1
          "pivot_tag_id" => 1
          "pivot_created_at" => "0000-00-00 00:00:00"
          "pivot_updated_at" => "0000-00-00 00:00:00"
        ]
        #relations: array:1 [
          "pivot" => Illuminate\Database\Eloquent\Relations\Pivot {#613
            #parent: TechUp\Model\Post {#22}
            #foreignKey: "post_id"
            #otherKey: "tag_id"
            #guarded: []
            #connection: null
            #table: "post_tag"
            #primaryKey: "id"
            #keyType: "int"
            #perPage: 15
            +incrementing: true
            +timestamps: true
            #attributes: array:4 [
              "post_id" => 1
              "tag_id" => 1
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #original: array:4 [
              "post_id" => 1
              "tag_id" => 1
              "created_at" => "-0001-11-30 00:00:00"
              "updated_at" => "-0001-11-30 00:00:00"
            ]
            #relations: []
            #hidden: []
            #visible: []
            #appends: []
            #fillable: []
            #dates: []
            #dateFormat: null
            #casts: []
            #touches: []
            #observables: []
            #with: []
            #morphClass: null
            +exists: true
            +wasRecentlyCreated: false
          }
        ]
        #hidden: []
        #visible: []
        #appends: []
        #guarded: array:1 [
          0 => "*"
        ]
        #dates: []
        #dateFormat: null
        #casts: []
        #touches: []
        #observables: []
        #with: []
        #morphClass: null
        +exists: true
        +wasRecentlyCreated: false
        #usesTimestampsInIndex: true
        #isDocument: false
        #documentScore: null
        #documentVersion: null
      }
    ]
  }
]

dump($post->getAttributes()); shows the following-

array:10 [
  "id" => 1
  "user_id" => 1
  "post_type" => "tech"
  "title" => "This is the test post for techup"
  "link" => "http://www.ruecker.com/consectetur-aut-omnis-consequatur-doloribus-molestias"
  "description" => "Optio nemo error cupiditate sed similique id. Enim qui ex qui quis ipsam recusandae facilis quia. Et ipsa vero necessitatibus occaecati molestias eaque. Ratione placeat exercitationem commodi sequi vel ut porro."
  "slack_id" => "esse"
  "created_at" => "2016-06-12 01:25:25"
  "updated_at" => "2003-11-13 14:44:05"
  "exported" => 0
]

Kind of lost at this point, does anyone have any idea whats going wrong?