neumino / thinky

JavaScript ORM for RethinkDB
http://justonepixel.com/thinky/
Other
1.12k stars 128 forks source link

Insert only Id on relationships? #636

Closed Extarys closed 6 years ago

Extarys commented 6 years ago

Is there a way to insert the document Id instead of the whole document on relations? I have huge documents and I'm afraid it'll get too slow in the long run to have everything under the parent. :\

Simplistic representation
{
   user: "someone",
   posts: ["00001","00002","00003"],
}
grantcarthew commented 6 years ago

Hi again @Extarys. That is the whole idea of relationships. Read about it here: http://justonepixel.com/thinky/documentation/relations/

Extarys commented 6 years ago

Hey @grantcarthew ! I like you, you are a very resourceful person ;) Maybe an example will demonstrate my fear. I'll have 3 tables: company, brands,products,ingredients. I will also have some tables for ratings, company ratings, ingredients ratings, modifications, voting about accuracy of information, etc. So with all childs, here's what 1 entry of comapny would look like with 1 entry of each.

{
  "comapny": "Unilever",
  "year": 1990,
  "categories": [
    "Food",
    "Beverages",
    "Cleaning agents",
    "Personal care",
  ],
  "brands": {
    {
      "id": "0000-0000-0000-0000",
      "company": "unilever",
      "brand": "Lynx",
      "aka": {
        "na": "Axe",
      },
      "products": {
        {
          "id": "0000-0000-0000-0000",
          "company": "unilever",
          "brand": "Axe",
          "parent_category": "Personal care",
          "category": "Shower gel",
          "product": "Axe Dark Temptation",
          "qte": "473",
          "qte_unit": "ml",
          "ean": "011111241992",
          "manuf": {
            "manufactured_by": "Unilever Canada",
            "zip": "M4W 3R2"
          }'
          "Ingredients": [
            {
              "id": "0000-0000-0000-0000",
              "name": "Aqua",
              "aka": [{
                  "fr": "Eau"
              }],
              "ratings": {
                "toxicity": 0,
                "isChemical": false,
                "source": null
              }
            }
          ]
        }
      }
    }
  },
}

For an exhaustive list of Unilever brands look here.

A lot and a lot of entry. IfI need to replicate everything, 1: it will get huge, second: retrieving and saving documents will result in a lot of disk IO. :\

Also, (more of a rethinkdb question) any idea if it get compressed on disk? Can we choose the type of compression?

Or maybe I didn't understand exactly how this works? I am also used to MySQL, so it might me how document-based storage is supposed to be.

I also haven't figure out how to store categories.

It's like a book having multiple authors and tags. Data, data everywhere

grantcarthew commented 6 years ago

Hey @Extarys, I still think this is about relationships as I linked above. Are you sure a document database is what you need? A rigid database schema like you find in a SQL database sounds like a good fit here. I don't know about how RethinkDB stores the content, compressed or not?

Extarys commented 6 years ago

Not compressed yet I think. But since my data is highly unpredictable I think I would prefer document-based. Products may or may not contain ingredients, a sub-package type (like individually warped product ie: yogurt), or allergens for example. At the end it would be a better fit. I may ditch thinky for this and implement this as a relational db in the backend and just save the IDs as reference for some documents.

Very nice of you to follow-up ;)

I'll close this as this is more a discussion.