LibreCat / Catmandu

Catmandu - a data processing toolkit
https://librecat.org
176 stars 31 forks source link

Option "bag" is not loaded from config (catmandu.yml) #116

Closed jorol closed 9 years ago

jorol commented 9 years ago

My config:


---
store:
  mdb:
   package: MongoDB
   options:
    database_name: test
    bag: test
  els:
   package: Elasticsearch
   options:
    index_name: test
    bag: test

My cmd:

echo {"test":"test"} | catmandu import JSON to mdb
echo {"test":"test"} | catmandu import JSON to els

Error:

MongoDB: Data is loaded in default bag/collection "data" Elasticsearch: "_type" is set to default "data"

vpeil commented 9 years ago

I think it is bags instead of bag.

jorol commented 9 years ago

bags doesn't work:

$ catmandu config
{
   "store" : {
      "mdb" : {
         "options" : {
            "bags" : {
               "collection" : {
                  "plugins" : []
               }
            },
            "database_name" : "store"
         },
         "package" : "MongoDB"
      }
   }
}
$ echo '{"key":"value"}' | catmandu import -v JSON to mdb
imported 1 object
done
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use store
switched to db store
> show collections
data
system.indexes
> 

default_bag seems to do it:

$ catmandu config
{
   "store" : {
      "mdb" : {
         "package" : "MongoDB",
         "options" : {
            "database_name" : "store",
            "default_bag" : "collection"
         }
      }
   }
}
$ echo '{"key":"value"}' | catmandu import -v JSON to mdb
imported 1 object
done
$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> use store
switched to db store
> show collections
collection
system.indexes
> db.collection.find()
{ "_id" : "47302544-6F10-11E4-9F82-1C0302EA08D7", "key" : "value" }
nics commented 9 years ago

default_bag is the way to go here, bag isn't reallly an attribute of the store it's a bit confusing that --bag appears between the other attributes on the line: catmandu export MongoDB --bag mybag --database-name ... better would probably have been: catmandu export MongoDB mybag --database-name ...