Support for TEXT ARRAY

[adborden]

Here's my collections.yml snippet:

  products:
    :columns:
    - id:
      :source: _id
      :type: TEXT
    - foodshed: TEXT
    - name: TEXT
    - price: INTEGER
    - tags: TEXT ARRAY
    :meta:
      :table: products

But I'm getting an error:

LINE 1: ...ini Pumpkin Spice Trio', "price" = 6.49, "tags" = '["Cookies...
PG::InvalidTextRepresentation: ERROR:  missing dimension value (Sequel::DatabaseError)

Full stack trace is here https://gist.github.com/adborden/de92b19a2bbe1feebbdc

Is this expected?

[nelhage]

Your input document has

"tags"=>"[]"

i.e. it looks like the value is the stringified form of a JSON array. MoSQL expects ARRAY columns to be actual BSON arrays in the document, not JSON arrays. So the conversion is getting confused (and unfortunately we're giving a terrible error message).

[adborden]

Thanks, @nelhage. Is there a way that I can confirm this? As far as I can tell, this field /is/ an Array, though not sure how to tell BSON from JSON.

> db.products.find({_id: ObjectId('4f3aee0fd2f645040000000c')}, {tags: true}).map(function (p) { return  p.tags instanceof Array; });
[ true ]

[nelhage]

4fa0c49bc43e9e0700000085 is the record in your pasted logs I was looking at -- what do you get if you try the same check on that _id?

[adborden]

> db.products.find({_id: ObjectId('4fa0c49bc43e9e0700000085')}, {tags: true}).map(function (p) { return  p.tags instanceof Array; });
[ true ]

> db.products.find({_id: ObjectId('4fa0c49bc43e9e0700000085')}, {tags: true}).map(function (p) { return  p._id instanceof Array; });
[ false ]

FWIW, the above log is from mosql 0.3.1 and would crash mosql. I've upgraded to 0.4.1, it no longer crashes, imports most rows, but still prints a similar error -- only with verbose -v logging.