IDs are not necessarily unique

[wglasshusain]

Hi,

I've got a web service (written in Lusty, using lusty-store-mongo) which creates records in MongoDb.

When i call the web service repeatedly to create records for the same collection and return the id from MongoDb, I see that the object id sometimes repeats.

I wrote a test in Java to rapidly call my web service to repeatedly create records and output the id. Out of every 100 queries, I get 2-10 that have dup ids.

Example (first 10 records created). You can see the 6th id is the same as the first.

13:52:04.138 ID 543d889b636239a227000001
13:52:04.281 ID 543d889b636239a127000001
13:52:04.420 ID 543d889b6362399f27000001
13:52:04.561 ID 543d889b6362399f27000001
13:52:04.701 ID 543d889b6362399f27000001
13:52:04.844 ID 543d889b636239a227000001  - CONFLICTING ID: 543d889b636239a227000001
13:52:04.988 ID 543d889b636239a127000001
13:52:05.127 ID 543d889c636239a127000001
13:52:05.277 ID 543d889c636239a127000001
13:52:05.422 ID 543d889c636239a027000001

The web service itself is simple. It does the following

local allowableFields = {"account",  "project",  "group",  "name",  "roles",  "minUsers", "optionalRoles"}
local record = {}
for _, v in pairs(allowableFields) do
  if inputItem[v] then
    record[v] = inputItem[v]
  end
end

record.created = date(true):fmt('${iso}.000Z')
record.lastModified = date(true):fmt('${iso}.000Z')

store.post(record)

store is a Lusty store object. After this bit of code, it returns record._id.

Looking at the Mongol code, I see that the id includes a timestamp and also a incrementing number. I hypothesize that the number is not correctly incrementing, and that creating a bunch of records in a short time can generate the same id. Does this seem plausible?

It's possible that the issue is also in lusty-store-mongo. I am currently attempting to duplicate this in a test which calls Mongol Resty driver directly, but my Lua skills are a bit rusty and I haven't pulled together the Busted test yet.

[DorianGray]

I pushed a new version to address this (0.8-2).

The problem was that the resty-mongol driver was getting the process id from the master nginx process instead of the process id from the worker processes.

Try it out!

[DorianGray]

@wglasshusain Did you get a chance to see if this takes care of your issue?

[wglasshusain]

Not yet -- will try it out in the next few days.

Thanks!

WGH

On Mon, Oct 20, 2014 at 3:29 PM, Robert Andrew Ditthardt < notifications@github.com> wrote:

@wglasshusain https://github.com/wglasshusain Did you get a chance to see if this takes care of your issue?

— Reply to this email directly or view it on GitHub https://github.com/Olivine-Labs/resty-mongol/issues/2#issuecomment-59850485 .

William Glass-Husain /forio | +1 (415) 440 7500 x89 | forio.com http://www.forio.com/

[wglasshusain]

Sadly, my test code still shows this error. I confirmed my vagrant box has only resty-mongol 0.8-2 installed. Rapidly generating 100 records still generates some records with conflicting ids.