fluent / fluent-plugin-mongo

MongoDB input and output plugin for Fluentd
https://docs.fluentd.org/output/mongo
173 stars 61 forks source link

Restore creation of capped collections #91

Closed aferreira closed 6 years ago

aferreira commented 7 years ago

In order to start collections with specific options (such as capped collections), it is necessary to create collections explicitly.

The expression

@client[collection, @collection_options].insert_many(records)

implicitly creates the collection without applying @collection_options. This is the cause of issue #90.

This change checks to see if the collection to be written to already exists in the database – if yes, it just remembers it. If the collection does not exist, it is created according to the configured plugin options. A new @collections attribute helps to keep track of the collections already seen.

aferreira commented 7 years ago

Now I see the reason of the failure in Travis – the MongoDB "listCollections" command is only available for 3.0.0+ – https://docs.mongodb.com/manual/reference/command/listCollections/ Probably Travis is using an older MongoDB version.

The benefit of "listCollections" is to be able to query about the existence of a collection without an expensive operation like listing all collections of a database (which can be prohibitive if there are hundreds or thousands of collections).

That said, I think I can work out something that checks MongoDB server version and falls back into a compatible solution.

aferreira commented 7 years ago

Any comments or objections? Does anybody else need this bug fix at all?

cosmo0920 commented 6 years ago

:+1: for this.

cosmo0920 commented 6 years ago

I registered separatable additional work to restore capped collection feature.

repeatedly commented 6 years ago

Sorry for the late. Just merged.