mongo indexes

[brianlball]

when the /docker/server/start-server.sh script executes, the following error shows up in the docker.log:

** Execute db:mongoid:create_indexes rake aborted! Mongo::Error::OperationFailure: [197:InvalidIndexSpecificationOption]: The field 'background' is not valid for an _id index specification. Specification: { key: { _id: 1 }, background: false, name: "_id_1", v: 2 } (on db:27017)

This is due to creating a index on id, when there is already one on _id in the models.

we also see the following in the docker.log

Mongo::Error::OperationFailure ([11000]: E11000 duplicate key error collection index: uuid_1 dup key: { uuid: null } legacy retry, attempt 1

This indicates that datapoints are getting initially created with a blank uuid. This might be because of the recent version updates for mongo and mongoid, but should be investigated and fixed.

[brianlball]

A successful start with creating indexes should have a docker.log that looks like the following below. A grep for Created indexes will indicate success, as this does not show up in the log if the errors above are found by >docker logs xxx

[17:36:15.918689 INFO] MONGOID: Created indexes on Analysis: [17:36:15.918733 INFO] MONGOID: Index: {:uuid=>1}, Options: {:unique=>true} [17:36:15.918743 INFO] MONGOID: Index: {:name=>1}, Options: {} [17:36:15.918751 INFO] MONGOID: Index: {:created_at=>1}, Options: {} [17:36:15.918758 INFO] MONGOID: Index: {:updated_at=>-1}, Options: {} [17:36:15.918764 INFO] MONGOID: Index: {:project_id=>1}, Options: {} [17:36:15.946597 INFO] MONGOID: Created indexes on Delayed::Backend::Mongoid::Job: [17:36:15.946639 INFO] MONGOID: Index: {:locked_by=>-1, :priority=>1, :run_at=>1}, Options: {} [17:36:16.001699 INFO] MONGOID: Created indexes on ComputeNode: [17:36:16.001748 INFO] MONGOID: Index: {:hostname=>1}, Options: {} [17:36:16.001758 INFO] MONGOID: Index: {:ip_address=>1}, Options: {} [17:36:16.001765 INFO] MONGOID: Index: {:node_type=>1}, Options: {} [17:36:16.001772 INFO] MONGOID: Index: {:name=>1, :hostname=>1}, Options: {:unique=>true} [17:36:16.136308 INFO] MONGOID: Created indexes on DataPoint: [17:36:16.136346 INFO] MONGOID: Index: {:analysis_id=>1}, Options: {:background=>true} [17:36:16.136356 INFO] MONGOID: Index: {:uuid=>1}, Options: {:unique=>true} [17:36:16.136363 INFO] MONGOID: Index: {:name=>1}, Options: {} [17:36:16.136370 INFO] MONGOID: Index: {:status=>1}, Options: {} [17:36:16.136377 INFO] MONGOID: Index: {:analysis_id=>1, :created_at=>1}, Options: {} [17:36:16.136383 INFO] MONGOID: Index: {:created_at=>1}, Options: {} [17:36:16.136390 INFO] MONGOID: Index: {:uuid=>1, :status=>1}, Options: {} [17:36:16.136398 INFO] MONGOID: Index: {:analysis_id=>1, :status=>1, :ip_address=>1}, Options: {} [17:36:16.136404 INFO] MONGOID: Index: {:run_start_time=>-1, :name=>1}, Options: {} [17:36:16.136410 INFO] MONGOID: Index: {:run_end_time=>-1, :name=>1}, Options: {} [17:36:16.136418 INFO] MONGOID: Index: {:analysis_id=>1, :iteration=>1, :sample=>1}, Options: {} [17:36:16.136426 INFO] MONGOID: Index: {:analysis_id=>1, :status=>1, :status_message=>1, :created_at=>1}, Options: {} [17:36:16.182594 INFO] MONGOID: Created indexes on Job: [17:36:16.182629 INFO] MONGOID: Index: {:created_at=>1}, Options: {} [17:36:16.182637 INFO] MONGOID: Index: {:analysis_id=>1}, Options: {} [17:36:16.182645 INFO] MONGOID: Index: {:analysis_id=>1, :index=>1, :analysis_type=>1}, Options: {} [17:36:16.248248 INFO] MONGOID: Created indexes on Measure: [17:36:16.248290 INFO] MONGOID: Index: {:uuid=>1}, Options: {:unique=>true} [17:36:16.248299 INFO] MONGOID: Index: {:name=>1}, Options: {} [17:36:16.248307 INFO] MONGOID: Index: {:analysis_id=>1}, Options: {} [17:36:16.248313 INFO] MONGOID: Index: {:analysis_id=>1, :uuid=>1}, Options: {} [17:36:16.248318 INFO] MONGOID: Index: {:analysis_id=>1, :name=>1}, Options: {:unique=>true} [17:36:16.286318 INFO] MONGOID: Created indexes on Pareto: [17:36:16.286355 INFO] MONGOID: Index: {:uuid=>1}, Options: {:unique=>true} [17:36:16.286364 INFO] MONGOID: Index: {:analysis_id=>1}, Options: {} [17:36:16.323327 INFO] MONGOID: Created indexes on Project: [17:36:16.323369 INFO] MONGOID: Index: {:uuid=>1}, Options: {:unique=>true} [17:36:16.323378 INFO] MONGOID: Index: {:name=>1}, Options: {} [17:36:16.407949 INFO] MONGOID: Created indexes on Variable: [17:36:16.407990 INFO] MONGOID: Index: {:analysis_id=>1}, Options: {:background=>true} [17:36:16.408001 INFO] MONGOID: Index: {:uuid=>1}, Options: {:unique=>true} [17:36:16.408007 INFO] MONGOID: Index: {:name=>1}, Options: {} [17:36:16.408014 INFO] MONGOID: Index: {:r_index=>1}, Options: {} [17:36:16.408022 INFO] MONGOID: Index: {:analysis_id=>1, :uuid=>1}, Options: {} [17:36:16.408034 INFO] MONGOID: Index: {:analysis_id=>1, :perturbable=>1}, Options: {} [17:36:16.408040 INFO] MONGOID: Index: {:analysis_id=>1, :name=>1}, Options: {}

[brianlball]

the warnings in the docker.log about overwriting _id

[17:36:15.783999 WARN] Overwriting existing field _id in class Analysis. [17:36:15.799098 WARN] Overwriting existing field _id in class DataPoint.

are fine since we are changing the models _id field to a string in all the /app/models/xxx.rb

[brianlball]

this test will look for the successful creation of indexes in the debug_log, which should be shown like in the previous comment.

The other lines just look for FATAL and OperationFailure, although OperationFailure is intermittent in the docker.log, but can show up in the >docker logs xxx

[brianlball]

you can also verify that the indexes are made by running bundle exec rake db:mongoid:create_indexes in the web container

root@8311dda6c546:/opt/openstudio/server# bundle exec rake db:mongoid:create_indexes Top level ::CompositeIO is deprecated, require 'multipart/post' and useMultipart::Post::CompositeReadIOinstead! Top level ::Parts is deprecated, require 'multipart/post' and useMultipart::Post::Partsinstead! root@8311dda6c546:/opt/openstudio/server# bundle exec rake db:mongoid:create_indexes --trace Top level ::CompositeIO is deprecated, require 'multipart/post' and useMultipart::Post::CompositeReadIOinstead! Top level ::Parts is deprecated, require 'multipart/post' and useMultipart::Post::Partsinstead! ** Invoke db:mongoid:create_indexes (first_time) ** Invoke environment (first_time) ** Execute environment ** Invoke db:mongoid:load_models (first_time) ** Execute db:mongoid:load_models ** Execute db:mongoid:create_indexes root@8311dda6c546:/opt/openstudio/server# exit