GSoC Idea: Implement the Social Currency Metrics System in GrimoireLabs

[GeorgLink]

Idea: Implement the Social Currencyy Metrics System in Grimoire Labs

The Social Currency Metrics System (SCMS) is a qualitative data collection, processing, and measurement system that augments quantitative community management metrics already available in CHAOSS properties. Implementing the SCMS will ultimately help community leaders, power users, and other stakeholders leverage qualitative data for social listening so that they can rely less on the behaviors quantitative data tracks and more on community sentiment. The SCMS empowers community leaders to make decisions based on what community members freely share about their opinions, wants, and needs.

The SCMS shows why trends occur and identifies commonly missed pitfalls in conclusions taken from quantitative data. With an SCMS platform built natively into a CHAOSS Bitergia property, open-source communities can use it to facilitate members’ input in decisions essential to community health.

The purpose of this project is to: Build the SCMS in one of CHAOSS project’s systems that collects qualitative data from several channels, displays that information for manipulation and tagging, and outputs it to a metric dashboard such as the SCMS information page found here.

The aims of this project are to:

• Create an API that collects and returns data collected from a community of the GSoC participants’ choice.
• Gain familiarity with and create a way to grade, tag, and process large amounts of data.
• Develop creative ways to display complex datasets in the Bitergia Analytics System.
• Investigate ways to process qualitative data at scale using AI or similar technology.
• Encourage the larger development of social listening tools based on qualitative data.

Difficulty: Medium to Hard based on the level of implementation (machine learning).

Requirements: Python programming. Interest in machine learning. API Development some understanding of the social scientific process and qualitative data analysis

Recommended: Other API languages, interest in science of community management and anthropological studies of online worlds, willingness to learn CHAOSS Property Structures

Mentors: Dylan Marcy (SociallyConstructed.Online), Venia Logan (SociallyConstructed.Online), Valereo (Grimoire Labs), Georg (Advising only)

Micro-tasks

For becoming familiar with GrimoireLab, you can start by reading some documentation. You can find useful information at:

• GrimoireLab Tutorial
• Perceval: Software Project Data at Your Will
• The GitHub repositories hosting the tools (https://github.com/chaoss/grimoirelab-perceval, https://github.com/chaoss/grimoirelab-sirmordred, https://github.com/chaoss/grimoirelab-...) and https://github.com/Bitergia/archimedes.

Once you're familiar with Grimoirelab, you can have a look at the following microtasks.

• Microtask 0: Download PyCharm and get familiar with it (for instance, you can follow this tutorial).

• Microtask 1: Set up Perceval to be executed from PyCharm.

• Microtask 2: Create a Python script to execute Perceval via its Python interface using the GitLab and GitHub backends. Feel free to select any target repository.

• Microtask 3: Based on the JSON documents produced by Perceval and its source code, try to answer the following questions:

  • What is the meaning of the JSON attribute 'timestamp'?
  • What is the meaning of the JSON attribute 'updated_on'?
  • What is the meaning of the JSON attribute 'origin'?
  • What is the meaning of the JSON attribute 'category'?
  • How many categories do the GitHub and GitLab backends have?
  • What is the meaning of the JSON attribute 'uuid'?
  • What is the meaning of the JSON attribute search_fields?
  • What is stored in the attribute data of each JSON document produced by Perceval?

• Microtask 4: Set up a dev environment to work on GrimoireLab. Have a look to https://github.com/chaoss/grimoirelab-sirmordred#setting-up-a-pycharm-dev-environment.

• Microtask 5: Execute micro-mordred to collect, enrich and visualize data from Git and GitHub repositories.

• Microtask 6: Using the dev tools in Kibiter, create a query that counts the number of unique authors on a Git repository from 2018-01-01 until 2019-01-01.

• Microtask 7: Install and use elasticdump to download the mapping and data of an ElasticSearch index (it can be anyone created in Microtask 5).

• Microtask 8: Execute micro-mordred to collect and enrich data from a groupsio repository. You need to register to a group (e.g., https://lists.onap.org/g/main) and follow the instructions at https://github.com/chaoss/grimoirelab-sirmordred#groupsio. Then, write a script to read the enriched index and import the attributes uuid, project, project_1, origin, grimoirelab_creation_date, body and subject_analyzed to a CSV file. Import the obtained file to an excel sheet (in a manual or automatic way).

• Microtask 9: Build a Data Table visualization in Kibiter (you can use the [CHAOSS community dashboard](https://chaoss.biterg.io/app/kibana#/visualize/new?_g=())) that shows for emails (mbox index) the text of emails (split row by Termbody_extract field).

• Microtask 10: Submit at least a PR to one of the GrimoireLab repositories to fix an issue, improve the documentation, etc.

[abhiandthetruth]

ok!

[snack0verflow]

@valeriocos @abhiandthetruth I think I have removed the bots_names param in an earlier PR

[snack0verflow]

https://github.com/chaoss/grimoirelab-sirmordred/pull/406 this one

[abhiandthetruth]

@snack0verflow it was for the setup.cfg in the grimoirelab-sirmordred repo this one is for the tutorials!

[snack0verflow]

@abhiandthetruth my bad, you can go ahead and fix that.

[abhiandthetruth]

@valeriocos finished all the micro-tasks!. I have made a private repo for that. It would be really helpful if you could review my work before submitting. So how should I proceed, shall I make it public and share the link? Thanks :). Also there is no Microtask-8 there in the list provided.

[valeriocos]

@abhiandthetruth I'm going to add a task today (sorry for the inconvenience). WRT sharing your work, you can either invite me to your private repo or you can make it public and share the link. Choose what you prefer :)

[abhiandthetruth]

@valeriocos I have made it public for now :). Here is the link https://github.com/abhiandthetruth/Chaoss-Gsoc2020. Please review. Please tell if more detailed answers are needed.

[valeriocos]

https://github.com/chaoss/grimoirelab/issues/288#issuecomment-596364832

Task added! Please have a look, thanks

[singhaltanvi]

Hi!! I am Tanvi Singhal an outreachy applicant interested in working on this project

[valeriocos]

Hi @singhaltanvi , welcome on board! Don't hesitate to write here if you need help/feedback, thanks!

[animeshk08]

Hello mentors, I have a few questions regarding the implementation of Social Currency Metric System :)

• Is there going to be a feature to add these social currencies manually? In that case who can do so? Also how often they are to be added to generate a good reflective result?

• How does one decide the weight score for these metrics?

• The process of tagging the data and inserting these metrics is not going to be performed by GrimoireLab, to best of my understanding. What procedure will be followed to do so? Is it done manually? How much time and effort does it require to do such a task?

Thank you so much.

EDIT: Also for anyone interested in this project, some clarity can be gained by looking at this issue. :)

[abhiandthetruth]

Is the implementation architecture been decided yet or is it still in progress?

[GeorgLink]

Hi @animeshk08 ,

Hello mentors, I have a few questions regarding the implementation of Social Currency Metric System :)

Is there going to be a feature to add these social currencies manually? In that case who can do so? Also how often they are to be added to generate a good reflective result?

Social currency is an abstract idea. The concete things this project will be working with is data that GrimoireLab already collects: email text, tweet text, issue comment text, and other text. That text needs to be annotated or tagged (see metric definition). This may be a manual process where a person reads these messages and adds tags. What we need is that user interface that pulls in the messages from GrimoireLab and allows someone to add tags which are stored back to GrimoireLab. The other thing we need are visualizations to show how often a tag was used and report that back.

How does one decide the weight score for these metrics?

That is determined based on business goals someone has. In the visualization or reporting feature of this project, I imagine someone can set weights. However, weights are optional at this point and can be an advanced feature.

The process of tagging the data and inserting these metrics is not going to be performed by GrimoireLab, to best of my understanding. What procedure will be followed to do so? Is it done manually? How much time and effort does it require to do such a task?

I described the process above. GrimoireLab does not do tagging, a person will do the tagging manually in the first iteration and maybe an artificial intelligence in a stretch goal. How long it takes to do tagging depends on the size of the community and how many messages the members produce.

[animeshk08]

Thank you @GeorgLink for the all the clarifications :) One more thing the user interface for adding the tags will also be provide be Grimoirelab or just the data fetching and visualisation part?? As I was having the notion that the data will be exported in excel or CSV format and then later imported again.

[samanthavenialogan]

Hello everyone!

I feel a little silly for not commenting on the main thread yet but I've been supporting some side messages so I'm going to put a master here.

But first, Introduction! Skip this if you don't really care :)

My name is Samantha Venia Logan. I go by Venia. I am the originating creator of the Social Currency Metrics System and the CEO of SociallyConstructed.Online.

Both my co-founder, Dylan, and I have social-scientific backgrounds so we don't have any coding knowledge beyond marketing requirements. That said however I am a full-stack marketer and online community manager whereas Dylan is a copywriter and anthropologist by trade. We are mentors on this project to provide some insight into how the system works, what it's used for, and how the system should be built to support the community management and social capital process.

@GeorgLink and CHAOSS, however, are so wonderful and on the ball about this that it seems a lot of questions can be answered through the present resources provided.

I still want to extend my assistance to you all and offer myself up as a resource. I am primarily available by email at samantha@sociallyconstructed.online and I LOVE video calls so just let me know if you'd like to have an extended conversation. I'm also more than happy to schedule something as a group ^^ Let me know if you'd prefer that.

I am already very impressed with the passion, suggestions, and questions though. You've all got amazing ideas like the ability to add a voice-service to the system! A very innovative way to use it!

So to answer a few questions that have come around:

Is there going to be a feature to add these social currencies manually? In that case who can do so? Also how often they are to be added to generate a good reflective result?

The social currencies are to be defined in the codex which is the most important part. You must have a clear grounded definition for the term, an example of when it is applicable and an example of when it is not. Then you can either manually tag a term or teach a language processor to do it.

The SCMS is built to scale with the community. As it stands because it doesn't have a "home" system you can usually build it in the preferred tech stack of the community and as it scales new options for what you can do emerge. For CHAOSS, obviously, that will be a part of your build.

Off the cuff, it's more than possible to tack a language processing AI or algorithm to the back end of the system but by no means is this necessary. This means that as you tag content you're training the system to tag larger quantities of traffic and the system will automatically scale.

BUT,

Dylan and I STRONGLY recommend tagging your own traffic 1 hour per week no matter what you do, and then we suggest that you have 1 hour a week to meet with others to discuss the process as a whole. The MOST important part of this system is the tag codex because if you're tagging your data wrong (just like tagging UTM traffic in a marketing campaign) it can have huge consequences.

If you want to tack the language processer on this is a good opportunity to own and spearhead that part of the project, but it is VERY MUCH a stretch goal. :)

• How does one decide the weight score for these metrics?

Score weights, as Georg stated are based on business goals. The MOST important part of the system is defining the terms in the codex. Using these tags you can then decide, "what counts as bad for this tag and what counts as a good response?" When you fill out when to use the term and when not to, how you should weight it typically falls in to place based on the tone of the sentiment and whether a comment counted against or for the tag. It should also be noted that research in marketing has shown that a 7-point scale is usually most accurate without being overwhelming. So I use a rating system of from -3 to +3 with 0 as a neutral default.

• The process of tagging the data and inserting these metrics is not going to be performed by GrimoireLab, to best of my understanding. What procedure will be followed to do so? Is it done manually? How much time and effort does it require to do such a task?

As I stated above we will NEVER recommend that you not tag your traffic. The system scales and adding AI is possible but you should be reading the comments. If you give it to an automation task to read you're doing your community a massive disservice because you, the one with the power, aren't listening. So, 1 hour per week per person and then a 1 hour meeting to discuss:

1. Best and worst comments,
2. Trends found in the data,
3. What you'd like to track,
4. and how previous tracks have stacked up against the prior week.
5. Look at the dashboards to see if there's an initiative you can launch that improves the results
6. And poke holes at the validity of the data and see if there are any data sources you're missing.

This system works surprisingly like google analytics and tracking traffic quantitatively. You have to get all your traffic in to lanes, plan the way you're getting and analyzing it, and always trust, but verify your data

Reading through some materials online about Social Currency had a different set of dimensions.

• Affiliation – Offering a sense of community among your customers.
• Conversation – Engaging in the customer conversation around your brand.
• Utility – Creating utility and value for customers. While helping them become more socially relevant.
• Advocacy – Developing strong feelings of loyalty to a brand. To the point where customers recommend and even defend it.
• Information – Giving valuable and shareable information.
• Identity – Providing customers with their own identity in a group. Can you provide me some references where the said dimensions in SCMS doc are mentioned? :) Or how are some of the former dimensions are dependent on the latter ones.

These are correct from an academic standpoint but a quick look will tell you, analyzing these from week to week in-situ with your community leaves a lot of processing and subjectivity. They're largely up in the air and hard to define. We wanted actionable definitions you could predict the next week on so we did 3 things to them:

1. It consolidated several different definitions and concepts from sociology, psychology, and anthropology to remove some ambiguity from the underlying concepts,
2. simplified the more abstract language into more commonly used terms in communities as best it could and
3. turned them in to lead metrics as opposed to lag metrics that can be used to predict a measurable response in the community from week to week.

These changes make it viable as a daily analytics system by making the collecting tagging and analytics portions easy to understand, applicable to any community, and capable of predicting community response to actions.

Conclusion / TL;DR These are all amazing questions and I hope the long response doesn't disappoint. The TL;DR is that this system you're making is to support a cultural process so there are aspects of automation and user experience that should be left to the community like definitions. That said, however, building a system that facilitates more objective, actionable, and easier use of the qualitative data communities are producing is important. This system takes a complicated, lengthy, vague, and historical process and makes it a simple, predictive, concrete, and immersive process that scales to match the community it's reporting on.

If you have any questions, I'm more than happy to talk here, over email, over the phone, or on a hangout. It may, however, be best to keep the conversation here though so others benefit from your amazing questions).

[samanthavenialogan]

One more thing the user interface for adding the tags will also provide be Grimoirelab or just the data fetching and visualization part?? As I was having the notion that the data will be exported in Excel or CSV format and then later imported again.

Regarding this, it's best to make the tagging a part of the native platform because then you have to handle collaboration rules and permissions, but the CSV should export with the needed container columns to tag in whatever sheet software you prefer.

We have seen it the way you're envisioning though. In specific implementations like my use for Comcast Xfinity customer support, we had the system implemented in Microsoft SharePoint website. Support agents would export a CSV to desktop, fill in ticket notes with every call, and then use a micosoft sharepoiint addon to add their data back to the MySQL via CSV at the end of the day. It wasn't feasible to have all the agents on the floor input their data directly until we were confident it would work and then we eventually built the system in Medallion afterward.

[animeshk08]

Thank you @samanthavenialogan for giving such detailed answers to our questions.:)

Regarding this, it's best to make the tagging a part of the native platform because then you have to handle collaboration rules and permissions, but the CSV should export with the needed container columns to tag in whatever sheet software you prefer.

Now I understand the direction in which you want to project to move forward and I will explore more to integrate this way of tagging the data on the SCMS platform.

[animeshk08]

Hi, @valeriocos I have been going through the microtasks. However, while running micromordred I am getting the following error could you help me with it.

Thank you

[valeriocos]

Hi @animeshk08 ,

• Which docker-compose version are you running? The one with or without searchguard (https://github.com/chaoss/grimoirelab-sirmordred#docker-compose-with-searchguard)?

• Can you post the output of curl -XGET <elasticsearch-url> -k ?

• Can you post the logs of the elasticsearch container (i.e., docker logs <es-container>) ?

• Please check the section https://github.com/chaoss/grimoirelab-sirmordred#troubleshooting, to see if what you are experiencing is already documented?

Thanks

[animeshk08]

Hi @valeriocos Thank you for the quick response.

Which docker-compose version are you running? The one with or without searchguard (https://github.com/chaoss/grimoirelab-sirmordred#docker-compose-with-searchguard)?

Without search guard

Can you post the output of curl -XGET -k?

curl: (7) Failed to connect to localhost port 9200: Connection refused

Can you post the logs of the elasticsearch container (i.e., docker logs ) ?

While running this command docker-compose -f docker-compose.yml up I am getting the following logs.

Creating network "utils_default" with the default driver
Creating utils_mariadb_1       ... done
Creating utils_elasticsearch_1 ... done
Creating utils_kibiter_1       ... done
Attaching to utils_elasticsearch_1, utils_mariadb_1, utils_kibiter_1
elasticsearch_1  | [2020-03-14T09:43:44,198][INFO ][o.e.n.Node               ] [] initializing ...
elasticsearch_1  | [2020-03-14T09:43:45,711][INFO ][o.e.e.NodeEnvironment    ] [UZ6jRxG] using [1] data paths, mounts [[/ (overlay)]], net usable_space [36.5gb], net total_space [116.1gb], types [overlay]
elasticsearch_1  | [2020-03-14T09:43:45,711][INFO ][o.e.e.NodeEnvironment    ] [UZ6jRxG] heap size [1.9gb], compressed ordinary object pointers [true]
mariadb_1        | Initializing database
elasticsearch_1  | [2020-03-14T09:43:45,714][INFO ][o.e.n.Node               ] node name [UZ6jRxG] derived from node ID [UZ6jRxGpThm8DQCHjKjsLw]; set [node.name] to override
mariadb_1        | 200314  9:43:41 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 68 ...
elasticsearch_1  | [2020-03-14T09:43:45,714][INFO ][o.e.n.Node               ] version[6.1.4], pid[1], build[d838f2d/2018-03-14T08:28:22.470Z], OS[Linux/5.3.0-40-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314  9:43:42 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314  9:43:42 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Using Linux native AIO
elasticsearch_1  | [2020-03-14T09:43:45,715][INFO ][o.e.n.Node               ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -XX:+HeapDumpOnOutOfMemoryError, -Xms2g, -Xmx2g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Using CPU crc32 instructions
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [aggs-matrix-stats]
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [analysis-common]
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [ingest-common]
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Initializing buffer pool, size = 256.0M
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [lang-expression]
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [lang-mustache]
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314  9:43:42 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created!
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [lang-painless]
mariadb_1        | 200314  9:43:42 [Note] InnoDB: Database physically writes the file full: wait...
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [mapper-extras]
elasticsearch_1  | [2020-03-14T09:43:46,534][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [parent-join]
mariadb_1        | 200314  9:43:43 [Note] InnoDB: Setting log file ./ib_logfile101 size to 48 MB
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [percolator]
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [reindex]
mariadb_1        | 200314  9:43:44 [Note] InnoDB: Setting log file ./ib_logfile1 size to 48 MB
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [repository-url]
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
mariadb_1        | 200314  9:43:47 [Warning] InnoDB: New log files created, LSN=45781
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Doublewrite buffer not found: creating new
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [transport-netty4]
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Doublewrite buffer created
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded module [tribe]
mariadb_1        | 200314  9:43:47 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314  9:43:47 [Warning] InnoDB: Creating foreign key constraint system tables.
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded plugin [ingest-geoip]
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Foreign key constraint system tables created
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Creating tablespace and datafile system tables.
elasticsearch_1  | [2020-03-14T09:43:46,535][INFO ][o.e.p.PluginsService     ] [UZ6jRxG] loaded plugin [ingest-user-agent]
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Tablespace and datafile system tables created.
elasticsearch_1  | [2020-03-14T09:43:47,767][INFO ][o.e.d.DiscoveryModule    ] [UZ6jRxG] using discovery type [zen]
mariadb_1        | 200314  9:43:47 [Note] InnoDB: Waiting for purge to start
elasticsearch_1  | [2020-03-14T09:43:48,326][INFO ][o.e.n.Node               ] initialized
mariadb_1        | 200314  9:43:47 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 0
elasticsearch_1  | [2020-03-14T09:43:48,326][INFO ][o.e.n.Node               ] [UZ6jRxG] starting ...
elasticsearch_1  | [2020-03-14T09:43:48,487][INFO ][o.e.t.TransportService   ] [UZ6jRxG] publish_address {172.20.0.2:9300}, bound_addresses {0.0.0.0:9300}
elasticsearch_1  | [2020-03-14T09:43:48,495][INFO ][o.e.b.BootstrapChecks    ] [UZ6jRxG] bound or publishing to a non-loopback address, enforcing bootstrap checks
elasticsearch_1  | ERROR: [1] bootstrap checks failed
elasticsearch_1  | [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
elasticsearch_1  | [2020-03-14T09:43:48,502][INFO ][o.e.n.Node               ] [UZ6jRxG] stopping ...
elasticsearch_1  | [2020-03-14T09:43:48,525][INFO ][o.e.n.Node               ] [UZ6jRxG] stopped
elasticsearch_1  | [2020-03-14T09:43:48,525][INFO ][o.e.n.Node               ] [UZ6jRxG] closing ...
elasticsearch_1  | [2020-03-14T09:43:48,531][INFO ][o.e.n.Node               ] [UZ6jRxG] closed
utils_elasticsearch_1 exited with code 78
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["status","plugin:kibana@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["status","plugin:elasticsearch@6.1.4-1","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["status","plugin:console@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["status","plugin:metrics@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["error","elasticsearch","admin"],"pid":1,"message":"Request error, retrying\nHEAD http://elasticsearch:9200/ => connect ECONNREFUSED 127.0.0.1:9200"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["status","plugin:timelion@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:55Z","tags":["status","plugin:elasticsearch@6.1.4-1","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch at http://elasticsearch:9200.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
mariadb_1        | 200314  9:43:57 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314  9:43:57 [Note] InnoDB: Starting shutdown...
mariadb_1        | 200314  9:43:58 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:58Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:43:58Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314  9:43:59 [Note] InnoDB: Shutdown completed; log sequence number 1616697
mariadb_1        | 200314  9:44:00 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 94 ...
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314  9:44:00 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314  9:44:00 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314  9:44:00 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314  9:44:00 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616697
mariadb_1        | 200314  9:44:00 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314  9:44:00 [Note] InnoDB: Starting shutdown...
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:00Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:00Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314  9:44:01 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Shutdown completed; log sequence number 1616707
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:03Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:03Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 
mariadb_1        | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
mariadb_1        | To do so, start the server, then issue the following commands:
mariadb_1        | 
mariadb_1        | '/usr/bin/mysqladmin' -u root password 'new-password'
mariadb_1        | '/usr/bin/mysqladmin' -u root -h  password 'new-password'
mariadb_1        | 
mariadb_1        | Alternatively you can run:
mariadb_1        | '/usr/bin/mysql_secure_installation'
mariadb_1        | 
mariadb_1        | which will also give you the option of removing the test
mariadb_1        | databases and anonymous user created by default.  This is
mariadb_1        | strongly recommended for production servers.
mariadb_1        | 
mariadb_1        | See the MariaDB Knowledgebase at http://mariadb.com/kb or the
mariadb_1        | MySQL manual for more instructions.
mariadb_1        | 
mariadb_1        | Please report any problems at http://mariadb.org/jira
mariadb_1        | 
mariadb_1        | The latest information about MariaDB is available at http://mariadb.org/.
mariadb_1        | You can find additional information about the MySQL part at:
mariadb_1        | http://dev.mysql.com
mariadb_1        | Consider joining MariaDB's strong and vibrant community:
mariadb_1        | https://mariadb.org/get-involved/
mariadb_1        | 
mariadb_1        | Database initialized
mariadb_1        | 200314  9:44:03 [Note] mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 124 ...
mariadb_1        | MySQL init process in progress...
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314  9:44:03 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314  9:44:03 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314  9:44:03 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314  9:44:03 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314  9:44:04 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616707
mariadb_1        | 200314  9:44:04 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 200314  9:44:04 [Warning] 'user' entry 'root@2650998d4990' ignored in --skip-name-resolve mode.
mariadb_1        | 200314  9:44:04 [Warning] 'proxies_priv' entry '@% root@2650998d4990' ignored in --skip-name-resolve mode.
mariadb_1        | 200314  9:44:04 [Note] mysqld: ready for connections.
mariadb_1        | Version: '10.0.38-MariaDB-1~xenial'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
mariadb_1        | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:06Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:06Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314  9:44:06 [Warning] 'proxies_priv' entry '@% root@2650998d4990' ignored in --skip-name-resolve mode.
mariadb_1        | 
mariadb_1        | 200314  9:44:06 [Note] mysqld: Normal shutdown
mariadb_1        | 200314  9:44:06 [Note] Event Scheduler: Purging the queue. 0 events
mariadb_1        | 200314  9:44:06 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314  9:44:06 [Note] InnoDB: Starting shutdown...
mariadb_1        | 200314  9:44:06 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
mariadb_1        | 200314  9:44:07 [Note] InnoDB: Shutdown completed; log sequence number 1616717
mariadb_1        | 200314  9:44:08 [Note] mysqld: Shutdown complete
mariadb_1        | 
mariadb_1        | 
mariadb_1        | MySQL init process done. Ready for start up.
mariadb_1        | 
mariadb_1        | 200314  9:44:08 [Note] mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 1 ...
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314  9:44:08 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314  9:44:08 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314  9:44:08 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314  9:44:08 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314  9:44:08 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616717
mariadb_1        | 200314  9:44:08 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 200314  9:44:08 [Note] Server socket created on IP: '::'.
mariadb_1        | 200314  9:44:08 [Warning] 'proxies_priv' entry '@% root@2650998d4990' ignored in --skip-name-resolve mode.
mariadb_1        | 200314  9:44:08 [Note] mysqld: ready for connections.
mariadb_1        | Version: '10.0.38-MariaDB-1~xenial'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:08Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:08Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:11Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:11Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:13Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:13Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:16Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:16Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:18Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:18Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:21Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:21Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:23Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:23Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:26Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:26Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:28Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:28Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:31Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:31Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:33Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T09:44:33Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}

Please check the section https://github.com/chaoss/grimoirelab-sirmordred#troubleshooting, to see if what you are experiencing is already documented?

Sorry I could not find something related to my problem here.

[ria18405]

Hi @animeshk08

Your error lies in max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

You could try increasing it by : sysctl -w fs.file-max=65536

Hope it helps Thanks

[animeshk08]

HI @ria18405 I have tried running sysctl -w fs.file-max=65536 command already however the error still persists. I have cross-checked that the value has changed as running sysctl fs.file-max gives me the following output: fs.file-max = 65536

EDIT: I also went ahead and tried to make this change permanent by add the following values to etc/security/limits.conf of my system.

* soft nofile 65536

* hard nofile 65536

[ria18405]

@animeshk08 Have you tried composing the docker again? Composing the docker again might resolve the issue.

[abhiandthetruth]

@animeshk08 can you share you docker-compose.yml or the link to the tutorial you are following?

[animeshk08]

Hi @abhiandthetruth. Thank you for the quick response

My docker-compose.yml

version: '2.2'

services:
    mariadb:
      image: mariadb:10.0
      expose:
        - "3306"
      environment:
        - MYSQL_ROOT_PASSWORD=
        - MYSQL_ALLOW_EMPTY_PASSWORD=yes

    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.1.4
      command: /elasticsearch/bin/elasticsearch -E network.bind_host=0.0.0.0
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g

    kibiter:
      restart: on-failure:5
      image: bitergia/kibiter:optimized-v6.1.4-4
      environment:
        - PROJECT_NAME=Demo
        - NODE_OPTIONS=--max-old-space-size=1000
        - ELASTICSEARCH_URL=http://elasticsearch:9200
      links:
        - elasticsearch
      ports:
        - 5601:5601

[abhiandthetruth]

I'm trying to debug the problem by running it on my system. BTW have you tried using the with searchguard version. It worked just fine with me just make sure you set the correct vm.max_map_count sudo sysctl -w vm.max_map_count=262144

[animeshk08]

Hi @abhiandthetruth @ria18405 @valeriocos Thank you everyone for looking into this issue. Its been resolved.

Modify the docker-compse.yml by adding the following lines inder elastic:

 ulimits:
          nofile:
              soft: 65536
              hard: 65536

The new docker-compose.yml looks as follows:

version: '2.2'

services:
    mariadb:
      image: mariadb:10.0
      expose:
        - "3306"
      environment:
        - MYSQL_ROOT_PASSWORD=
        - MYSQL_ALLOW_EMPTY_PASSWORD=yes

    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.1.4
      command: /elasticsearch/bin/elasticsearch -E network.bind_host=0.0.0.0
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g
      ulimits:
        nofile:
          soft: 65536
          hard: 65536

    kibiter:
      restart: on-failure:5
      image: bitergia/kibiter:optimized-v6.1.4-4
      environment:
        - PROJECT_NAME=Demo
        - NODE_OPTIONS=--max-old-space-size=1000
        - ELASTICSEARCH_URL=http://elasticsearch:9200
      links:
        - elasticsearch
      ports:
        - 5601:5601

@valeriocos Let me know if you would like to document this :)

[animeshk08]

However, This error still persists: https://github.com/chaoss/grimoirelab/issues/288#issuecomment-599031158 :(

[abhiandthetruth]

whats the output of the curl command now?

[abhiandthetruth]

@animeshk08 have you changed the corresponding urls in setup.cfg?

[animeshk08]

Output for curl:

{
  "name" : "Mmigh3Z",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "3QN9WZN4RzipqKaFeaGMtA",
  "version" : {
    "number" : "6.1.4",
    "build_hash" : "d838f2d",
    "build_date" : "2018-03-14T08:28:22.470Z",
    "build_snapshot" : false,
    "lucene_version" : "7.1.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Logs for docker-compose -f docker-compose.yml up:

Recreating utils_mariadb_1       ... done
Recreating utils_elasticsearch_1 ... done
Recreating utils_kibiter_1       ... 
^CGracefully stopping... (press Ctrl+C again to force)
Stopping utils_mariadb_1         ... 
Stopping utils_elasticsearch_1   ... 
^CKilling utils_mariadb_1          ... 
Killing utils_elasticsearch_1    ... 
^CERROR: Aborting.
animesh@msi:~/glsource/grimoirelab-sirmordred/utils$ docker-compose -f docker-compose.yml down
Stopping utils_kibiter_1 ... done
Removing utils_kibiter_1              ... done
Removing utils_mariadb_1              ... done
Removing utils_elasticsearch_1        ... done
Removing 92bf5b79bc87_utils_kibiter_1 ... done
Removing network utils_default
animesh@msi:~/glsource/grimoirelab-sirmordred/utils$ docker-compose -f docker-compose.yml up
Creating network "utils_default" with the default driver
Creating utils_mariadb_1       ... done
Creating utils_elasticsearch_1 ... done
Creating utils_kibiter_1       ... done
Attaching to utils_elasticsearch_1, utils_mariadb_1, utils_kibiter_1
mariadb_1        | Initializing database
mariadb_1        | 200314 10:57:02 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 68 ...
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 10:57:02 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 10:57:02 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 10:57:02 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created!
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB
mariadb_1        | 200314 10:57:02 [Note] InnoDB: Database physically writes the file full: wait...
mariadb_1        | 200314 10:57:03 [Note] InnoDB: Setting log file ./ib_logfile101 size to 48 MB
mariadb_1        | 200314 10:57:04 [Note] InnoDB: Setting log file ./ib_logfile1 size to 48 MB
mariadb_1        | 200314 10:57:06 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
mariadb_1        | 200314 10:57:06 [Warning] InnoDB: New log files created, LSN=45781
mariadb_1        | 200314 10:57:06 [Note] InnoDB: Doublewrite buffer not found: creating new
mariadb_1        | 200314 10:57:07 [Note] InnoDB: Doublewrite buffer created
mariadb_1        | 200314 10:57:07 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 10:57:07 [Warning] InnoDB: Creating foreign key constraint system tables.
mariadb_1        | 200314 10:57:07 [Note] InnoDB: Foreign key constraint system tables created
mariadb_1        | 200314 10:57:07 [Note] InnoDB: Creating tablespace and datafile system tables.
mariadb_1        | 200314 10:57:07 [Note] InnoDB: Tablespace and datafile system tables created.
mariadb_1        | 200314 10:57:07 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 10:57:07 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 0
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["status","plugin:kibana@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["status","plugin:elasticsearch@6.1.4-1","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["error","elasticsearch","admin"],"pid":1,"message":"Request error, retrying\nHEAD http://elasticsearch:9200/ => connect ECONNREFUSED 172.29.0.2:9200"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["status","plugin:console@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["status","plugin:elasticsearch@6.1.4-1","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch at http://elasticsearch:9200.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["status","plugin:metrics@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["status","plugin:timelion@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:10Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:13Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:13Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:15Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:15Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314 10:57:17 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314 10:57:17 [Note] InnoDB: Starting shutdown...
mariadb_1        | 200314 10:57:17 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:18Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:18Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
elasticsearch_1  | [2020-03-14T10:57:18,538][INFO ][o.e.n.Node               ] [] initializing ...
elasticsearch_1  | [2020-03-14T10:57:18,822][INFO ][o.e.e.NodeEnvironment    ] [p2Tnmy3] using [1] data paths, mounts [[/ (overlay)]], net usable_space [35.3gb], net total_space [116.1gb], types [overlay]
elasticsearch_1  | [2020-03-14T10:57:18,823][INFO ][o.e.e.NodeEnvironment    ] [p2Tnmy3] heap size [1.9gb], compressed ordinary object pointers [true]
elasticsearch_1  | [2020-03-14T10:57:18,825][INFO ][o.e.n.Node               ] node name [p2Tnmy3] derived from node ID [p2Tnmy3HRW6aZVQoOcVC8Q]; set [node.name] to override
elasticsearch_1  | [2020-03-14T10:57:18,826][INFO ][o.e.n.Node               ] version[6.1.4], pid[1], build[d838f2d/2018-03-14T08:28:22.470Z], OS[Linux/5.3.0-40-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
elasticsearch_1  | [2020-03-14T10:57:18,826][INFO ][o.e.n.Node               ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -XX:+HeapDumpOnOutOfMemoryError, -Xms2g, -Xmx2g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Shutdown completed; log sequence number 1616697
mariadb_1        | 200314 10:57:19 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 94 ...
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 10:57:19 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 10:57:19 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 10:57:19 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 10:57:20 [Note] InnoDB: Highest supported file format is Barracuda.
elasticsearch_1  | [2020-03-14T10:57:20,226][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [aggs-matrix-stats]
elasticsearch_1  | [2020-03-14T10:57:20,226][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [analysis-common]
elasticsearch_1  | [2020-03-14T10:57:20,226][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [ingest-common]
elasticsearch_1  | [2020-03-14T10:57:20,226][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [lang-expression]
elasticsearch_1  | [2020-03-14T10:57:20,226][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [lang-mustache]
elasticsearch_1  | [2020-03-14T10:57:20,226][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [lang-painless]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [mapper-extras]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [parent-join]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [percolator]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [reindex]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [repository-url]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [transport-netty4]
elasticsearch_1  | [2020-03-14T10:57:20,227][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded module [tribe]
elasticsearch_1  | [2020-03-14T10:57:20,228][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded plugin [ingest-geoip]
elasticsearch_1  | [2020-03-14T10:57:20,228][INFO ][o.e.p.PluginsService     ] [p2Tnmy3] loaded plugin [ingest-user-agent]
mariadb_1        | 200314 10:57:20 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 10:57:20 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 10:57:20 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616697
mariadb_1        | 200314 10:57:20 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314 10:57:20 [Note] InnoDB: Starting shutdown...
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:20Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:20Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314 10:57:21 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
elasticsearch_1  | [2020-03-14T10:57:22,224][INFO ][o.e.d.DiscoveryModule    ] [p2Tnmy3] using discovery type [zen]
elasticsearch_1  | [2020-03-14T10:57:22,686][INFO ][o.e.n.Node               ] initialized
elasticsearch_1  | [2020-03-14T10:57:22,687][INFO ][o.e.n.Node               ] [p2Tnmy3] starting ...
elasticsearch_1  | [2020-03-14T10:57:22,859][INFO ][o.e.t.TransportService   ] [p2Tnmy3] publish_address {172.29.0.2:9300}, bound_addresses {0.0.0.0:9300}
elasticsearch_1  | [2020-03-14T10:57:22,865][INFO ][o.e.b.BootstrapChecks    ] [p2Tnmy3] bound or publishing to a non-loopback address, enforcing bootstrap checks
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:23Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:23Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Shutdown completed; log sequence number 1616707
mariadb_1        | 
mariadb_1        | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
mariadb_1        | To do so, start the server, then issue the following commands:
mariadb_1        | 
mariadb_1        | '/usr/bin/mysqladmin' -u root password 'new-password'
mariadb_1        | '/usr/bin/mysqladmin' -u root -h  password 'new-password'
mariadb_1        | 
mariadb_1        | Alternatively you can run:
mariadb_1        | '/usr/bin/mysql_secure_installation'
mariadb_1        | 
mariadb_1        | which will also give you the option of removing the test
mariadb_1        | databases and anonymous user created by default.  This is
mariadb_1        | strongly recommended for production servers.
mariadb_1        | 
mariadb_1        | See the MariaDB Knowledgebase at http://mariadb.com/kb or the
mariadb_1        | MySQL manual for more instructions.
mariadb_1        | 
mariadb_1        | Please report any problems at http://mariadb.org/jira
mariadb_1        | 
mariadb_1        | The latest information about MariaDB is available at http://mariadb.org/.
mariadb_1        | You can find additional information about the MySQL part at:
mariadb_1        | http://dev.mysql.com
mariadb_1        | Consider joining MariaDB's strong and vibrant community:
mariadb_1        | https://mariadb.org/get-involved/
mariadb_1        | 
mariadb_1        | Database initialized
mariadb_1        | MySQL init process in progress...
mariadb_1        | 200314 10:57:23 [Note] mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 124 ...
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 10:57:23 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 10:57:23 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314 10:57:23 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 10:57:23 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 10:57:23 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616707
mariadb_1        | 200314 10:57:23 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 200314 10:57:23 [Warning] 'user' entry 'root@8e7aa3d8a2d8' ignored in --skip-name-resolve mode.
mariadb_1        | 200314 10:57:23 [Warning] 'proxies_priv' entry '@% root@8e7aa3d8a2d8' ignored in --skip-name-resolve mode.
mariadb_1        | 200314 10:57:23 [Note] mysqld: ready for connections.
mariadb_1        | Version: '10.0.38-MariaDB-1~xenial'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
mariadb_1        | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:25Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:25Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"}
mariadb_1        | 200314 10:57:25 [Warning] 'proxies_priv' entry '@% root@8e7aa3d8a2d8' ignored in --skip-name-resolve mode.
mariadb_1        | 
mariadb_1        | 200314 10:57:25 [Note] mysqld: Normal shutdown
mariadb_1        | 200314 10:57:25 [Note] Event Scheduler: Purging the queue. 0 events
mariadb_1        | 200314 10:57:25 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314 10:57:25 [Note] InnoDB: Starting shutdown...
elasticsearch_1  | [2020-03-14T10:57:25,917][INFO ][o.e.c.s.MasterService    ] [p2Tnmy3] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {p2Tnmy3}{p2Tnmy3HRW6aZVQoOcVC8Q}{GvYFKX8XT5OrVVx1oKA3TQ}{172.29.0.2}{172.29.0.2:9300}
elasticsearch_1  | [2020-03-14T10:57:25,923][INFO ][o.e.c.s.ClusterApplierService] [p2Tnmy3] new_master {p2Tnmy3}{p2Tnmy3HRW6aZVQoOcVC8Q}{GvYFKX8XT5OrVVx1oKA3TQ}{172.29.0.2}{172.29.0.2:9300}, reason: apply cluster state (from master [master {p2Tnmy3}{p2Tnmy3HRW6aZVQoOcVC8Q}{GvYFKX8XT5OrVVx1oKA3TQ}{172.29.0.2}{172.29.0.2:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
elasticsearch_1  | [2020-03-14T10:57:25,934][INFO ][o.e.h.n.Netty4HttpServerTransport] [p2Tnmy3] publish_address {172.29.0.2:9200}, bound_addresses {0.0.0.0:9200}
elasticsearch_1  | [2020-03-14T10:57:25,935][INFO ][o.e.n.Node               ] [p2Tnmy3] started
mariadb_1        | 200314 10:57:25 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
elasticsearch_1  | [2020-03-14T10:57:26,088][INFO ][o.e.g.GatewayService     ] [p2Tnmy3] recovered [0] indices into cluster_state
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Shutdown completed; log sequence number 1616717
mariadb_1        | 200314 10:57:27 [Note] mysqld: Shutdown complete
mariadb_1        | 
mariadb_1        | 
mariadb_1        | MySQL init process done. Ready for start up.
mariadb_1        | 
mariadb_1        | 200314 10:57:27 [Note] mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 1 ...
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 10:57:27 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 10:57:27 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314 10:57:27 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 10:57:27 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 10:57:27 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616717
mariadb_1        | 200314 10:57:27 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 200314 10:57:27 [Note] Server socket created on IP: '::'.
mariadb_1        | 200314 10:57:27 [Warning] 'proxies_priv' entry '@% root@8e7aa3d8a2d8' ignored in --skip-name-resolve mode.
mariadb_1        | 200314 10:57:27 [Note] mysqld: ready for connections.
mariadb_1        | Version: '10.0.38-MariaDB-1~xenial'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:28Z","tags":["warning"],"pid":1,"kibanaVersion":"6.1.4-1","nodes":[{"version":"6.1.4","http":{"publish_address":"172.29.0.2:9200"},"ip":"172.29.0.2"}],"message":"You're running Kibana 6.1.4-1 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.1.4 @ 172.29.0.2:9200 (172.29.0.2)"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T10:57:28Z","tags":["status","plugin:elasticsearch@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from red to green - Ready","prevState":"red","prevMsg":"Unable to connect to Elasticsearch at http://elasticsearch:9200."}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:54Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1"},"res":{"statusCode":200,"responseTime":64,"contentLength":9},"message":"GET / 200 64ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:55Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/","upgrade-insecure-requests":"1"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/"},"res":{"statusCode":200,"responseTime":252,"contentLength":9},"message":"GET /app/kibana 200 252ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:55Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/bundles/kibana.style.css?v=1","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"text/css,*/*;q=0.1","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 17:15:31 GMT","if-none-match":"\"3db9d568504a51be3350c2ef4051f03178d8c55d-/bundles/-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":36,"contentLength":9},"message":"GET /bundles/kibana.style.css?v=1 200 36ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:55Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/bundles/commons.style.css?v=1","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"text/css,*/*;q=0.1","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":43,"contentLength":9},"message":"GET /bundles/commons.style.css?v=1 200 43ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:55Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/bundles/commons.bundle.js?v=1","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 17:15:31 GMT","if-none-match":"\"cc24657c5f57c8846b0ca2480431a2505a30d4ae-/bundles/-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":106,"contentLength":9},"message":"GET /bundles/commons.bundle.js?v=1 200 106ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:55Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/bundles/kibana.bundle.js?v=1","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 17:15:31 GMT","if-none-match":"\"6d1873a30f93e1e70e743bbaccfdfe0a666b914b-/bundles/-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":404,"contentLength":9},"message":"GET /bundles/kibana.bundle.js?v=1 200 404ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_5_0","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"application/json, text/javascript, */*; q=0.01","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","kbn-version":"6.1.4-1","x-requested-with":"XMLHttpRequest","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":27,"contentLength":9},"message":"GET /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_5_0 200 27ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/plugins/kibana/assets/discover.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":66,"contentLength":9},"message":"GET /plugins/kibana/assets/discover.svg 200 66ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/visualize.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 16:41:48 GMT","if-none-match":"\"4cc79a4d91bd0380d0c82a6b092f339d185670ef-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":304,"responseTime":8,"contentLength":9},"message":"GET /plugins/kibana/assets/visualize.svg 304 8ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/dashboard.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 16:41:48 GMT","if-none-match":"\"42c2161fa64691414784868afdd722444460763a-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":304,"responseTime":10,"contentLength":9},"message":"GET /plugins/kibana/assets/dashboard.svg 304 10ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/plugins/timelion/icon.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 16:41:48 GMT","if-none-match":"\"cb793d5314d680b7d5ce130f0393a70b51989541-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":304,"responseTime":16,"contentLength":9},"message":"GET /plugins/timelion/icon.svg 304 16ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/settings.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 16:41:48 GMT","if-none-match":"\"4f859e27d4917026ff1590805887902b14ce79d5-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":304,"responseTime":18,"contentLength":9},"message":"GET /plugins/kibana/assets/settings.svg 304 18ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/wrench.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 16:41:48 GMT","if-none-match":"\"088a9a98c99e406dca2354af14f688ad84826b97-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":304,"responseTime":20,"contentLength":9},"message":"GET /plugins/kibana/assets/wrench.svg 304 20ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/play-circle.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/app/kibana","if-modified-since":"Wed, 18 Sep 2019 16:41:48 GMT","if-none-match":"\"2433ecf38258f7121c835670b6993600e7657717-gzip\""},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET /plugins/kibana/assets/play-circle.svg 304 3ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/bundles/4246f178bc05db44a8b68c7acf264597.svg","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","connection":"keep-alive","referer":"http://localhost:5601/bundles/commons.style.css?v=1"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/bundles/commons.style.css?v=1"},"res":{"statusCode":200,"responseTime":6,"contentLength":9},"message":"GET /bundles/4246f178bc05db44a8b68c7acf264597.svg 200 6ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/?type=index-pattern&per_page=10000&page=1","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","kbn-version":"6.1.4-1","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":27,"contentLength":9},"message":"GET /api/saved_objects/?type=index-pattern&per_page=10000&page=1 200 27ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/?type=index-pattern&fields=title&per_page=10000&page=1","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","kbn-version":"6.1.4-1","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":8,"contentLength":9},"message":"GET /api/saved_objects/?type=index-pattern&fields=title&per_page=10000&page=1 200 8ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"post","statusCode":404,"req":{"url":"/elasticsearch/.kibana/_search","method":"post","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"6.1.4-1","content-length":"43","origin":"http://localhost:5601","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":404,"responseTime":32,"contentLength":9},"message":"POST /elasticsearch/.kibana/_search 404 32ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"post","statusCode":404,"req":{"url":"/elasticsearch/.kibana/_search","method":"post","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"6.1.4-1","content-length":"41","origin":"http://localhost:5601","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":404,"responseTime":28,"contentLength":9},"message":"POST /elasticsearch/.kibana/_search 404 28ms - 9.0B"}
kibiter_1        | {"type":"response","@timestamp":"2020-03-14T10:57:57Z","tags":[],"pid":1,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/*/_search","method":"post","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko/20100101 Firefox/73.0","accept":"application/json, text/plain, */*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"6.1.4-1","content-length":"68","origin":"http://localhost:5601","connection":"keep-alive","referer":"http://localhost:5601/app/kibana"},"remoteAddress":"172.29.0.1","userAgent":"172.29.0.1","referer":"http://localhost:5601/app/kibana"},"res":{"statusCode":200,"responseTime":34,"contentLength":9},"message":"POST /elasticsearch/*/_search 200 34ms - 9.0B"}
^CGracefully stopping... (press Ctrl+C again to force)
Stopping utils_kibiter_1       ... 
Stopping utils_mariadb_1       ... 
Stopping utils_elasticsearch_1 ... 
^CKilling utils_kibiter_1        ... 
Killing utils_mariadb_1        ... 
Killing utils_elasticsearch_1  ... 
^CERROR: Aborting.
animesh@msi:~/glsource/grimoirelab-sirmordred/utils$ docker-compose -f docker-compose.yml down
Removing utils_kibiter_1       ... done
Removing utils_mariadb_1       ... done
Removing utils_elasticsearch_1 ... done
Removing network utils_default
animesh@msi:~/glsource/grimoirelab-sirmordred/utils$ docker-compose -f docker-compose.yml up
Creating network "utils_default" with the default driver
Creating utils_mariadb_1       ... done
Creating utils_elasticsearch_1 ... done
Creating utils_kibiter_1       ... done
Attaching to utils_elasticsearch_1, utils_mariadb_1, utils_kibiter_1
mariadb_1        | Initializing database
mariadb_1        | 200314 11:00:53 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 68 ...
elasticsearch_1  | [2020-03-14T11:00:55,154][INFO ][o.e.n.Node               ] [] initializing ...
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Using mutexes to ref count buffer pool pages
elasticsearch_1  | [2020-03-14T11:00:55,809][INFO ][o.e.e.NodeEnvironment    ] [Mmigh3Z] using [1] data paths, mounts [[/ (overlay)]], net usable_space [35.2gb], net total_space [116.1gb], types [overlay]
mariadb_1        | 200314 11:00:54 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 11:00:54 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Compressed tables use zlib 1.2.8
elasticsearch_1  | [2020-03-14T11:00:55,810][INFO ][o.e.e.NodeEnvironment    ] [Mmigh3Z] heap size [1.9gb], compressed ordinary object pointers [true]
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Using CPU crc32 instructions
elasticsearch_1  | [2020-03-14T11:00:55,920][INFO ][o.e.n.Node               ] node name [Mmigh3Z] derived from node ID [Mmigh3ZvRAKJ6jLpeeiHCA]; set [node.name] to override
elasticsearch_1  | [2020-03-14T11:00:55,920][INFO ][o.e.n.Node               ] version[6.1.4], pid[1], build[d838f2d/2018-03-14T08:28:22.470Z], OS[Linux/5.3.0-40-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_161/25.161-b14]
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Completed initialization of buffer pool
elasticsearch_1  | [2020-03-14T11:00:55,921][INFO ][o.e.n.Node               ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -XX:+HeapDumpOnOutOfMemoryError, -Xms2g, -Xmx2g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]
mariadb_1        | 200314 11:00:54 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created!
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB
mariadb_1        | 200314 11:00:54 [Note] InnoDB: Database physically writes the file full: wait...
elasticsearch_1  | [2020-03-14T11:00:56,586][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [aggs-matrix-stats]
mariadb_1        | 200314 11:00:55 [Note] InnoDB: Setting log file ./ib_logfile101 size to 48 MB
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [analysis-common]
mariadb_1        | 200314 11:00:56 [Note] InnoDB: Setting log file ./ib_logfile1 size to 48 MB
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [ingest-common]
mariadb_1        | 200314 11:00:59 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
mariadb_1        | 200314 11:00:59 [Warning] InnoDB: New log files created, LSN=45781
mariadb_1        | 200314 11:00:59 [Note] InnoDB: Doublewrite buffer not found: creating new
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [lang-expression]
mariadb_1        | 200314 11:01:00 [Note] InnoDB: Doublewrite buffer created
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [lang-mustache]
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [lang-painless]
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [mapper-extras]
mariadb_1        | 200314 11:01:00 [Note] InnoDB: 128 rollback segment(s) are active.
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [parent-join]
mariadb_1        | 200314 11:01:00 [Warning] InnoDB: Creating foreign key constraint system tables.
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [percolator]
mariadb_1        | 200314 11:01:00 [Note] InnoDB: Foreign key constraint system tables created
mariadb_1        | 200314 11:01:00 [Note] InnoDB: Creating tablespace and datafile system tables.
mariadb_1        | 200314 11:01:01 [Note] InnoDB: Tablespace and datafile system tables created.
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [reindex]
mariadb_1        | 200314 11:01:01 [Note] InnoDB: Waiting for purge to start
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [repository-url]
mariadb_1        | 200314 11:01:01 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 0
elasticsearch_1  | [2020-03-14T11:00:56,587][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [transport-netty4]
elasticsearch_1  | [2020-03-14T11:00:56,588][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded module [tribe]
elasticsearch_1  | [2020-03-14T11:00:56,588][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded plugin [ingest-geoip]
elasticsearch_1  | [2020-03-14T11:00:56,588][INFO ][o.e.p.PluginsService     ] [Mmigh3Z] loaded plugin [ingest-user-agent]
elasticsearch_1  | [2020-03-14T11:00:57,684][INFO ][o.e.d.DiscoveryModule    ] [Mmigh3Z] using discovery type [zen]
elasticsearch_1  | [2020-03-14T11:00:58,029][INFO ][o.e.n.Node               ] initialized
elasticsearch_1  | [2020-03-14T11:00:58,030][INFO ][o.e.n.Node               ] [Mmigh3Z] starting ...
elasticsearch_1  | [2020-03-14T11:00:58,116][INFO ][o.e.t.TransportService   ] [Mmigh3Z] publish_address {172.30.0.2:9300}, bound_addresses {0.0.0.0:9300}
elasticsearch_1  | [2020-03-14T11:00:58,122][INFO ][o.e.b.BootstrapChecks    ] [Mmigh3Z] bound or publishing to a non-loopback address, enforcing bootstrap checks
elasticsearch_1  | [2020-03-14T11:01:01,173][INFO ][o.e.c.s.MasterService    ] [Mmigh3Z] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {Mmigh3Z}{Mmigh3ZvRAKJ6jLpeeiHCA}{k2LJhg6BSOKQRuSy5O-OiQ}{172.30.0.2}{172.30.0.2:9300}
elasticsearch_1  | [2020-03-14T11:01:01,179][INFO ][o.e.c.s.ClusterApplierService] [Mmigh3Z] new_master {Mmigh3Z}{Mmigh3ZvRAKJ6jLpeeiHCA}{k2LJhg6BSOKQRuSy5O-OiQ}{172.30.0.2}{172.30.0.2:9300}, reason: apply cluster state (from master [master {Mmigh3Z}{Mmigh3ZvRAKJ6jLpeeiHCA}{k2LJhg6BSOKQRuSy5O-OiQ}{172.30.0.2}{172.30.0.2:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
elasticsearch_1  | [2020-03-14T11:01:01,189][INFO ][o.e.h.n.Netty4HttpServerTransport] [Mmigh3Z] publish_address {172.30.0.2:9200}, bound_addresses {0.0.0.0:9200}
elasticsearch_1  | [2020-03-14T11:01:01,189][INFO ][o.e.n.Node               ] [Mmigh3Z] started
elasticsearch_1  | [2020-03-14T11:01:01,382][INFO ][o.e.g.GatewayService     ] [Mmigh3Z] recovered [0] indices into cluster_state
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["status","plugin:kibana@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["status","plugin:elasticsearch@6.1.4-1","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["status","plugin:console@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["status","plugin:metrics@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["status","plugin:timelion@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["warning"],"pid":1,"kibanaVersion":"6.1.4-1","nodes":[{"version":"6.1.4","http":{"publish_address":"172.30.0.2:9200"},"ip":"172.30.0.2"}],"message":"You're running Kibana 6.1.4-1 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.1.4 @ 172.30.0.2:9200 (172.30.0.2)"}
kibiter_1        | {"type":"log","@timestamp":"2020-03-14T11:01:03Z","tags":["status","plugin:elasticsearch@6.1.4-1","info"],"pid":1,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
mariadb_1        | 200314 11:01:11 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314 11:01:11 [Note] InnoDB: Starting shutdown...
mariadb_1        | 200314 11:01:12 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Shutdown completed; log sequence number 1616697
mariadb_1        | 200314 11:01:14 [Note] /usr/sbin/mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 94 ...
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 11:01:14 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 11:01:14 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 11:01:14 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314 11:01:15 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 11:01:15 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 11:01:15 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616697
mariadb_1        | 200314 11:01:15 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314 11:01:15 [Note] InnoDB: Starting shutdown...
mariadb_1        | 200314 11:01:16 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Shutdown completed; log sequence number 1616707
mariadb_1        | 
mariadb_1        | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
mariadb_1        | To do so, start the server, then issue the following commands:
mariadb_1        | 
mariadb_1        | '/usr/bin/mysqladmin' -u root password 'new-password'
mariadb_1        | '/usr/bin/mysqladmin' -u root -h  password 'new-password'
mariadb_1        | 
mariadb_1        | Alternatively you can run:
mariadb_1        | '/usr/bin/mysql_secure_installation'
mariadb_1        | 
mariadb_1        | which will also give you the option of removing the test
mariadb_1        | databases and anonymous user created by default.  This is
mariadb_1        | strongly recommended for production servers.
mariadb_1        | 
mariadb_1        | See the MariaDB Knowledgebase at http://mariadb.com/kb or the
mariadb_1        | MySQL manual for more instructions.
mariadb_1        | 
mariadb_1        | Please report any problems at http://mariadb.org/jira
mariadb_1        | 
mariadb_1        | The latest information about MariaDB is available at http://mariadb.org/.
mariadb_1        | You can find additional information about the MySQL part at:
mariadb_1        | http://dev.mysql.com
mariadb_1        | Consider joining MariaDB's strong and vibrant community:
mariadb_1        | https://mariadb.org/get-involved/
mariadb_1        | 
mariadb_1        | Database initialized
mariadb_1        | MySQL init process in progress...
mariadb_1        | 200314 11:01:18 [Note] mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 124 ...
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 11:01:18 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 11:01:18 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314 11:01:18 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 11:01:18 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 11:01:18 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616707
mariadb_1        | 200314 11:01:18 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 200314 11:01:18 [Warning] 'user' entry 'root@e2fd6f2028c6' ignored in --skip-name-resolve mode.
mariadb_1        | 200314 11:01:18 [Warning] 'proxies_priv' entry '@% root@e2fd6f2028c6' ignored in --skip-name-resolve mode.
mariadb_1        | 200314 11:01:19 [Note] mysqld: ready for connections.
mariadb_1        | Version: '10.0.38-MariaDB-1~xenial'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
mariadb_1        | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
mariadb_1        | 200314 11:01:20 [Warning] 'proxies_priv' entry '@% root@e2fd6f2028c6' ignored in --skip-name-resolve mode.
mariadb_1        | 
mariadb_1        | 200314 11:01:20 [Note] mysqld: Normal shutdown
mariadb_1        | 200314 11:01:20 [Note] Event Scheduler: Purging the queue. 0 events
mariadb_1        | 200314 11:01:20 [Note] InnoDB: FTS optimize thread exiting.
mariadb_1        | 200314 11:01:20 [Note] InnoDB: Starting shutdown...
mariadb_1        | 200314 11:01:21 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
mariadb_1        | 200314 11:01:22 [Note] InnoDB: Shutdown completed; log sequence number 1616717
mariadb_1        | 200314 11:01:23 [Note] mysqld: Shutdown complete
mariadb_1        | 
mariadb_1        | 
mariadb_1        | MySQL init process done. Ready for start up.
mariadb_1        | 
mariadb_1        | 200314 11:01:23 [Note] mysqld (mysqld 10.0.38-MariaDB-1~xenial) starting as process 1 ...
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Using mutexes to ref count buffer pool pages
mariadb_1        | 200314 11:01:23 [Note] InnoDB: The InnoDB memory heap is disabled
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
mariadb_1        | 200314 11:01:23 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Compressed tables use zlib 1.2.8
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Using Linux native AIO
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Using CPU crc32 instructions
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Initializing buffer pool, size = 256.0M
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Completed initialization of buffer pool
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Highest supported file format is Barracuda.
mariadb_1        | 200314 11:01:23 [Note] InnoDB: 128 rollback segment(s) are active.
mariadb_1        | 200314 11:01:23 [Note] InnoDB: Waiting for purge to start
mariadb_1        | 200314 11:01:23 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.42-84.2 started; log sequence number 1616717
mariadb_1        | 200314 11:01:23 [Note] Plugin 'FEEDBACK' is disabled.
mariadb_1        | 200314 11:01:23 [Note] Server socket created on IP: '::'.
mariadb_1        | 200314 11:01:23 [Warning] 'proxies_priv' entry '@% root@e2fd6f2028c6' ignored in --skip-name-resolve mode.
mariadb_1        | 200314 11:01:23 [Note] mysqld: ready for connections.

[animeshk08]

Hi @abhiandthetruth

@animeshk08 have you changed the corresponding urls in setup.cfg?

Sorry I didn't get you.

[abhiandthetruth]

the setup.cfg file needs the url to connect to elasticsearch, in the provided file the urls are https://admin:admin@localhost:9200 which is not applicable in your case, you should change it too http://localhost:9200. Are you using the default setup.cfg? if yes then you need to change these params.

[animeshk08]

Thank you @abhiandthetruth The issues have been resolved!!

[abhiandthetruth]

@animeshk08 I am curious as to what was the issue bcoz I couldn't reproduce the same on my machine. Was it a proxy issue? It'll help me understand the working of the system better :).

[animeshk08]

@animeshk08 I am curious as to what was the issue bcoz I couldn't reproduce the same on my machine. Was it a proxy issue?

Hi @abhiandthetruth the initial issue was due to file descripter limit which was resolved as I mentioned. I think the docker was not able to read the limit configurations of my system. I still need to figure out why was that happening though. Will ping you once I have updates :)

[abhiandthetruth]

Thanks for the reply @animeshk08.

[valeriocos]

Thank you @abhiandthetruth and @animeshk08 to sort this problem out. @animeshk08 could you add a section to the troubleshooting in sirmordred about the problem you had (https://github.com/chaoss/grimoirelab/issues/288#issuecomment-599034741)?

I found something on stackoverflow about your problem ( https://stackoverflow.com/questions/46771233/max-file-descriptors-for-elasticsearch-process-is-too-low), however I wasn't able to reproduce it on my machine.

Which are the specs of your machine?

[abhiandthetruth]

@valeriocos I think the readme size is becoming quite large. We can refract it. Like creating a troubleshooting.md. WDYT?

[animeshk08]

I think the readme size is becoming quite large. We can refract it. Like creating a troubleshooting.md. WDYT?

I think the same. We can reference it in the current README and create a new one. @valeriocos Are you doing this? I will add the trouble shooting for my issue once you have done in that case :)

[valeriocos]

yes, it sounds good! @abhiandthetruth @animeshk08 do you think that the new md file should contain only troubleshooting info or maybe we can create a getting started md with a troubleshooting section into it? For instance the new fille could have this structure:

• getting started (content from https://github.com/chaoss/grimoirelab-sirmordred#getting-started)
  • setting up Pycharm dev environment (content from https://github.com/chaoss/grimoirelab-sirmordred#setting-up-a-pycharm-dev-environment)
  • setting up docker-compose (content from https://github.com/chaoss/grimoirelab-sirmordred#source-code-and-docker)
  • micro mordred (content from https://github.com/chaoss/grimoirelab-sirmordred#micro-mordred)
• troubleshooting (content from https://github.com/chaoss/grimoirelab-sirmordred#troubleshooting)

please share your thoughts?

[animeshk08]

@valeriocos I think we can have a complete "Getting Started" README file in which troubleshooting section is also included.

[valeriocos]

ok! please @animeshk08 can you kickstart this new getting started readme? Maybe you can add the troubleshooting section plus the problem you found, @abhiandthetruth you could then focus on the content of the first section (getting started). WDYT?

[animeshk08]

@valeriocos Sure I will start with it! :)

[valeriocos]

thanks @animeshk08

[abhiandthetruth]

ok!

[animeshk08]

Hello Everyone, Anyone who is working on Microtask 8, a few issues I came across might help you:

• If you are trying to fetch the message from a group make sure it has "download_archives": true permission activated. This can be checked through this API. For more details check this issue: https://github.com/chaoss/grimoirelab-perceval/issues/586

• Some of the index fields may differ. I found the following: grimoirelab_creation_date ->grimoire_creation_date body->body_extract subject_analyzed -> Subject_analyzed

@valeriocos Correct me if I am wrong anywhere. Also, should I change the logs for groupio in perceval similar to pagure 404 exception?

[valeriocos]

Thank you for reporting this info @animeshk08 . Consider to add it to the troubleshooting section (https://github.com/chaoss/grimoirelab-sirmordred/blob/master/Getting-Started.md#troubleshooting-).

@valeriocos Correct me if I am wrong anywhere. Also, should I change the logs for groupio in perceval similar to pagure 404 exception?

Yes, this will be really useful, thanks!

[animeshk08]

Sure! I will get on with it! :)