Changes made in Dashboard is not seen after docker-compose down command

[shikha0208]

Hi ,

I am using https://github.com/chaoss/grimoirelab/blob/master/docker-compose/docker-compose.yml for the execution of dashboard.

Changes made by me in the dashboard and saved with different name is not seen in the next execution. Steps:

First i executed docker-compose up -d Made changes and saved the dashboard Then docker-compose down command Last docker-compose up -d command Not able to see the changes

With docker-compose restart , it is available but why not it is working without restart command? May i know where this data is getting saved for kibana, can i export data into json or is there any way to store the data permanently as if someone uses docker-compose downcommand then all the data will be vanished.

Thank you

[zhquan]

Hi @shikha0208,

With docker-compose restart , it is available but why not it is working without restart command?

When you execute docker-compose down will remove everything. You don't need to recreate all containers every time when you change the Mordred configuration file (setup.cfg). If you make changes in Mordred you only have to restart the Mordred container docker restart <mordred_ID> or if you prefer you can recreate only the Mordred container, but you don't need to recreate mariadb, kibiter, hatstall, and elasticsearch.

If you want to use docker-compose execute stop instead of down this way will not remove anything. More info at:

• https://docs.docker.com/compose/reference/stop/
• https://docs.docker.com/compose/reference/down/

May i know where this data is getting saved for kibana, can i export data into json or is there any way to store the data permanently as if someone uses docker-compose down command then all the data will be vanished.

If you want your indexes and dashboard persistent you have to add a volume for /elasticsearch/data file and those files will store on your local. Make sure the permission of your local path <your_path>/elastic_search-index/ is accessible by docker. If you don't know what permission has docker you can set 0777.

    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.6
      command: elasticsearch -Enetwork.bind_host=0.0.0.0 -Ehttp.max_content_length=2000mb
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g
        - ANONYMOUS_USER=true
      volumes:
        - <your_path>/elastic_search-index/:/elasticsearch/data

I hope it helps you.

It would be great if you could add a new Q&A to the How to section in elasticsearch about this issue and solution.

Best, Quan

[shikha0208]

Hi @zhquan ,

Thank you zhquan for the explanation. Yes sure i will add this in Q&A to the How to section in elasticsearch , as i am successful with the solution.

As i want to store my created data ,so as you said i need to create volume that means i just have to change the docker compose yml file with the volume field as below:

elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.6
      command: elasticsearch -Enetwork.bind_host=0.0.0.0 -Ehttp.max_content_length=2000mb
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g
        - ANONYMOUS_USER=true
      volumes:
        - ../default-grimoirelab-settings/elastic_search-index/:/elasticsearch/data

Is this right?i have given the same local path where setup files and json file exists. So container has access also there. Do i have to create individually any volume file?Or just this file with this local location .I have just created a file named "elastic_search-index" type :file in the location../default-grimoirelab-settings/elastic_search-index

https://user-images.githubusercontent.com/63613092/87430417-7e938080-c5e5-11ea-9b6a-534aa646d04c.png

Also elastic_search-index/ is this slash needed before this : in the above file.

Commands: Made changes docker-compose down docker-compose up -d Observations: Local file is not loaded with any data and in the dashboard also no changes are saved. Kindly correct me.

[zhquan]

Hi @shikha0208,

Sorry the path is /usr/share/elasticsearch/data instead of /elasticsearch/data.

      volumes:
        - ../default-grimoirelab-settings/elastic_search-index/:/usr/share/elasticsearch/data

Create first the directory ../default-grimoirelab-settings/elastic_search-index in your local and make sure docker can access that directory with write permission.

Best, Quan

[shikha0208]

Hi @zhquan ,

I have changed the path 👍

 elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.6
      command: elasticsearch -Enetwork.bind_host=0.0.0.0 -Ehttp.max_content_length=2000mb
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g
        - ANONYMOUS_USER=true
      volumes:
        -  ../default-grimoirelab-settings/elastic_search-index/:/usr/share/elasticsearch/data

=>Directory is created and elastic_search-index file is also made. =>How can i make sure about 0777 write access to file using docker compose yml (Windows 10 , command prompt/powershell) I tried searching but i was not able to find the right way , do i have to write anything in the compose file?

As the index file is still 0kb and i execute docker-compose down and then docker-compose up , hence changes are lost.

``

[zhquan]

Hi @shikha0208,

You have to change the permission before starting the docker-compose. Go to the directory ../default-grimoirelab-settings/elastic_search-index/ and add write, and read permission for Windows I think are Read, Write, and Modify. The permission 0777 is the same if you allow all permissions $ chmod 0777 <file> (Linux). I found this for Windows https://www.windowscentral.com/how-take-ownership-files-and-folders-windows-10

If docker doesn't have read and write permissions it will not able to copy there the content and you will lose your changes whenever you execute docker-compose down

Best, Quan

[shikha0208]

Hi @zhquan ,

I went to the directory and clicked on the file and performed the steps for the write access. But still nothing is copied and changes are lost.

By anything i can verify that docker has access to write on index file in windows ?

Note :I gave access to docker-users in the file.

[zhquan]

Hi @shikha0208,

You can enter into the Mordred container and try to create a file.

Steps:

1. Enter Mordred container: docker exec -it <Mordred_ID> /bin/bash
2. Go to elasticsearch/data: cd /usr/share/elasticsearch/data 2.1 If the path is wrong you will see bash: cd: /usr/share/elasticsearch/data: No such file or directory 2.2 Search the directory: find / -print | grep -i 'elasticsearch/data' in my case the directory is /usr/share/elasticsearch/data see example:

   # find / -print | grep -i 'elasticsearch/data'
   /usr/share/elasticsearch/data
   /usr/share/elasticsearch/data/nodes
   /usr/share/elasticsearch/data/nodes/0
   /usr/share/elasticsearch/data/nodes/0/node.lock
   /usr/share/elasticsearch/data/nodes/0/_state
   /usr/share/elasticsearch/data/nodes/0/_state/global-0.st
   /usr/share/elasticsearch/data/nodes/0/_state/node-0.st

   2.3 Change the path - ../default-grimoirelab-settings/elastic_search-index/:<new_path>

3. Try to create a file: touch test_file 3.1 If you have no permission you will see: touch: cannot touch 'test_file': Permission denied
4. Remove the file: rm test_file

Best, Quan

[shikha0208]

Hi @zhquan ,

After i changed the docker compose file and executed , i get this error: https://user-images.githubusercontent.com/63613092/87559089-1fec0680-c6ba-11ea-95f2-1d9ce114183b.png

How should i resolve this and run, so that i check containers.

[zhquan]

Hi @shikha0208,

After i changed the docker compose file and executed

What change?

https://user-images.githubusercontent.com/63613092/87559089-1fec0680-c6ba-11ea-95f2-1d9ce114183b.png

It seems the path does not exist, check the path first. I see you are using the path ...../merged/usr/share/elasticsearch/data

PS: I don't know how Windows handle paths

[shikha0208]

Hi @zhquan ,

I changed to volumes: - ./Elasticsearch/data/elastic_search-index/:/usr/share/elasticsearch/data It is not showing error with the above path , is that fine ?

I have given full control for the folder (777).

Next i went inside the mordred container,

cd /usr/elasticsearch/data
bash: cd: /usr/share/elasticsearch/data: No such file or directory

$ find / -print | grep -i 'elasticsearch/data'
find: ‘/var/cache/ldconfig’: Permission denied
find: ‘/var/cache/apt/archives/partial’: Permission denied
find: ‘/var/lib/apt/lists/partial’: Permission denied
find: ‘/tmp/tmplvefwc1z’: Permission denied
find: ‘/proc/tty/driver’: Permission denied
find: ‘/root’: Permission denied
find: ‘/etc/ssl/private’: Permission denied

With the new path also , it is denied.Strange i gave full control for the index file. Is something else also i need to give permissions?Where am i going wrong.

I tried lot yesterday ,It is not working out.As in your case , it is not denied.Could you please check on this.

[shikha0208]

Hi @zhquan ,

I found out the problem , now i am able to save the changes to the dashboard. Solution I created a volume in the docker compose yml file at the services level with the same name i mounted the container.

services:
   elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.6
      command: elasticsearch -Enetwork.bind_host=0.0.0.0 -Ehttp.max_content_length=2000mb
      ports:
        - 9200:9200
      environment:
        - ES_JAVA_OPTS=-Xms2g -Xmx2g
        - ANONYMOUS_USER=true
      volumes:
        - <volumename>:/usr/share/elasticsearch/data
volumes:
  <volumename>

I will write this in Q&A to the How to section in elasticsearch about this issue and solution.

Thank you.

[shikha0208]

Hi @zhquan

I have written in the How to section in elasticsearch,i get this page.

https://user-images.githubusercontent.com/63613092/87660087-c7bb1000-c75e-11ea-90d3-8fa7ef3ac0ff.png

[shikha0208]

Hi @zhquan ,

Issue is resolved with the above solution. Closing the issue.

Thank you