search

tudngn / swarm-q-learning

The code for Swarm Q-learning method introduced in research paper: "Swarm Q-learning With Knowledge Sharing Within Environments for Formation Control" in 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
MIT License
4 stars 1 forks source link

issue #2

Open djm0539 opened 3 years ago

djm0539 commented 3 years ago

why fail to importdata cant open file

fail to world_init_swarm file = importdata(filename);

fail to SQL [NumberStateRows,NumberStateCols,flag_num,agent_num,obstacle_num,flag_position,obstacle_position,agent_position,ROI_area] = world_init_swarm(str1,str2);

fail to main SQL('50','4')

I have study sometimes，please teache me ，thanks

tudngn commented 3 years ago

The world_init_swarm.m will initialize the world with "str1" - size of environment, and "str2" - number of agents/robots. You cannot import the file because there is no parameter file of the environment yet (the positions of the agents and obstacles, etc.)

  1. First you need to run the script: "create_environment.m" in the parent directory.

Now, before running "create_environment.m", you may notice a line:

filename = strcat('../data/environmentswarm',num2str(sizeworld),'',num2str(agent_num),'.mat'); This specifies that the information of the environment will be saved to a .mat file in the Folder called "data". You need to create a folder named "data". Then run the code "create_environment.m"

You can manually edit the script to generate the number of agents and size of environment, and even the locations of the agents and obstacles as you like.

  1. When you run the main.m code in SQL folder, just make sure that:
    • SQL(str1, str2): str1 and str2 should be the correct environment size and number of agents, respectively. For example, str1='50' and str2='9' are for environment with size of 50 and 9 agents.
    • The directory in 'world_init_swarm.m': filename = strcat('../../data/environmentswarm',str1,'_',str2,'.mat'); You need to make sure that the directory of the .mat file created above matches the declaration.

I think the main reason why you cannot run the main code is because there is no environment file (.mat) yet. And also, you need to change the code where you declare the location of the files so that they match with each other. This you need to figure out on your own as you may have configured the directories different from mine.

If you face further issue, please tell me.

djm0539 commented 3 years ago

dear teacher I have get all data under your guidance now I want to generate the image by ScalableAnalysis.m and sensitivity_analysis.m but i get a blank imagine so i want to get the solution do i need to create a folder for files_share_9 and Folder_ROI？

djm0539 commented 3 years ago

dear teacher I have get all data under your guidance now I want to generate the image by ScalableAnalysis.m and sensitivity_analysis.m but i get a blank imagine so i want to get the solution do i need to create a folder for files_share_9 and Folder_ROI？

tudngn commented 3 years ago

You need to create a folder for files_share_9. you can comment out the code related to Folder_ROI as you possibly do not need it.

I am not sure what problem you are facing. Can you provide some screenshot of the errors that you face and the project directory.

Maybe you can comment out the code lines that involve the use of data like share_4 or share_16 if you do not have them at the moment. The image will then show the figure of the files_share_9 only. Again I am not sure what issues you face until I see what happens.

djm0539 commented 3 years ago

I have sent you an email there are two picture

djm0539 commented 3 years ago

hello teacher. I have run your code completely . First of all, thank you for solving my doubts ， Secondly, I would like to ask a question. If I want to join a formation-containment controller ,I need to add it to which folder or what would need to be changed ，And it would be better if you can provide some source of code about this controller . Looking forward to your response

tudngn commented 3 years ago

Sorry that I have not responsed promptly to your previous question. It's wonderful that you can figure out and run the code successfully.

Regarding your question, I am not sure what your question is. Are you asking how to build a formation-containment controller with the idea of this code? Or are you asking about how to implement this code as a component of the formation-containment controller that you are working on?

Honestly, I do not have much experience in designing a formation-containment controller. This code is an initial study on how to use Reinforcement Learning to form a specific formation. Now, this means that it has a lot of potential to actually deploy in a real system to create a formation to do a lot of applications. However, I also believe that this code is not mature enough to actually be applied in a formation-containment controller yet.

This does not mean it is not possible though. There are a way to apply this algorithm to solve such problem. The SQL algorithm in this github is for the agent to form a shape that the user designs. For a full formation-containment problem:

  1. You can design some algorithms for one/multiple searching robots to identify the positions of the target robots that need to be contained.
  2. Generating the locations/vertices of the formation/shape that contains the target robots.
  3. Using the SQL algorithm in this to lead the searching robots to the vertices of the shape/formation to complete the containment. I think that the algorithm in this github repo might help you to do the 3rd step. You need to find algorithms to solve the 1st and 2nd steps. Hope this help.