Error running RunTest_Simulink.bat

[jiakangqiang]

Dear ValentinChb, My FAST.Farm runs normally, but the following error was reported when running RunTest_Simulink.bat. May I ask what potential problem might lead to my error? (I am new to code and learned about this project to build a controller co-simulation with FAST.Farm in simulink.) Best regards, jiakangqiang

[ValentinChb]

Dear jiakangqiang,

These are common OpenFAST warnings (nothing to do with the connection to Simulink). The upper warnings are due to your domain being too small to encompass your specified number of wake planes as they advect downstream, but this is of no consequence. The lower warnings might just be transients at the start of the simulation, if the simulation does not crash and the controller does not shut down the turbine (pitch to 90 degrees, 0 power), this is of no consequence. I suggest you read OpenFAST documentation and take a look at NREL's FAQ forum.

When that is said, if you are new to code, coupling FAST.Farm with Simulink might be an overly ambitious endeavour. This repository provides the necessary information assuming some knowledge about the various protocols and coding languages. This is not an official NREL product, and user support will be limited.

BR

Valentin

[jiakangqiang]

Dear ValentinChb, Thank you for your positive reply! I understood that my above problems are the problems of FAST.Farm itself, so I consulted the official documents and simplified FAST.Farm_N3.fstf to some extent, and then successfully ran it in powershell window. Now I have a new problem, I run RunTest_Simulink.bat in the Windows cmd, where FAST.Farm works successfully (as above), but SC_Simulink doesn't seem to work, nor does Mex_Simulink_Blocks.slx. I try to run SC_Simulink directly in the directory, but the result is that matlab is always busy, is there a problem with my matlab? (All file paths have been added to matlab) Finally, I appreciate your concern about the work I will do. I understand that the co-simulation of matlab and FAST.Farm is a difficult project, but I am determined to conduct the research on wind farm control based on FAST.Farm, so your timely help is very important to me, so I hope you can find time to reply to my questions in your busy schedule. Best regards, jiakangqiang

[jiakangqiang]

Dear ValentinChb, I recently conducted a further investigation of the above problems, and found that MATLAB entered endless busy when running this line of code, may I ask how this situation is going? (At the same time, my Powershell window can open normally but MATLAB still can not open normally, the above is the problem I found when running SC_Simulink manually) I sincerely hope that you can take time out of your busy schedule to reply me.

Best regards,

jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

You are encountering this issue because MATLAB does not manage to initialise the MPI connection with FAST.Farm. Some understanding of the internals of the communication are needed to fix this. Here is a short summary: The function MPIServer_Init is implemented in a mex function, compiled from MPIServer_Init.cpp. This in turns calls a fortran function of the same name from SC_MPIServerSubs.dll compiled from MPIServerSubs.f90. If activated (true by default), this routine outputs status info to the file stdout_MPIServerSubs.txt. Based on the content of this file, you should be able to find out whether the communication is locked on the server or on the client side. If you don't know what the client is, take a look at my replies to the other issues in this repo.

BR,

Valentin

[jiakangqiang]

Dear ValentinChb, Thank you very much for your guidance! I have reviewed your answers to the relevant questions and found the relevant MPI communication file, and it seems that the communication is normal. The only problem seems to be that the MPI_shared.dat path instructed in SC_input is in the previous directory, but in fact MPI_shared.dat is in the previous directory of SC_input (but this does not seem to affect its normal communication). So what documents should I continue to check? Once again, thank you for your patience with me, I wish you all the best! Best regards,

jiakangqiang

[ValentinChb]

Dear JiakangQiang,

I am not sure I understand your issue with SC_input and MPI_shared. The path given in SC_input is relative to the working directory, which should be that of the .fstf file. so ../MPI_shared.dat means that the file is located one folder up from the .fstf file in the path tree.

Indeed the client (turbine controller dll) seems to have correctly established communication with the server (Matlab). It is then waiting for a control message from the server to proceed to the next time step. So everything looks ok on the client side. You should then look at the status on the server side in stdout_MPIServerSubs.txt, and understand why no message is sent to the client. You may aslo take a look at stdout_SCClientSubs.txt from the other turbines to see if they, as turbine 1, have sent their first message and are waiting for a response from the server.

BR,

Valentin

[jiakangqiang]

Dear ValentinChb,

Now I understand what you said about the path of MPI_shared.dat, It has been set up correctly; Waiting for control message might be a reason that the MATLAB entered endless busy when running that line of code. I checked the files according to your instructions, They all appear to be working successfully:

Look forward to your new guidance.

Best regards,

jiakangqiang

[jiakangqiang]

Dear ValentinChb,

I am still working hard to build a wind farm controller in Simulink based on FAST.Farm (I manually ran Mex_Simulink_Blocks.slx and understood the parameter communication process), but I am still stuck in running SC_Simulink.

I'm sure that after I run RunTest_Simulink.bat, the SC_Simulink that pops up doesn't execute everything when it says "Ready to connect", it's still stuck in the next step of disp("Ready to connect") : [nT,nc]=MPIServer_Init(MPI_sharedfile); And at this point Matlab is busy indefinitely, as I've shown you before (when I run it manually, Matlab doesn't even start when I run RunTest_Simulink.bat directly).

Therefore, I found that the Powershell window would run Fast.Farm successfully even if I chose to proceed with RunTest_Simulink.bat when "Ready to connect" was not displayed.

I have tried to check the documents related to MPI communication according to your instructions, but the problem still hasn't been solved. I am eagerly waiting for your guidance or any other way to achieve my research purpose.

Best regards,

jiakangqiang

[ValentinChb]

Dear JiakangQiang

Comparing the content of your stdout_MPIServerSubs.txt with the one provided in the repository, it looks likes the server never comes to the point that it prints "Connection established with communicators: [...]", i.e. it is stuck at line 92 "call MPI_COMM_ACCEPT" in MPIServerSubs.f90. However, the client does connect to some server, as shown in the stdout_SCClientSubs.txt file.

At this point I would:

• check if the port name in stdout_MPIServerSubs.txt is the same as the first line in MPIshared.dat (the server is supposed to export it there for reading by the client)
• try printing out the communication ID client_comm (i.e. add "write(11,*) client_comm" in MPIServerSubs.f90 and recompile) and check if it matches with the ID printed by the client (second line in MPIshared.dat).

If these communication details match, I would suspect some firewall setting preventing the communication from being established. You may check your firewall activity and rules.

If they don't match, the client is connecting to a different server than your current Matlab/Simulink session. That happens typically when things are not run in the right order: the client tries to connect before the server has actually initialised and updated MPIshared.dat, and the previous MPI_COMM_ACCEPT attempt from the previous Matlab/Simulink launch is still running in the background because unproperly terminated, so the client connects to this one. Can you try re-starting your PC and looking at the files (stdout_MPIServerSubs.txt, stdout_SCClientSubs.txt for turbine 1 and MPIshared.dat) after the first try?

BR,

Valentin

[jiakangqiang]

Dear ValentinChb,

I quickly confirmed that stdout_MPIServerSubs.txt and the first line of MPIshared.dat are the same:

But I can't find the second line in MPIshared.dat. It only has one line. I also checked the computer's firewalls, and they're all off

In addition, where exactly should I add "write (11, *) client_comm" in MPIServerSubs.f90?

Best regards,

jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

MPIshared.dat should have 5 lines. The first is written by the server, the others by the client. If it has only one, the client has not updated it. That conflicts with your first observations that the client actually connected. So either it has actually not connected, or it uses another MPIshared.dat file. What you may try is specifying absolute paths (instead of relative) for MPIshared.dat in SC_input.dat (for the client) and in SC_Matlab.m/SC_Simulink.m (for the server).

When you say that FAST.Farm would actually connect without Matlab being ready, I'm really thinking a connection with a previous session is going on. Could you restart your machine and give the content of the files as suggested in my previous answer? And the last time modified for each file?

Regarding where to put "write (11, *) client_comm" in MPIServerSubs.f90, I do hope you can find this out yourself, else I fear coupling FAST.Farm and Simulink is beyond your programming abilities. But if you don't feel like setting up a fortran compiler now, check out the other debugging tracks first.

BR,

Valentin

[jiakangqiang]

Dear ValentinChb,

Thank you for your patience. I restarted the machine as you said, and the result found that the modification date of stdout_SCClientSubs.txt did not match the other two log files, which verified your guess, what is the cause of this?

And stdout_MPIServerSubs.txt is still the same as the first line MPIshared.dat (MPIshared.dat still has only one line).

What should I do now to ensure that the client is communicating properly?

Best regards,

jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

Right, the client stdout file has not been updated for three weeks, which is about the date of your first post, so I presume the client has actually never been called. In your second post https://github.com/ValentinChb/FASTFarm2Simulink/issues/3#issuecomment-1972839999, I can see from your screenshot that ServoDyn is not initialised, and you said that you simplified FAST.Farm, did you deactivate ServoDyn or the call to SCClient.dll somehow? I updated the Readme for the repository in the meanwhile, I suggest you read it again to gain some more understanding.

BR,

Valentin

[jiakangqiang]

Dear ValentinChb,

Thank you for your guidance! I did turn off CompServo to 0 in the fst file while simplifying the model, and when I re-set it to 1, the SC_Simulink window started trying to run MATLAB, but ended up with an error:

Meanwhile, the Powershell window had the initial problem:

In addition, the dates of the individual log files have been updated synchronously and the correct content appears：

Is this issue highly relevant to CompServo? Or can I try to simplify something else? Looking forward to your instructions.

Best regards,

jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

Switching CompServo to 0 deactivates control, so linking OpenFAST/FAST.Farm to Simulink becomes pointless... I would say you should become familiar with how a wind turbine controller and its implementation in OpenFAST works before going any further. Also, read the readme for this repository and issues posted by others for linking to DTUWEC or ROSCO #2. If you use Matlab/Simulink for wind turbine control with the template files provided without implementing your own controller, your turbine will basically be out of control and the simulation will crash. I won't provide support at this level of understanding.

I will focus on the MPI communication. Is the content of stdout_MPIServerSubs.txt the same as before, i.e. stopping at printing the port name without establishing communication? Then this means MPI_COMM_ACCEPT fails, making SC_MPIServerSubs.dll and hence MPIServer_Init.mex64 crash, causing the error you see in Matlab.

Kind Regards,

Valentin

[jiakangqiang]

Dear ValentinChb,

Thank you for your reply. The stdout_MPIServerSubs.txt content still never reached print "Connection established with communicators: [...] ".

At the same time, the ID in stdout_SCClientSubs.txt matches the second line in MPIshared.dat.

As for the establishment of the controller, my expectation is to build a controller for the wind farm on simulink (before that, I have completed the layout of the controller on the simulink platform with a stand-alone model provided by OpenFAST).

Therefore, my biggest problem now is that I really don't know much about the communication between FAST.Farm and simulink. Your suggestions will be of great help to me, and I am trying to understand these problems.

Best regards,

Jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

I have added debugging info for the MPI connection in MPIServerSubs.dll, in hope this will help identifying the problem. Can you pull changes from the repo, try running again and share the content of stdout_MPIServerSubs.txt?

Kind Regards

Valentin

[jiakangqiang]

Dear ValentinChb,

I ran the newly provided file and got the extra logs in stdout_MPIServerSubs.txt. The connection established with communicators is consistent with stdout_SCClientSubs.txt and MPIshared.dat：

Best regards,

Jiakangqiang

[ValentinChb]

Ok, then the MPI communication is actually established. Where does it stop now?

[jiakangqiang]

Just stop in the same position:

[ValentinChb]

With Matlab still showing the same (with the new MPI debug info, MPI functions should return an error code but not make the calling program crash)?

[jiakangqiang]

Unfortunately yes：

[ValentinChb]

Is it still MPIServer_Init or is it MPIServer_AnyReceive that crashes in SC_Matlab.m? I am not sure but I would suspect incompatibility between Matlab versions, so you should try recompiling the mex files.

[jiakangqiang]

Dear ValentinChb,

It is still MPIServer_Init.

As for the version problem you mentioned, I tried to recompile all cpp files, and found that MPIServer_Stop.cpp and MPIServer_Init.cpp could not be successfully compiled (the other two could be successfully), I am still dealing with the error information, thank you for your support!

Best wishes,

Jiakangqiang

[jiakangqiang]

Dear ValentinChb,

When I recompiled these four cpp files, I found that MPIServer_AnyReceive.cpp and MPIServer_OneSend.cpp could be recompiled successfully, but MPIServer_Init.cpp and MPIServer_Stop.cpp could not be successfully compiled. Could it be a problem with the version?

Here is the Matlab error:

Best wishes,

Jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

unfortunately I have little experience with mex files (this was the part of Coen Jan Smits in his master thesis), so here stops my ability to help you... are you sure the Matlab compiler is properly set (run mex -setup cpp)? Your build command should look like
mex "Mex Function C++ Code/MPIServerInit.cpp" -L . -lSC_MPIServerSubs when called from the SC_MPIServer directory, linking to libSC_MPIServerSubs.a located in current directory (.). Strange that it works for some files though.

Kind regards,

Valentin

[jiakangqiang]

Dear ValentinChb,

I've been working on recompilation issues recently, and I've noticed that MPIServer_Init.cpp and MPIServer_Stop.cpp (the other two don't) both contain functions defined in MPIServerSubs.f90, So I need to compile MPIServerSubs.f90 into MPIServerSubs.o, and then combine MPIServer_Init.cpp (same for MPIServer_Stop.cpp) with MPIServerSubs.o.

But I keep getting errors along the way (even though I make sure all the library files are in the right place), I guess maybe it's the wrong version of the compiler like gfortran or MinGW64?

So if I'm lucky you can provide me with a compiler version that compiles MPIServerSubs.f90 and MPIServer_Init.cpp, I think my problem should be solved.

Best wishes,

Jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

MPIServer_OneSend.cpp and MPIServer_AnyReceive.cpp should call the corresponding functions in MPIServerSubs.f90 ( not sure why they are commented out). I use gcc version 10.1.0 in Msys2/MinGW64. But compiling the source code should be quite version-independent... I do not exactly know where your are getting errors and how they look like (though for instance I don't see why you need MPIServerSubs.o when you should be linking to the MPIServerSubs dll in the mex command in Matlab), but I think this is a basic building/linking issue that you should be able to solve by yourself starting with basic tutorials and examples.

Kind Regards,

Valentin

[jiakangqiang]

Dear ValentinChb,

I've been working on compilation issues for the last few days, and I've learned more about the specific communication logic. Fortunately MATLAB will no longer be endlessly busy when running SC_Simulink, it will now report a definite error on the specified line of MPIServer_Init, now I am not quite sure how this error should be resolved. In addition, the update times of the individual log files are correct, and the output of the Powershell window is the same as before. (You can ignore the initial warning that the file name was changed separately when I tried to compile, and the current folder has not changed)

I hope I will be lucky enough to continue to have you continue to support me in the debugging process. I wish you all the best.

Best wishes,

Jiakangqiang

[ValentinChb]

Dear Jiakangqiang,

Unfortunately, these are Matlab-specific issues which I have only little knowledge about. It seems you are having issues with printing out error handling in Matlab. Have you successfully managed to create and run a simpled mex-file (tutorial)? Then you should try to debug by printing out step by step progression in the .cpp. When that is said, printing out error handling only occurs if you do have an error when calling the MPIServer_Init function from the dll, so there is more than pure .mex file stuff going wrong. Whan does stdout_MPIServerSubs.txt say?

BR,

Valentin