problems running Structure_threader - Structure path problems with "c:\programfiles (x86)\..."

[Sparassus]

Hello.

I am trying to run Structure_threader to use the multi-core capacity put I am stock on this problem.

I have tried solving the path problem with C:\Program~2\ and "c:\programfiles (x86)..." but neither worked as I still get the same error that Structure cannot be found in the specified path...

Do you have a workaround to this issue?

I would really be happy if I could use your solution has right now I have 7 idle cores...

Thanks! Pedro

[StuntsPT]

Hi Pedro, We don't often test Structure_threader on windows, and I'm not sure what the preferred way to provide the paths is on that system. However, you can use an old trick: Just place structure.exe in a directory close to the root, such as C:\structure\structure.exe. When doing this keep in mind that the windows filesystem is not case sensitive, but Python is. I will try to figure that out in a VM later today, but for now try this workaround. You may wish to do the same (use short paths) for any other files.

[Sparassus]

Hi Francisco.

Thanks for the tips.

I think I solved my problem by placing a copy of Structure in a different folder.

I got this error message when running it: (https://cloud.githubusercontent.com/assets/26387469/23896389/27e01ed8-08a1-11e7-9904-13c784f24385.png)

Can you confirm me that this means Structure_threader is now wirking in Windows? I think what may be lacking is the definition of the runs in Structure, which I did not had in this trial folder because I am running a normal Structure run in the folder with my dataset.

What I am trying to confirm is if what is missing is this part, as described in the manual of Struture: "8.6 Batch runs. You can schedule a series of structure runs by going to [Project]→[Start a Job]. This opens up a scheduler that allows you to pick (1) parameter sets (use control+mouse to select multiple parameter sets) (2) values of K, (2) number of runs for each. See Figure 8."

Can you please also elaborate a bit more on how to use klist? My run is from k=4 to K=13, and I can't seem to make klist to work.

Thanks for the help!

Pedro

[StuntsPT]

Hi Pedro. Last night I hit that same problem you describe. We are working to resolve it (it seems to occur only on windows and is related to passing a global variable to the multiprocess module). Hopefully it will be fixed later today. As for your values of K, if you pass the argument "-K 13" all values of K from 1 to 13 will be tested. The "Klist" argument can be used like this: "Klist 4 5 6 7 8 9 10 11 12 13" and it will run K values from 4 to 13 as you wanted. I'll report here once a new release is ready with the bug fixed

[StuntsPT]

I have just fixed that bug with the global variables and the new version is released. Let me know if it has solved your problems.

[Sparassus]

Thanks Francisco!

I will give it a try and let you known how that went.

Pedro

[StuntsPT]

Great. Feel free to open up any more issues that may come up!

[Sparassus]

I think it is now running Francisco. Thanks :)

But I am getting a strange error about permissions.

https://cloud.githubusercontent.com/assets/26387469/23945970/727882c6-0970-11e7-80e3-8d386735167b.png

I have tried to understand it by myself but my knowledge on python is very limited. Can you help me understand if this error results from structure_threader, plotly or miniconda?

If I try to summon plotly I get this error. " 'plotly' is not recognized as an internal or external command, operable or batch file" , which I find strange because I have confirmed that plotly is installed.

Perhaps I need to reinstall miniconda?

Any help would be most welcome!

Pedro

[StuntsPT]

I Pedro, Don't worry about that plotly warning, it's harmless. I am more worried about the errors from structure, though. In order to figure out what went wrong, I'll need some log files that Structure_threaded has generated for you: Inside the directory L:\PedroS\Structure\Buthus_IP_nuDNA_sorted\Buthus_IP_nuDNA_sorted_221\50k500k_noAdx_Corr_inf_lamb\ you will find some files that should have the extension ".stlog". It should say what went wrong. Try to read it's contents and figure out what went wrong, and if you can't, just post it here and I will try to help you figure it out.

[Sparassus]

Good morning Francisco,

Sorry for the delay but I stopped having access to the PC were I am running Structure.

There is noting inside those .stlog files.

But I have noticed that when Structure_threaded starts running I get a pop up window named "Structure" with the error: failure to start structure: cannot locate class file

Is this happening because I create a copy of Structure in a different location from the Windows install due to the problem in the path name?

Pedro

[StuntsPT]

Hi Pedro, The issue is because you are pointing the -st switch to the GUI frontend of STRUCTURE. Somewhere inside the structure directory you should find another "structure.exe" file that is the command line version that Structure_threader requires to run. According to your last screenshot, you should find the CLI version of STRUCTURE in this path L:\PedroS\Structure234clone\bin\structure.exe. If you don't find that ".exe" you can download the CLI version from the author's website Let me know if that works for you.

[Sparassus]

Francisco that resolved the problem, thanks.

But know I get the error: Can't open the file "mainparams"

I don't understand why because in my command line the -o path leads to the folder were the Structure project is, and I have created a mainparams in Structure that is also in that same folder, with the "Generate parameter files" option

Does this make sense to you?

Pedro

[StuntsPT]

Try using the option --params to tell STRUCTURE where your mainparams file is. I think you're getting close now.

[Sparassus]

:) yes, that solved the mainparams file Francisco.

But now in the log file the error is that it cannot find the "extraparams" file.

I had not generated this file. But I do not see any argument in Structure_threaded for it either.

What am I missing now?

Pedro

[StuntsPT]

I could be wrong, but I thought that the extraparams was a requirement for structure. In fact, it is in that file that you define important parameters such as "use admixture". In this case you can pick up a default "extraparams" file and place it in the same directory as your "mainparams" file and STRUCTURE should pick it up there too.

[Sparassus]

I believe you are right Francisco.

But the Structure version I am using (2.3.4) is creating empty "extraparams" files for my different runs (which run without problems)

When I use Structure to generate the parameters it only creates mainparams files. But they do have the information that should be in the "extraparams" file.

I am starting to think this is due to de fact that I am using the GUi interface of Structure, and not the command line. Could it be so?

Pedro

[StuntsPT]

I think the GUI version creates both the mainparams and extraparams file. Try placing an empty "extraparams" file in the same dir as "mainparams". That should keep the software happy. I'll open an issue here to try to work around that problem.

[StuntsPT]

I have opened #52 to deal with the missing parameter file. In the meantime, does using an empty file work?

[Sparassus]

Not sure Francisco.

This is the command line am using: structure_threader run -Klist 4 5 -R 1 -i L:\PedroS\Structure\Buthus_IP_nuDNA\project_data -o L:\PedroS\Structure\Buthus_IP_nuDNA_sorted\Buthus_IP_nuDNA_sorted_221\ST_test6 -t 6 -st L:\PedroS\console\structure.exe --params L:\PedroS\Structure\Buthus_IP_nuDNA_sorted\Buthus_IP_nuDNA_sorted_221\ST_test6 --log 1

I get this error in the svlog file: Can't open the file "L:\PedroS\Structure\Buthus_IP_nuDNA_sorted\Buthus_IP_nuDNA_sorted_221\ST_test6"

But this a path in -o argumnet, not a file, correct?

[StuntsPT]

Try like this: structure_threader run -Klist 4 5 -R 1 -i L:\PedroS\Structure\Buthus_IP_nuDNA\project_data -o L:\PedroS\Structure\Buthus_IP_nuDNA_sorted\Buthus_IP_nuDNA_sorted_221\ST_test6 -t 6 -st L:\PedroS\console\structure.exe --params L:\PedroS\Structure\Buthus_IP_nuDNA_sorted\Buthus_IP_nuDNA_sorted_221\ST_test6\mainparams --log 1

[StuntsPT]

Ok, the extraparams problem is fixed via 2b8df5aecc41540b1a2bb2aeedf3da1a55fd76e9 and it should be available in the next release. Creating an empty extraparams file will work around the issue if you can't use the latest git. (I should make a new release tonight)

[StuntsPT]

Version 0.4.1 is released both on github and on Pypi. Should take care of the extraparams issue.

[StuntsPT]

Hi @Sparassus , is this still an issue for you? Or can it just be closed? Thanks.

[StuntsPT]

Closing, since the problem seems to have been solved a long time ago.