TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'

[btmartin721]

Hi Ekaterina,

I am sorry for bothering you so much. However, I ran into another error when trying to run the moments engine with 3 populations.

TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'

The full traceback can be found at the bottom of the attached *.err.txt file, and my params file and fs file are also attached. Do you know what could be wrong? It seems like the error is occurring during the search when either theta or theta0 is NoneType.

The model seems to run for a while before encountering this error. I have tried using both the optimize_powell and BFGS optimizers.

Thank you for your time.

Here is the end of the traceback:

    return super(MomentsEngine, self).get_N_ancestral(values, dt_fac)
  File "/scrfs/storage/btm002/home/nmt/4.gadma/.gadma2/lib/python3.9/site-packages/gadma/engines/dadi_moments_common.py", >
    return self.get_N_ancestral_from_theta(theta)
  File "/scrfs/storage/btm002/home/nmt/4.gadma/.gadma2/lib/python3.9/site-packages/gadma/engines/dadi_moments_common.py", >
    return theta / theta0
TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'

nmt_gadma_params_K3.txt R-nmt_gadma2_K3_2.217345.err.txt nmt_K3.fs.txt

[noscode]

Dear Bradley,

Do not worry, I am always happy to get feedback.

Your problem seems to be rather unusual. According to the output file, there are warnings from moments saying that it failed to generate some parts of expected SFS. I think that can be the reason why theta is None. I assume that if moments fail to generate whole expected SFS then this theta can easily become None. However, I have never seen it before.

Moreover, I tried to reproduce your issue on my laptop with files you attached. It worked fine... My version of moments is 1.1.15.

One more thing that I can think of: you are using the upper bound on population split time. I am not sure, but it also could potentially cause your error. Could you please set it to None and check if the error will happen then?

Best regards, Ekaterina

[btmartin721]

I will give it a try. I will let you know if it works. Thank you!

[btmartin721]

It is currently running, but the best model seems to have an Nanc of -2:

Run 8 -1881.58 3813.16 [Nanc = -2] [ [ 1 pop split 93.08% (s1) [0.931(s1*_Nanc_size),

Is that out of the ordinary? If so maybe that is what is contributing to the issue.

I am also running moments v1.1.5 and gadma v2.0.0rc26 with python v3.9.6

[noscode]

Wow, that is something unusual and very interesting!

What version of moments are you using? 1.1.5 or 1.1.15?

Could you also find a generated python code for model that is usually located in output_dir/best_logLL_model_moments_code? What is the output if you run this script? Could you please send me the output and this file together, so then I can check if the output will be the same on my computer?

Thank you for all your help to find the reason of this error.

[btmartin721]

Yes, sorry my moments version is 1.1.15.

I have attached the code for the best model here. Note that I changed the extension to .txt because GitHub doesn't allow .py files to be attached here.

best_logLL_model_moments_code.txt

[btmartin721]

Here is the plotted model.

[noscode]

Yes, perfect! The error is now reproducing on my computer! Thank you, I will figure out what is going on and come back to you.

Best regards, Ekaterina

[btmartin721]

Ok thank you! I appreciate your help.

[noscode]

Hi, I have found the reason of this error: moments is not masking entries of SFS if they are negative. I think that old versions of moments did it, but I am not sure. I have to mask them manually, and I pushed a new version in devel branch. Could you please install it from there and check if it works?

Sometimes installation from the repo could cause confusion in GADMA versions, so it is better to uninstall all previous versions before installation of that one.

To install this version: 1) Clone the repo git clone https://github.com/ctlab/GADMA.git 2) Go to GADMA directory: cd GADMA 3) Switch to branch devel: git checkout devel 4) Install version pip install . 5) Check gadma version: gadma --version Version should be 2.0.0rc27.dev3

[btmartin721]

That's great! I will start the run and make sure it works. Thank you!

[btmartin721]

Hi,

Unfortunately the error happened again. I wonder if it is an issue with my SFS. You mentioned that there were negative values in it. I generated it using moments v1.15, but perhaps if I just supply a VCF file to gadma, maybe it will not generate the negative values like moments did?

[noscode]

Hmm, can you please send me the last output of the error? Did you use an upper bound on population split or not?

It is definitely not a problem with your input data. The SFS that I mentioned is the SFS that moments generate from model in order to evaluate likelihood value.

[noscode]

I have also updated a repo with additional precaution that avoids division when theta is None (which is exactly your error). If you have time, you can download the last version of repo, reinstall GADMA from devel branch and run it once more. However, I am not sure what will happen when GADMA will try to print the output: it may cause another error as now ancestral size is allowed to be None. To test that myself, I will need some time.

[btmartin721]

Sure, I have attached the output logs here. I was running a slurm job, so the error is separated from the standard output here. Just in case, I also ran gadma --version just to make sure I was using the dev branch and it showed gadma2 v2.0.0rc27_dev2.

Also yes I did use the upper bound on the population split this time. I will set it to null for future runs. I forgot to change it this time.

gadma2_K3.stdout.log.txt gadms2_K3_stderr.txt

Thank you for being so helpful and responsive to my issue.

[noscode]

I am sorry to bother you so much, but do you still have the output for the last run with error? Could you please send me a generated python code for the best model of run 20: output_dir/20/current_best_logLL_model_moments_code.py? I just noticed that in the output file you sent Nanc is negative for this model, and it should not be like that - I thought I fixed it.

I will also try to ask the authors of moments if they changed something in last versions. I think that it was not a case some time ago.

Thank you very much for all your help and that you still want to use GADMA even though it takes so much time.

Best regards, Ekaterina

[btmartin721]

Here is the python code.

And thank you for all your help! It is much appreciated! Let me know if you need anything else.

current_best_logLL_model_moments_code.txt

[noscode]

Hi, I have released a new version of GADMA, you can install it via pip. It should avoid negative population sizes and solve your error. Please let me know if it works or not.

[btmartin721]

Hi,

Sorry for my delayed reply.

It works now! My runs were able to finish cleanly.

I greatly appreciate your help!

[noscode]

I am glad, you are welcome, thank you for your help and patience! You helped to improve GADMA!