evaluation nested too deeply: infinite recursion / options(expressions=)? in find_graphs() when using initgraph

jaurbanChicago commented 10 months ago

Hi, I am trying to run find_graphs() but if I submit an initgraph, I keep getting this error

> find_graphs("./data_pops_qpGraph",initgraph=igraph_to_optimize)
i Reading precomputed data for 13 populations...
i Reading f2 data for pair 91 out of 91...
Error: evaluation nested too deeply: infinite recursion / options(expressions=)?
In addition: Warning message:
In read_f2(dir, pops, pops2, type = type, remove_na = remove_na,  :
  Discarding 1 block(s) due to missing values!
Discarded block(s): 25

Why is this happening? :(

Thanks!

uqrmaie1 commented 10 months ago

Can you share your data and initgraph with me, so I can reproduce the problem? If not, I might be able to figure out what the issue is if you post the full stack trace of the error message.

jaurbanChicago commented 10 months ago

I can share the full stack trace of the error message:

Error: C stack usage  7972624 is too close to the limit
In addition: Warning message:
In read_f2(dir, pops, pops2, type = type, remove_na = remove_na,  :
  Discarding 1 block(s) due to missing values!
Discarded block(s): 25
> traceback()
608: mode(expr)
607: mode(expr) %in% c("call", "expression", "(", "function")
606: deparse(x[[1L]])
605: mode(expr)
604: mode(expr) %in% c("call", "expression", "(", "function")
603: deparse(expr, width.cutoff, ...)
602: paste(deparse(expr, width.cutoff, ...), collapse = collapse)
601: deparse1(substitute(x))
600: force(nm)
599: as.data.frame.numeric(x[[i]], optional = TRUE)
598: as.data.frame(x[[i]], optional = TRUE)
597: data.frame(n = c("ignore", "sum", "prod", "min", "max", "random", 
         "first", "last", "mean", "median", "concat"), i = c(0, 3, 
         4, 5, 6, 7, 8, 9, 10, 11, 12), stringsAsFactors = FALSE)
596: FUN(X[[i]], ...)
595: lapply(comb, function(x) {
         if (!is.character(x)) {
             x
         }
         else {
             known <- data.frame(n = c("ignore", "sum", "prod", "min", 
                 "max", "random", "first", "last", "mean", "median", 
                 "concat"), i = c(0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 
                 12), stringsAsFactors = FALSE)
             x <- pmatch(tolower(x), known[, 1])
             if (is.na(x)) {
                 stop("Unknown/unambigous attribute combination specification")
             }
             known[, 2][x]
         }
     })
594: igraph.i.attribute.combination(edge.attr.comb)
593: igraph::simplify(.)
592: graph %<>% igraph::simplify()
591: simplify_graph(.)
590: graph %<>% simplify_graph()
589: simplify_graph(.)
588: graph %<>% simplify_graph()
587: simplify_graph(.)
586: graph %<>% simplify_graph()
585: simplify_graph(.)
584: graph %<>% simplify_graph()
583: simplify_graph(.)
582: graph %<>% simplify_graph()
581: simplify_graph(.)
580: graph %<>% simplify_graph()
579: simplify_graph(.)
578: graph %<>% simplify_graph()
577: simplify_graph(.)
576: graph %<>% simplify_graph()
575: simplify_graph(.)
574: graph %<>% simplify_graph()
573: simplify_graph(.)
572: graph %<>% simplify_graph()
571: simplify_graph(.)
570: graph %<>% simplify_graph()
569: simplify_graph(.)
568: graph %<>% simplify_graph()
567: simplify_graph(.)
566: graph %<>% simplify_graph()
565: simplify_graph(.)
564: graph %<>% simplify_graph()
563: simplify_graph(.)
562: graph %<>% simplify_graph()
561: simplify_graph(.)
560: graph %<>% simplify_graph()
559: simplify_graph(.)
558: graph %<>% simplify_graph()
557: simplify_graph(.)
556: graph %<>% simplify_graph()
555: simplify_graph(.)
554: graph %<>% simplify_graph()
553: simplify_graph(.)
552: graph %<>% simplify_graph()
551: simplify_graph(.)
550: graph %<>% simplify_graph()
549: simplify_graph(.)
548: graph %<>% simplify_graph()
547: simplify_graph(.)
546: graph %<>% simplify_graph()
545: simplify_graph(.)
544: graph %<>% simplify_graph()
543: simplify_graph(.)
542: graph %<>% simplify_graph()
541: simplify_graph(.)
540: graph %<>% simplify_graph()
539: simplify_graph(.)
538: graph %<>% simplify_graph()
537: simplify_graph(.)
536: graph %<>% simplify_graph()
535: simplify_graph(.)
534: graph %<>% simplify_graph()
533: simplify_graph(.)
532: graph %<>% simplify_graph()
531: simplify_graph(.)
530: graph %<>% simplify_graph()
529: simplify_graph(.)
528: graph %<>% simplify_graph()
527: simplify_graph(.)
526: graph %<>% simplify_graph()
525: simplify_graph(.)
524: graph %<>% simplify_graph()
523: simplify_graph(.)
522: graph %<>% simplify_graph()
521: simplify_graph(.)
520: graph %<>% simplify_graph()
519: simplify_graph(.)
518: graph %<>% simplify_graph()
517: simplify_graph(.)
516: graph %<>% simplify_graph()
515: simplify_graph(.)
514: graph %<>% simplify_graph()
513: simplify_graph(.)
512: graph %<>% simplify_graph()
511: simplify_graph(.)
510: graph %<>% simplify_graph()
509: simplify_graph(.)
508: graph %<>% simplify_graph()
507: simplify_graph(.)
506: graph %<>% simplify_graph()
505: simplify_graph(.)
504: graph %<>% simplify_graph()
503: simplify_graph(.)
502: graph %<>% simplify_graph()
501: simplify_graph(.)
500: graph %<>% simplify_graph()
499: simplify_graph(.)
498: graph %<>% simplify_graph()
497: simplify_graph(.)
496: graph %<>% simplify_graph()
495: simplify_graph(.)
494: graph %<>% simplify_graph()
493: simplify_graph(.)
492: graph %<>% simplify_graph()
491: simplify_graph(.)
490: graph %<>% simplify_graph()
489: simplify_graph(.)
488: graph %<>% simplify_graph()
487: simplify_graph(.)
486: graph %<>% simplify_graph()
485: simplify_graph(.)
484: graph %<>% simplify_graph()
483: simplify_graph(.)
482: graph %<>% simplify_graph()
481: simplify_graph(.)
480: graph %<>% simplify_graph()
479: simplify_graph(.)
478: graph %<>% simplify_graph()
477: simplify_graph(.)
476: graph %<>% simplify_graph()
475: simplify_graph(.)
474: graph %<>% simplify_graph()
473: simplify_graph(.)
472: graph %<>% simplify_graph()
471: simplify_graph(.)
470: graph %<>% simplify_graph()
469: simplify_graph(.)
468: graph %<>% simplify_graph()
467: simplify_graph(.)
466: graph %<>% simplify_graph()
465: simplify_graph(.)
464: graph %<>% simplify_graph()
463: simplify_graph(.)
462: graph %<>% simplify_graph()
461: simplify_graph(.)
460: graph %<>% simplify_graph()
459: simplify_graph(.)
458: graph %<>% simplify_graph()
457: simplify_graph(.)
456: graph %<>% simplify_graph()
455: simplify_graph(.)
454: graph %<>% simplify_graph()
453: simplify_graph(.)
452: graph %<>% simplify_graph()
451: simplify_graph(.)
450: graph %<>% simplify_graph()
449: simplify_graph(.)
448: graph %<>% simplify_graph()
447: simplify_graph(.)
446: graph %<>% simplify_graph()
445: simplify_graph(.)
444: graph %<>% simplify_graph()
443: simplify_graph(.)
442: graph %<>% simplify_graph()
441: simplify_graph(.)
440: graph %<>% simplify_graph()
439: simplify_graph(.)
438: graph %<>% simplify_graph()
437: simplify_graph(.)
436: graph %<>% simplify_graph()
435: simplify_graph(.)
434: graph %<>% simplify_graph()
433: simplify_graph(.)
432: graph %<>% simplify_graph()
431: simplify_graph(.)
430: graph %<>% simplify_graph()
429: simplify_graph(.)
428: graph %<>% simplify_graph()
427: simplify_graph(.)
426: graph %<>% simplify_graph()
425: simplify_graph(.)
424: graph %<>% simplify_graph()
423: simplify_graph(.)
422: graph %<>% simplify_graph()
421: simplify_graph(.)
420: graph %<>% simplify_graph()
419: simplify_graph(.)
418: graph %<>% simplify_graph()
417: simplify_graph(.)
416: graph %<>% simplify_graph()
415: simplify_graph(.)
414: graph %<>% simplify_graph()
413: simplify_graph(.)
412: graph %<>% simplify_graph()
411: simplify_graph(.)
410: graph %<>% simplify_graph()
409: simplify_graph(.)
408: graph %<>% simplify_graph()
407: simplify_graph(.)
406: graph %<>% simplify_graph()
405: simplify_graph(.)
404: graph %<>% simplify_graph()
403: simplify_graph(.)
402: graph %<>% simplify_graph()
401: simplify_graph(.)
400: graph %<>% simplify_graph()
399: simplify_graph(.)
398: graph %<>% simplify_graph()
397: simplify_graph(.)
396: graph %<>% simplify_graph()
395: simplify_graph(.)
394: graph %<>% simplify_graph()
393: simplify_graph(.)
392: graph %<>% simplify_graph()
391: simplify_graph(.)
390: graph %<>% simplify_graph()
389: simplify_graph(.)
388: graph %<>% simplify_graph()
387: simplify_graph(.)
386: graph %<>% simplify_graph()
385: simplify_graph(.)
384: graph %<>% simplify_graph()
383: simplify_graph(.)
382: graph %<>% simplify_graph()
381: simplify_graph(.)
380: graph %<>% simplify_graph()
379: simplify_graph(.)
378: graph %<>% simplify_graph()
377: simplify_graph(.)
376: graph %<>% simplify_graph()
375: simplify_graph(.)
374: graph %<>% simplify_graph()
373: simplify_graph(.)
372: graph %<>% simplify_graph()
371: simplify_graph(.)
370: graph %<>% simplify_graph()
369: simplify_graph(.)
368: graph %<>% simplify_graph()
367: simplify_graph(.)
366: graph %<>% simplify_graph()
365: simplify_graph(.)
364: graph %<>% simplify_graph()
363: simplify_graph(.)
362: graph %<>% simplify_graph()
361: simplify_graph(.)
360: graph %<>% simplify_graph()
359: simplify_graph(.)
358: graph %<>% simplify_graph()
357: simplify_graph(.)
356: graph %<>% simplify_graph()
355: simplify_graph(.)
354: graph %<>% simplify_graph()
353: simplify_graph(.)
352: graph %<>% simplify_graph()
351: simplify_graph(.)
350: graph %<>% simplify_graph()
349: simplify_graph(.)
348: graph %<>% simplify_graph()
347: simplify_graph(.)
346: graph %<>% simplify_graph()
345: simplify_graph(.)
344: graph %<>% simplify_graph()
343: simplify_graph(.)
342: graph %<>% simplify_graph()
341: simplify_graph(.)
340: graph %<>% simplify_graph()
339: simplify_graph(.)
338: graph %<>% simplify_graph()
337: simplify_graph(.)
336: graph %<>% simplify_graph()
335: simplify_graph(.)
334: graph %<>% simplify_graph()
333: simplify_graph(.)
332: graph %<>% simplify_graph()
331: simplify_graph(.)
330: graph %<>% simplify_graph()
329: simplify_graph(.)
328: graph %<>% simplify_graph()
327: simplify_graph(.)
326: graph %<>% simplify_graph()
325: simplify_graph(.)
324: graph %<>% simplify_graph()
323: simplify_graph(.)
322: graph %<>% simplify_graph()
321: simplify_graph(.)
320: graph %<>% simplify_graph()
319: simplify_graph(.)
318: graph %<>% simplify_graph()
317: simplify_graph(.)
316: graph %<>% simplify_graph()
315: simplify_graph(.)
314: graph %<>% simplify_graph()
313: simplify_graph(.)
312: graph %<>% simplify_graph()
311: simplify_graph(.)
310: graph %<>% simplify_graph()
309: simplify_graph(.)
308: graph %<>% simplify_graph()
307: simplify_graph(.)
306: graph %<>% simplify_graph()
305: simplify_graph(.)
304: graph %<>% simplify_graph()
303: simplify_graph(.)
302: graph %<>% simplify_graph()
301: simplify_graph(.)
300: graph %<>% simplify_graph()
299: simplify_graph(.)
298: graph %<>% simplify_graph()
297: simplify_graph(.)
296: graph %<>% simplify_graph()
295: simplify_graph(.)
294: graph %<>% simplify_graph()
293: simplify_graph(.)
292: graph %<>% simplify_graph()
291: simplify_graph(.)
290: graph %<>% simplify_graph()
289: simplify_graph(.)
288: graph %<>% simplify_graph()
287: simplify_graph(.)
286: graph %<>% simplify_graph()
285: simplify_graph(.)
284: graph %<>% simplify_graph()
283: simplify_graph(.)
282: graph %<>% simplify_graph()
281: simplify_graph(.)
280: graph %<>% simplify_graph()
279: simplify_graph(.)
278: graph %<>% simplify_graph()
277: simplify_graph(.)
276: graph %<>% simplify_graph()
275: simplify_graph(.)
274: graph %<>% simplify_graph()
273: simplify_graph(.)
272: graph %<>% simplify_graph()
271: simplify_graph(.)
270: graph %<>% simplify_graph()
269: simplify_graph(.)
268: graph %<>% simplify_graph()
267: simplify_graph(.)
266: graph %<>% simplify_graph()
265: simplify_graph(.)
264: graph %<>% simplify_graph()
263: simplify_graph(.)
262: graph %<>% simplify_graph()
261: simplify_graph(.)
260: graph %<>% simplify_graph()
259: simplify_graph(.)
258: graph %<>% simplify_graph()
257: simplify_graph(.)
256: graph %<>% simplify_graph()
255: simplify_graph(.)
254: graph %<>% simplify_graph()
253: simplify_graph(.)
252: graph %<>% simplify_graph()
251: simplify_graph(.)
250: graph %<>% simplify_graph()
249: simplify_graph(.)
248: graph %<>% simplify_graph()
247: simplify_graph(.)
246: graph %<>% simplify_graph()
245: simplify_graph(.)
244: graph %<>% simplify_graph()
243: simplify_graph(.)
242: graph %<>% simplify_graph()
241: simplify_graph(.)
240: graph %<>% simplify_graph()
239: simplify_graph(.)
238: graph %<>% simplify_graph()
237: simplify_graph(.)
236: graph %<>% simplify_graph()
235: simplify_graph(.)
234: graph %<>% simplify_graph()
233: simplify_graph(.)
232: graph %<>% simplify_graph()
231: simplify_graph(.)
230: graph %<>% simplify_graph()
229: simplify_graph(.)
228: graph %<>% simplify_graph()
227: simplify_graph(.)
226: graph %<>% simplify_graph()
225: simplify_graph(.)
224: graph %<>% simplify_graph()
223: simplify_graph(.)
222: graph %<>% simplify_graph()
221: simplify_graph(.)
220: graph %<>% simplify_graph()
219: simplify_graph(.)
218: graph %<>% simplify_graph()
217: simplify_graph(.)
216: graph %<>% simplify_graph()
215: simplify_graph(.)
214: graph %<>% simplify_graph()
213: simplify_graph(.)
212: graph %<>% simplify_graph()
211: simplify_graph(.)
210: graph %<>% simplify_graph()
209: simplify_graph(.)
208: graph %<>% simplify_graph()
207: simplify_graph(.)
206: graph %<>% simplify_graph()
205: simplify_graph(.)
204: graph %<>% simplify_graph()
203: simplify_graph(.)
202: graph %<>% simplify_graph()
201: simplify_graph(.)
200: graph %<>% simplify_graph()
199: simplify_graph(.)
198: graph %<>% simplify_graph()
197: simplify_graph(.)
196: graph %<>% simplify_graph()
195: simplify_graph(.)
194: graph %<>% simplify_graph()
193: simplify_graph(.)
192: graph %<>% simplify_graph()
191: simplify_graph(.)
190: graph %<>% simplify_graph()
189: simplify_graph(.)
188: graph %<>% simplify_graph()
187: simplify_graph(.)
186: graph %<>% simplify_graph()
185: simplify_graph(.)
184: graph %<>% simplify_graph()
183: simplify_graph(.)
182: graph %<>% simplify_graph()
181: simplify_graph(.)
180: graph %<>% simplify_graph()
179: simplify_graph(.)
178: graph %<>% simplify_graph()
177: simplify_graph(.)
176: graph %<>% simplify_graph()
175: simplify_graph(.)
174: graph %<>% simplify_graph()
173: simplify_graph(.)
172: graph %<>% simplify_graph()
171: simplify_graph(.)
170: graph %<>% simplify_graph()
169: simplify_graph(.)
168: graph %<>% simplify_graph()
167: simplify_graph(.)
166: graph %<>% simplify_graph()
165: simplify_graph(.)
164: graph %<>% simplify_graph()
163: simplify_graph(.)
162: graph %<>% simplify_graph()
161: simplify_graph(.)
160: graph %<>% simplify_graph()
159: simplify_graph(.)
158: graph %<>% simplify_graph()
157: simplify_graph(.)
156: graph %<>% simplify_graph()
155: simplify_graph(.)
154: graph %<>% simplify_graph()
153: simplify_graph(.)
152: graph %<>% simplify_graph()
151: simplify_graph(.)
150: graph %<>% simplify_graph()
149: simplify_graph(.)
148: graph %<>% simplify_graph()
147: simplify_graph(.)
146: graph %<>% simplify_graph()
145: simplify_graph(.)
144: graph %<>% simplify_graph()
143: simplify_graph(.)
142: graph %<>% simplify_graph()
141: simplify_graph(.)
140: graph %<>% simplify_graph()
139: simplify_graph(.)
138: graph %<>% simplify_graph()
137: simplify_graph(.)
136: graph %<>% simplify_graph()
135: simplify_graph(.)
134: graph %<>% simplify_graph()
133: simplify_graph(.)
132: graph %<>% simplify_graph()
131: simplify_graph(.)
130: graph %<>% simplify_graph()
129: simplify_graph(.)
128: graph %<>% simplify_graph()
127: simplify_graph(.)
126: graph %<>% simplify_graph()
125: simplify_graph(.)
124: graph %<>% simplify_graph()
123: simplify_graph(.)
122: graph %<>% simplify_graph()
121: simplify_graph(.)
120: graph %<>% simplify_graph()
119: simplify_graph(.)
118: graph %<>% simplify_graph()
117: simplify_graph(.)
116: graph %<>% simplify_graph()
115: simplify_graph(.)
114: graph %<>% simplify_graph()
113: simplify_graph(.)
112: graph %<>% simplify_graph()
111: simplify_graph(.)
110: graph %<>% simplify_graph()
109: simplify_graph(.)
108: graph %<>% simplify_graph()
107: simplify_graph(.)
106: graph %<>% simplify_graph()
105: simplify_graph(.)
104: graph %<>% simplify_graph()
103: simplify_graph(.)
102: graph %<>% simplify_graph()
101: simplify_graph(.)
100: graph %<>% simplify_graph()
99: simplify_graph(.)
98: graph %<>% simplify_graph()
97: simplify_graph(.)
96: graph %<>% simplify_graph()
95: simplify_graph(.)
94: graph %<>% simplify_graph()
93: simplify_graph(.)
92: graph %<>% simplify_graph()
91: simplify_graph(.)
90: graph %<>% simplify_graph()
89: simplify_graph(.)
88: graph %<>% simplify_graph()
87: simplify_graph(.)
86: graph %<>% simplify_graph()
85: simplify_graph(.)
84: graph %<>% simplify_graph()
83: simplify_graph(.)
82: graph %<>% simplify_graph()
81: simplify_graph(.)
80: graph %<>% simplify_graph()
79: simplify_graph(.)
78: graph %<>% simplify_graph()
77: simplify_graph(.)
76: graph %<>% simplify_graph()
75: simplify_graph(.)
74: graph %<>% simplify_graph()
73: simplify_graph(.)
72: graph %<>% simplify_graph()
71: simplify_graph(.)
70: graph %<>% simplify_graph()
69: simplify_graph(.)
68: graph %<>% simplify_graph()
67: simplify_graph(.)
66: graph %<>% simplify_graph()
65: simplify_graph(.)
64: graph %<>% simplify_graph()
63: simplify_graph(.)
62: graph %<>% simplify_graph()
61: simplify_graph(.)
60: graph %<>% simplify_graph()
59: simplify_graph(.)
58: graph %<>% simplify_graph()
57: simplify_graph(.)
56: graph %<>% simplify_graph()
55: simplify_graph(.)
54: graph %<>% simplify_graph()
53: simplify_graph(.)
52: graph %<>% simplify_graph()
51: simplify_graph(.)
50: graph %<>% simplify_graph()
49: simplify_graph(.)
48: graph %<>% simplify_graph()
47: simplify_graph(.)
46: graph %<>% simplify_graph()
45: simplify_graph(.)
44: graph %<>% simplify_graph()
43: simplify_graph(.)
42: graph %<>% simplify_graph()
41: simplify_graph(.)
40: graph %<>% simplify_graph()
39: simplify_graph(.)
38: graph %<>% simplify_graph()
37: simplify_graph(.)
36: graph %<>% simplify_graph()
35: simplify_graph(.)
34: graph %<>% simplify_graph()
33: simplify_graph(.)
32: graph %<>% simplify_graph()
31: simplify_graph(.)
30: graph %<>% simplify_graph()
29: simplify_graph(.)
28: graph %<>% simplify_graph()
27: simplify_graph(.)
26: graph %<>% simplify_graph()
25: simplify_graph(.)
24: graph %<>% simplify_graph()
23: simplify_graph(.)
22: graph %<>% simplify_graph()
21: simplify_graph(.)
20: graph %<>% simplify_graph()
19: simplify_graph(.)
18: graph %<>% simplify_graph()
17: simplify_graph(.)
16: graph %<>% simplify_graph()
15: simplify_graph(.)
14: graph %<>% simplify_graph()
13: simplify_graph(.)
12: graph %<>% simplify_graph()
11: simplify_graph(graph)
10: eval_tidy(xs[[j]], mask)
9: tibble_quos(xs, .rows, .name_repair)
8: tibble(g = list(simplify_graph(graph)))
7: mutate(., res = map(g, qpgfun), hash = map_chr(g, graph_hash), 
       lasthash = graph_hash(graph))
6: is.data.frame(x)
5: check_data_frame(data)
4: unnest_wider(., res)
3: transmute(., gen2 = 0, hash, g, edges, score)
2: tibble(g = list(simplify_graph(graph))) %>% mutate(res = map(g, 
       qpgfun), hash = map_chr(g, graph_hash), lasthash = graph_hash(graph)) %>% 
       unnest_wider(res) %>% transmute(gen2 = 0, hash, g, edges, 
       score)
1: find_graphs("./data_pops_qpGraph", initgraph = igraph_narasimhan_nobounds)

Thanks a lot Robert! If you find out the data would be more useful, please let me know so I can ask for permission

uqrmaie1 commented 10 months ago

Thanks, that gives me a better idea about what's causing the error. If you don't get permission to share the data, can you share just the initgraph with arbitrarily labelled nodes?

And do you get this error every time you try to run find_graphs() on the data, or only sometimes?

jaurbanChicago commented 10 months ago

Only when trying to use an initial graph. If I run the program without the initgraph parameter, it runs just fine on the same data. They just gave me permission to share both the data and the initgraph file since it's all published data.

jaurbanChicago commented 10 months ago

library(admixtools)
library(tidyverse)
library(igraph)

# This generated the necessary data for find_graphs
extract_f2("Data_HONaka_ALG_posrefine.nohardfiltering.norm.onlybiallelicsnps_forqpgraph",pops = c("Altai_Neanderthal_snpAD.DG","Denisova3_snpAD.DG","Mbuti.DG","BMAC","Western_Steppe_EMBA","Papuan.DG","Dai.DG","Nicobarese","Onge","Indus_Periphery2", "Palliyar","Rajput","Juang"),outdir = "./data_pops_qpGraph",overwrite = TRUE,qpfstats = TRUE)

# This is the graph I want to use as initial graph for find_graphs
graph_narasimhan_nobounds <- read_delim("graph_nobounds_narasimhan.txt",col_names=TRUE,delim="\t")

#Convert to igraph format
matrix_graph_narasimhan_nobounds <- as.matrix(graph_narasimhan_nobounds)
igraph_narasimhan_nobounds <- graph_from_edgelist(matrix_graph_narasimhan_nobounds)

#This is the main command for find_graphs
find_graphs("./data_pops_qpGraph", initgraph = igraph_narasimhan_nobounds)

I am trying to run somethig like this.

uqrmaie1 commented 10 months ago

Thanks for sharing the data and code!

Most admixture graphs functions in Admixtools assume that an admixture graph has no polytomies, or multifurcations (nodes that split into more than two child nodes). There are 4 nodes in your graph with 3 child nodes, and that's what's causing the problem. You could use the function split_multifurcations() to split each of these 4 nodes into two separate nodes. Or, if it's important which split occurs first, you can manually change split those nodes. In functions that allow you to put constraints on individual edges (such as qpgraph(), but not find_graphs()), you can set the length of the newly introduced edges to 0; this will effectively preserve the topology of the original graph.

jaurbanChicago commented 10 months ago

Thanks a lot Robert! I see! I put the polytomies because they are depicted in the graph I am trying to recreate! (Narasimhan et al., 2019). qpgraph_Narasimhan_original . For example, the node prior to Palliyar in the bottom, has three child nodes (two of those child nodes are admixture nodes). I was actually capable of running qpgraph() on my graph and yielded the same topology with a worst_residual of ~4.13 and a score of 96.416 plot_nobounds_qpfstats . But I would like to use this graph as initgraph for find_graphs() and try to optimize the graph a bit more. So, would you recommend running split_multifurcations() and then run the find_graphs()on split graph? Thanks again!

uqrmaie1 commented 10 months ago

So, would you recommend running split_multifurcations() and then run the find_graphs() on split graph?

Yes, that's what I would do.

Both the original and the Admixtools 2 implementation of qpGraph happen to work for graphs with polytomies, even though they weren't designed with that in mind. Having no polytomies is not a big limitation, since there is no differences between a node with three child nodes, and a node with two child nodes where one child's child node has a distance of 0.

If you want to see how small changes to a specific graph affect the fit, find_graphs() might not be the best tool. The original purpose of that function is to find fitting topologies in a completely unconstrained way, by introducing random changes to the graph topology (the initgraph and constrain_ arguments compromise that goal a bit). Some of the functions used by find_graphs() (like those) might be more useful for what you want to do.

I should warn you that graphs as complex as the one you are looking at are most likely overfitted: you can find a lot of very different graphs which fit the data equally well. There isn't a clear cutoff of when a model is too complex, but I haven't seen examples of admixture graphs with more than 6 or 7 populations and more than 3 admixture events for which there aren't completely different alternative models which fit as well or better (see here).

jaurbanChicago commented 10 months ago

Thanks a lot for your help Robert and for pointing out these functions for me! I think they could also be helpful for what I want to do!

jaurbanChicago commented 9 months ago

source_from source_to dest_from dest_to Hi Robert! What do these column names mean ? This is from the output of

newgraphs %>%
  rowwise %>%
  mutate(res = list(qpgraph("./data_pops_qpGraph",graph,return_fstats=TRUE))) %>%
  unnest_wider(res) %>%
  arrange(score)

newgraphs_split_plusone_sortbyworstres <- newgraphs_split_plusone %>% arrange(worst_residual)

jaurbanChicago commented 9 months ago

Hi again @uqrmaie1 ! I got this error when running newgraphs_minusplus()

> newgraphs_minusplus = graph_minusplus(split_graph_narasimhan)
Error in `unnest()`:
! Can't duplicate names between the affected columns and the original data.
✖ These names are duplicated:
  ℹ `source_from` and `source_to`, from `graph2`.
ℹ Use `names_sep` to disambiguate using the column name.
ℹ Or use `names_repair` to specify a repair strategy.
Run `rlang::last_trace()` to see where the error occurred.

uqrmaie1 commented 9 months ago

Thanks for pointing that out, I fixed the problem.

However, the graph_minusplus() function is only useful for very small graphs; for most graphs it will be too slow. It first runs graph_minusone(), which generates all graphs the result from removing an admixture edge, and then on each of those graphs, it runs graph_plusone(), which adds all possible admixture edges. The number of combinations quickly grows very large, and in addition to that it takes very long to identify which of the resulting graphs are unique.

To address your earlier question, the columns in the output of graph_plusone() identify the edges that have been added in terms of the edges they connect to in the original graph. The connected edges themselves are identified by the two nodes they connect. So each added edge is identified by 4 nodes.

jaurbanChicago commented 9 months ago

Thanks @uqrmaie1 !!!!

uqrmaie1 / admixtools

evaluation nested too deeply: infinite recursion / options(expressions=)? in find_graphs() when using initgraph #51