HTTP response code 502 for phylographter nexson export

[josephwb]

Calling up the list of updated studies from treemachine returns:

{"from": "2013-03-23T08:51:49", "studies": [12, 13, 14, 18, 19, 20, 22, 23, 24, 26, 28, 29, 30, 32, 33, 34, 37, 38, 40, 50, 51, 56, 59, 60, 63, 64, 66, 67, 71, 72, 73, 75, 76, 78, 82, 83, 84, 85, 86, 88, 90, 91, 93, 94, 95, 96, 97, 99, 102, etc...], "to": "2013-03-24T08:51:49"}

Many of these have null ottolids, but they are being skipped as a temporary workaround. Anyway, it proceeds well until hitting study 60:

java.io.IOException: Server returned HTTP response code: 502 for URL: http://www.reelab.net/phylografter/study/export_NexSON.json/60

Peter suggests this may be a timeout issue (the study has 22 trees, each with 640 taxa).

[jar398]

As an aside, the "from" value doesn't seem consistent with the list of studies. Jonathan

On Sun, Mar 24, 2013 at 11:05 AM, Joseph W. Brown notifications@github.comwrote:

Calling up the list of updated studieshttp://www.reelab.net/phylografter/study/modified_list.json/returns:

{"from": "2013-03-23T08:51:49", "studies": [12, 13, 14, 18, 19, 20, 22, 23, 24, 26, 28, 29, 30, 32, 33, 34, 37, 38, 40, 50, 51, 56, 59, 60, 63, 64, 66, 67, 71, 72, 73, 75, 76, 78, 82, 83, 84, 85, 86, 88, 90, 91, 93, 94, 95, 96, 97, 99, 102, etc...], "to": "2013-03-24T08:51:49"}

Many of these have null ottolidshttps://github.com/OpenTreeOfLife/phylografter/issues/37, but they are being skipped as a temporary workaround. Anyway, it proceeds well until hitting study 60:

java.io.IOException: Server returned HTTP response code: 502 for URL: http://www.reelab.net/phylografter/study/export_NexSON.json/60

Peter suggests this may be a timeout issue (the study has 22 trees, each with 640 taxa).

— Reply to this email directly or view it on GitHubhttps://github.com/OpenTreeOfLife/phylografter/issues/38 .

[josephwb]

No longer times out, but hangs (+20 minutes at least).

[pmidford]

Is this the same study id?

On Mar 26, 2013, at 1:33 PM, Joseph W. Brown notifications@github.com wrote:

No longer times out, but hangs (+20 minutes at least).

— Reply to this email directly or view it on GitHub.

[josephwb]

Yes.

[josephwb]

Oops.Timeout is back.

[pmidford]

Hmm, it hung for me this afternoon too.

On Mar 26, 2013, at 2:37 PM, "Joseph W. Brown" notifications@github.com wrote:

Oops.Timeout is back.

— Reply to this email directly or view it on GitHub.

[pmidford]

I had a quick look at this: As an aside, the "from" value doesn't seem consistent with the list of studies. Jonathan

If there is a problem, it isn't in the filtering - there are a lot of studies that have last_modified timestamps at 2013-03-23 11:44:50 and I did a spot check and couldn't find a discrepancy. I don't have an answer to why those (and only those) were all (apparently) modified at the same time.

[jar398]

Ah, that's my doing... DOIs

-- apologies for brevity / using handheld gizmo --

On Mar 26, 2013, at 17:44, Peter Midford notifications@github.com wrote:

I had a quick look at this: As an aside, the "from" value doesn't seem consistent with the list of studies. Jonathan

If there is a problem, it isn't in the filtering - there are a lot of studies that have last_modified timestamps at 2013-03-23 11:44:50 and I did a spot check and couldn't find a discrepancy. I don't have an answer to why those (and only those) were all (apparently) modified at the same time.

— Reply to this email directly or view it on GitHubhttps://github.com/OpenTreeOfLife/phylografter/issues/38#issuecomment-15490611 .

[pmidford]

Fixes from yesterday seem to resolve the issue with study 60, but I found a larger study (227) that returns a 502 error, so the issue remains.

Study 438 is larger and also a problem.

[josephwb]

Tested upgrade on treemachine end. Big studies coming through without hanging or timing out. Good job Peter! Closing this now.

On treemachine end, currently have to up the memory (e.g. -Xmx2g) or suffer:

java.lang.OutOfMemoryError: Java heap space

I'll look into this.