Open jywarren opened 5 years ago
Cool! I’m heading out now but I’ll get on this in a couple of hours.
Great! Same!!
Yes this link as I had above is correct for logs:
On Sat, Jun 15, 2019, 9:32 AM Varun Gupta notifications@github.com wrote:
Cool! I’m heading out now but I’ll get on this in a couple of hours.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer-app/issues/19?email_source=notifications&email_token=AAAF6J26Z7UUHSCMIJS3CQ3P2TVNDA5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXYYIGI#issuecomment-502367257, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6JZ7RGRJJLC4ZWF4B4DP2TVNDANCNFSM4HWZEIJA .
Ok actually there are no logs i can find since the 10th, but there may have been one more error then:
https://gist.github.com/jywarren/e46994aa9dec8d774245822365fb446f
But I don't know why we aren't seeing any other logs! Do we need this express app to generate more logging, at least for requests?
Hmm, actually I'm seeing a This site can’t be reached / 34.74.118.242 refused to connect.
error... for this URL (after initial redirect): click here
Ah, but a few moments later it did load, and generated a good export!
Hmm, Maybe this is a browser issue, the browser might not be waiting enough time for the server to process the response! Let me integrate the bucket with our code!
Then we would have a better understanding as to what is going on!
Anyhow here are some maps with <10 images;
Are these working?
I'm trying them one at a time... starting with the last
ah for marshburg
i get the same "refused to connect" error... but i'll wait a bit and see it it'll load later like the other did.
Update: Chrome keeps trying to reload that page, so we'll see...
Okay! I'll look into the cloud stuff meanwhile!
Ah ok so it's showing in the cloud logs now!
{
insertId: "f5rcvu9l8f5t9h1go"
labels: {…}
logName: "projects/public-lab/logs/stderr"
receiveTimestamp: "2019-06-15T14:40:45.103722159Z"
resource: {…}
severity: "ERROR"
textPayload: "(node:21) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added. Use emitter.setMaxListeners() to increase limit
"
timestamp: "2019-06-15T14:40:39.359077955Z"
}
Okay! Let me set the max listeners then!
I thought you did already in https://github.com/publiclab/image-sequencer-app/commit/9aa949d79ce7d214785919705afd416bc64a82d3 ?
The marshburg
map finally loaded, but after spinning for a few seconds, i now see This page isn’t working 34.74.118.242 didn’t send any data.
Ah but in the logs i now see that it rebooted! Maybe it crashed. Maybe it has the maxListeners code loaded now...?
I did but maybe I need to set it for every request separately!
Oh!
Yeah i'm still getting the MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added. Use emitter.setMaxL...
error in the logs. It's quite helpful to run it and watch the logs live ! Try it, you should have permissions...
You can watch new versions build here, though they take a little longer to actually get to the cluster, i think: https://console.cloud.google.com/cloud-build/builds?authuser=1&project=public-lab
Awesome! How do I deploy the code which contains my credentials for uploading?
I mean I shouldn't just push this to github, right?
yeah... don't worry about cloud storage just yet! Maybe run it locally and use an environment variable if you can? Then we'll add the creds safely.
Okay, but how would do test it out then. I have a strong feeling that the browser might be sending repeat requests after every 30 seconds or so, which causes the cluster to crash. Maybe I'll try to request the URL with curl?
Okay, I'm getting connection refused error! Why is it crashing like this?
Okay I don't see the listeners exceeded log anymore, but for some reason I am not getting the correct response from the server.
@jywarren Are you also facing the same thing on your end?
I'm not at a computer right now, sorry!!!
On Sat, Jun 15, 2019, 11:51 AM Varun Gupta notifications@github.com wrote:
@jywarren https://github.com/jywarren Are you also facing the same thing on your end?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer-app/issues/19?email_source=notifications&email_token=AAAF6J7LJAVNGZIN2KWERGTP2UFXDA5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXY2X6Q#issuecomment-502377466, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J5DGH5T53X2YNV53SDP2UFXDANCNFSM4HWZEIJA .
Okay cool! I am working on the cloud stuff right now! Will circle back to this after that is working!
Okay @jywarren I have written the status endpoint code locally! Now how do we deploy it? Should I just gitignore my secrets and push it here, but that would break the current production build?
Just enable it with a flag maybe, or can you check presence of env variables and only upload if they exist?
On Sat, Jun 15, 2019, 12:54 PM Varun Gupta notifications@github.com wrote:
Okay @jywarren https://github.com/jywarren I have written the status endpoint code locally! Now how do we deploy it? Should I just gitignore my secrets and push it here, but that would break the current production build?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer-app/issues/19?email_source=notifications&email_token=AAAF6JZZL25WZBDPHL26ZADP2UNDFA5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXY35MQ#issuecomment-502382258, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6JYVBW6HVTEQ4BCPLRDP2UNDFANCNFSM4HWZEIJA .
I'm not really sure how to do that, maybe I can push the new commits in, once someone configures the cluster?
For now I'll move on to why it's crashing. I'm not even able to get a response for small maps like the ceres--2
Also i tried to redirect requests from the user, from the status endpoint to the file in the bucket once the export is done. But I don't think it's publicly accessible, we may want to change that.
You may need to make each file public.
Can we make cloud storage depend on a get parameter like &useCloudStorage=true ?
On Sat, Jun 15, 2019, 1:02 PM Varun Gupta notifications@github.com wrote:
Also i tried to redirect requests from the user, from the status endpoint to the file in the bucket once the export is done. But I don't think it's publicly accessible, we may want to change that.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer-app/issues/19?email_source=notifications&email_token=AAAF6J2LD4MQBH23Y5QRAPDP2UOA5A5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXY4CPI#issuecomment-502382909, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J32AWAGXPFXN67KGV3P2UOA5ANCNFSM4HWZEIJA .
Okay I was researching memory leaks and stack overflow says that even extensive logs can leak memory by tying down the event loop. Disabling the progress bars might just help!!
You may need to make each file public. Can we make cloud storage depend on a get parameter like &useCloudStorage=true ? … On Sat, Jun 15, 2019, 1:02 PM Varun Gupta @.***> wrote: Also i tried to redirect requests from the user, from the status endpoint to the file in the bucket once the export is done. But I don't think it's publicly accessible, we may want to change that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19?email_source=notifications&email_token=AAAF6J2LD4MQBH23Y5QRAPDP2UOA5A5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXY4CPI#issuecomment-502382909>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J32AWAGXPFXN67KGV3P2UOA5ANCNFSM4HWZEIJA .
Sure! We already have an upload option! I'll use it good idea! Also is there some extra parameter while creating the file which makes it public?
Yes it's in the Google documentation!
On Sat, Jun 15, 2019, 1:09 PM Varun Gupta notifications@github.com wrote:
You may need to make each file public. Can we make cloud storage depend on a get parameter like &useCloudStorage=true ? … <#m8720807423403805511> On Sat, Jun 15, 2019, 1:02 PM Varun Gupta @.***> wrote: Also i tried to redirect requests from the user, from the status endpoint to the file in the bucket once the export is done. But I don't think it's publicly accessible, we may want to change that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19 https://github.com/publiclab/image-sequencer-app/issues/19?email_source=notifications&email_token=AAAF6J2LD4MQBH23Y5QRAPDP2UOA5A5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXY4CPI#issuecomment-502382909>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6J32AWAGXPFXN67KGV3P2UOA5ANCNFSM4HWZEIJA .
Sure! We already have an upload option! I'll use it good idea! Also is there some extra parameter while creating the file which makes it public?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer-app/issues/19?email_source=notifications&email_token=AAAF6J42VTJRTQFSBCGS7DDP2UO2XA5CNFSM4HWZEIJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXY4GGY#issuecomment-502383387, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAF6JYDNHY5C3MSFCBLB33P2UO2XANCNFSM4HWZEIJA .
Okay pushed it in! Now we can enable it using upload=true in the query.
Working on making it public now!
Okay I put the redirect in place!! Now
http://localhost:4000/api/v2/export/?url=http://mapknitter.org/maps/pvdtest/warpables.json&upload=true
should give an immediate response and further, http://localhost:4000/api/v2/status/?pid=47516
is downloading the file. Right now It's embedded in an html page but we can change that.
@jywarren Please publish the new version of image sequencer onto npm whenever you can find the time, let's see if the pace progress bars are at fault here.
Okay I pushed in the new code, but it is not yet deployed! @jywarren any idea why?
Hi @tech4GT can you update the status here?
Varun Gupta @tech4GT Jun 18 21:28 - Yeah! The code which changes max listeners is now deployed! Let’s see if we still get the same error.
What should we test? This?
returns:
Your Image is exporting, to load Image please visit, http://34.74.118.242/api/v2/status/?pid=32
http://34.74.118.242/api/v2/status/?pid=32 then seems to hang!
Is it possible the requests aren't going to the same container instance? Do we need to use an identifier (unlike pid) that would be unique across the cluster?
Many commits were made -- https://github.com/publiclab/image-sequencer-app/commit/046bbf75cd85394646e76110a244af3e5e633c27
Maybe it was this one for @Divy123's questions?
https://github.com/publiclab/image-sequencer-app/commit/1256685ebe4e81e70cad5ad9ba1ffc70a8ebb69d
Ya that is something that can be done, but I don't know where this code should lie in IS.
@jywarren the upload option won’t work coz we don’t have credentials on the cluster as of now. Let’s test it without that option and see if we get the same error in the logs!
OK, testing: http://34.74.118.242/api/v2/export/?url=http://mapknitter.org/maps/pvdtest/warpables.json
OK it almost instantly worked... now something bigger:
http://34.74.118.242/api/v2/export/?url=http://mapknitter.org/maps/ceres--2/warpables.json
That worked. http://34.74.118.242/api/v2/export/?url=https://mapknitter.org/maps/marshburg/warpables.json next...
http://34.74.118.242/api/v2/export/?url=https://mapknitter.org/maps/marshburg/warpables.json stalled the first time, so i restarted it...
http://34.74.118.242/api/v2/process/?upload=undefined&steps=[%7B%22id%22:1,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314640/DJI_0077.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1099%252C1897%7Cne%3A2670%252C1772%7Cse%3A2843%252C2231%7Csw%3A1314%252C2357%7D%22%7D,%7B%22id%22:2,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314641/DJI_0074.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1536%252C2272%7Cne%3A2287%252C2242%7Cse%3A2346%252C2495%7Csw%3A1542%252C2508%7D%22%7D,%7B%22id%22:3,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314642/DJI_0076.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A955%252C1927%7Cne%3A2877%252C1812%7Cse%3A3055%252C2347%7Csw%3A1095%252C2440%7D%22%7D,%7B%22id%22:4,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314643/DJI_0078.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1225%252C1803%7Cne%3A2581%252C1697%7Cse%3A2732%252C2104%7Csw%3A1375%252C2210%7D%22%7D,%7B%22id%22:5,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314644/DJI_0075.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1440%252C2138%7Cne%3A2344%252C2100%7Cse%3A2444%252C2423%7Csw%3A1513%252C2465%7D%22%7D,%7B%22id%22:6,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314646/DJI_0081.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A715%252C1413%7Cne%3A2958%252C1151%7Cse%3A3197%252C1888%7Csw%3A1140%252C2096%7D%22%7D,%7B%22id%22:7,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314654/DJI_0088.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1231%252C707%7Cne%3A2498%252C565%7Cse%3A2686%252C993%7Csw%3A1378%252C1126%7D%22%7D,%7B%22id%22:8,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314655/DJI_0090.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1992%252C230%7Cne%3A3594%252C98%7Cse%3A3762%252C589%7Csw%3A2161%252C720%7D%22%7D,%7B%22id%22:9,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314658/DJI_0093.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1078%252C148%7Cne%3A2596%252C0%7Cse%3A2752%252C470%7Csw%3A1234%252C618%7D%22%7D,%7B%22id%22:10,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314671/DJI_0035.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A0%252C1781%7Cne%3A1270%252C1502%7Cse%3A1620%252C1850%7Csw%3A385%252C2141%7D%22%7D,%7B%22id%22:11,%22input%22:%22https://s3.amazonaws.com/grassrootsmapping/warpables/314689/DJI_0055.JPG%22,%22depends%22:[],%22steps%22:%22webgl-distort%7Bnw%3A1735%252C755%7Cne%3A994%252C755%7Cse%3A994%252C525%7Csw%3A1735%252C525%7D%22%7D,%7B%22id%22:12,%22input%22:1,%22depends%22:[1,2,3,4,5,6,7,8,9,10,11],%22steps%22:%22canvas-resize%7Bwidth:7524%7Cheight:5016%7Cx:1099%7Cy:1772%7D,import-image%7Burl:output%3E2%7D,overlay%7Bx:1536%7Cy:2242%7D,import-image%7Burl:output%3E3%7D,overlay%7Bx:955%7Cy:1812%7D,import-image%7Burl:output%3E4%7D,overlay%7Bx:1225%7Cy:1697%7D,import-image%7Burl:output%3E5%7D,overlay%7Bx:1440%7Cy:2100%7D,import-image%7Burl:output%3E6%7D,overlay%7Bx:715%7Cy:1151%7D,import-image%7Burl:output%3E7%7D,overlay%7Bx:1231%7Cy:565%7D,import-image%7Burl:output%3E8%7D,overlay%7Bx:1992%7Cy:98%7D,import-image%7Burl:output%3E9%7D,overlay%7Bx:1078%7Cy:0%7D,import-image%7Burl:output%3E10%7D,overlay%7Bx:0%7Cy:1502%7D,import-image%7Burl:output%3E11%7D,overlay%7Bx:994%7Cy:525%7D%22%7D]
Now, building on the list from https://github.com/publiclab/mapknitter-exporter-sinatra/issues/23, let's see how big images/maps it can handle!
http://34.74.118.242/api/v2/export/?url=http://mapknitter.org/maps/pvdtest/warpables.json&scale=30 is real fast. seconds.
http://34.74.118.242/api/v2/export/?url=http://mapknitter.org/maps/ceres--2/warpables.json&scale=30 -- It worked!!!! 25mb output in <2 minutes! Too big even to upload to GitHub! https://www.dropbox.com/s/m6v131cyynlwx0a/ceres--2.png?dl=0
(note that scale isn't working now -- #14 )
http://34.74.118.242/api/v2/export/?url=http://mapknitter.org/maps/irish-uk-border-mapping/warpables.json&scale=30 next:
Then: