search

mysociety / popit

DEPRECATED - Development on PopIt has stopped and it is no longer being maintained
https://goo.gl/Vvej4Q
Other
76 stars 33 forks source link

Deleting an image via the UI returns 504 #783

Open andylolz opened 9 years ago

andylolz commented 9 years ago

Refs #683. Similar to that issue, the operation is successful (i.e. the image is successfully deleted.)

Tested on yournextmp.popit.mysociety.org instance.

chrismytton commented 9 years ago

I can see in the logs that the PopIt process was restarted around 08:15 this morning, however this is no stack trace to give any clue as to why it might have restarted :confused:

@andylolz Can you give any more details? I've tried adding and deleting images on other instances and it appears to work ok. Is there a specific person or image that I can reproduce this with?

mhl commented 9 years ago

@chrismytton The 8:15 restart is from cron (see /etc/cron.d/vhost-popit-mysociety-org). This restart is to pick up the new databases for the Pombola-mirrored PopIt instances, so really it should happen earlier in the morning. I just set it up for then so that the first few times it ran, it would run at a time when @Flupsy was likely to be in work, in case anything went wrong - we can move it much earlier now. (It would be ideal not to have to restart it for this, of course.)

@andylolz Did this happen to be at 8:15 this morning? :)

andylolz commented 9 years ago

Okay – tested on http://yournextmp-staging.popit.staging.mysociety.org and everything works fine. I can happily add and delete images.

However, every time I add / delete an image on https://yournextmp.popit.mysociety.org, I get a 504 (but the operation is always successful).

I wonder if this is https-related? I don’t have write access elsewhere, so I’m afraid I can’t test that.

chrismytton commented 9 years ago

I've successfully reproduced this using a dump of the YNMP database locally. It seems to be something specific to the YNMP instance I think due to the large amount of data in it. The error I'm seeing is:

FATAL ERROR: JS Allocation failed - process out of memory
Aborted (core dumped)

npm ERR! popit@0.0.29 start: `node server.js`
npm ERR! Exit status 134
npm ERR! 
npm ERR! Failed at the popit@0.0.29 start script.
npm ERR! This is most likely a problem with the popit package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node server.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls popit
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.13.0-44-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "start"
npm ERR! cwd /vagrant
npm ERR! node -v v0.10.37
npm ERR! npm -v 1.4.28
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /vagrant/npm-debug.log
npm ERR! not ok code 0

Which suggests a memory leak of similar somewhere. Will investigate further and report back here with updates.

chrismytton commented 9 years ago

I've tracked down the problem to this commit - https://github.com/mysociety/popit/commit/282794a9b378fe1b9b49f4403b9aa45584e82ba7, which was a "temporary" workaround for the differences between the UI and the API, however because the YNMP instance has so much data in re-indexing the whole site whenever a single image has changed is clearly no longer feasible. The fix is to move to using the API directly rather than having duplicated code, which will mean the re-indexing will be done on a per-document level.

andylolz commented 9 years ago

Sounds like you’re getting closer, Chris!

I don’t know if it’s related, but I also have this problem when adding organization images.