search

CottageLabs / LanternPM

Lantern meta repository for product management
1 stars 0 forks source link

Huge papers cause problems #82

Closed markmacgillivray closed 8 years ago

markmacgillivray commented 8 years ago

http://www.tandfonline.com/doi/full/10.1080/15548627.2015.1100356

markmacgillivray commented 8 years ago

The licence check running on this has to do massive amounts of regexing

markmacgillivray commented 8 years ago

That is a webpage that is about 37MB in size. Trying to regex check on it locks the machine for a long time. Resetting processes leaves the previous machine stuck processing it, and allocates it onto another machine too. Enough resets and all machines are stuck trying to run 234 regexes over 37MB of text :(

markmacgillivray commented 8 years ago

Adding a catch to academic licence to not run if the content size is greater than 10m bytes for the time being.

richard-jones commented 8 years ago

Can we add something to the provenance record for such items, explaining what's happened, so the user knows to check manually?

markmacgillivray commented 8 years ago

Done in dev. To push to live soon.