search

hotosm / oam-uploader-api

DEPRECIATED - The OAM Uploader API server
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

mktemp: failed to create file via template '/tmp/tmp.XXXXXXXXXX' #82

Closed smit1678 closed 3 years ago

smit1678 commented 7 years ago

Processing error flagged by a user during a recent upload: https://upload.openaerialmap.org/#/status/595d00936fe81d0011b03069.

First glance at the logs seems that it may be related to the tmpfs changes that were recently made. It's a browser-based upload. 

170705/150659.033, [response], http://0.0.0.0:4000: post /uploads {} 200 (75ms)
170705/150659.249, [request,debug,email], data: {"message":"success"}
170705/150700.433, [log,worker,worker 595d00], data: Initialized
170705/150700.478, [log,worker,worker 595d00,info], data: ["Processing job",{"_id":"595d00936fe81d0011b03068","url":"https://s3.amazonaws.com/oam-uploader-production-temp/KhlongToeiBig1duplicate_Orthomosaic_export_WedJul05-w501uxpk.tif","status":"processing","messages":[],"_workerId":"595d009436ad7d03776d0b34","startedAt":"2017-07-05T15:07:00.475Z"}]
mktemp: failed to create file via template '/tmp/tmp.XXXXXXXXXX': No such file or directory
/opt/oam-dynamic-tiler/bin/process.sh: line 22: to_clean[@]: unbound variable
/opt/oam-dynamic-tiler/bin/process.sh: line 22: to_clean[@]: unbound variable
170705/150704.004, [log,worker,worker 595d00,error], data: {"message":"Exited with 1: mktemp: failed to create file via template '/tmp/tmp.XXXXXXXXXX': No such file or directory\n/opt/oam-dynamic-tiler/bin/process.sh: line 22: to_clean[@]: unbound variable\n/opt/oam-dynamic-tiler/bin/process.sh: line 22: to_clean[@]: unbound variable\n","stack":"Error: Exited with 1: mktemp: failed to create file via template '/tmp/tmp.XXXXXXXXXX': No such file or directory\n/opt/oam-dynamic-tiler/bin/process.sh: line 22: to_clean[@]: unbound variable\n/opt/oam-dynamic-tiler/bin/process.sh: line 22: to_clean[@]: unbound variable\n\n    at ChildProcess.<anonymous> (/app/worker/process-image.js:84:24)\n    at emitTwo (events.js:106:13)\n    at ChildProcess.emit (events.js:191:7)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)"}
170705/150704.122, [log,worker,worker 595d00], data: Cleaning up.
170705/150704.167, [log,worker,debug], data: Worker exited: 0

Running gdalinfo /vsicurl/https://s3.amazonaws.com/oam-uploader-production-temp/KhlongToeiBig1duplicate_Orthomosaic_export_WedJul05-w501uxpk.tif doesn't show anything in particular and so it seems this may be related to the tmpfs change.

cc @tombh

tombh commented 7 years ago

This is another fallout from the server change. For whatever reason, previously /tmp/uploader-api permanently existed on the host server, however now, as is normal for /tmp, files and folder are removed once they reach a certain age.

So I easily fixed this just now with mkdir /tmp/uploader-api. Problem is of course, that will then be deleted again in, I think in about 10 days. So I've also changed the mount declaration in docker-compose.yml to just mount /tmp to '/tmp', which just means all the upload imagery will just live in the base of /tmp, which is not much of a problem.

However, this fix and the ones in hotosm/oam-uploader#78 have not been committed to any repo's yet. I'm waiting for feedback/further problems/better ideas, etc.