search

sverhoeven / docker-cartodb

Dockerized CartoDB
BSD 3-Clause "New" or "Revised" License
148 stars 116 forks source link

Upload limit #63

Open mhkeller opened 6 years ago

mhkeller commented 6 years ago

Is there a hardcoded upload size limit? I'm trying to upload a csv that's over 1gb and it says it fails for an unknown reason. Same thing happens if I upload the shapefile output of that file from QGIS. If I upload a subset it works fine.

sverhoeven commented 6 years ago

Yes, there is an upload limit of 150Mb at https://github.com/sverhoeven/docker-cartodb/blob/master/config/cartodb.nginx.proxy.conf#L4

mhkeller commented 6 years ago

Thanks. I pulled the latest to get https://github.com/sverhoeven/docker-cartodb/commit/397ffca212a701615ec104d91063ecec3f998cf3 and started from scratch but I still get an unknown error on upload. I don't see any error doing docker logs <id>. Is there another place to look?

sverhoeven commented 6 years ago

The importing is handled by the resque worker.

It writes it log files to /cartodb/resque.log, which you can view with docker exec <container> cat /cartodb/resque.log.

mhkeller commented 6 years ago

Hmm, the logs don't show any activity. If I import a smaller shapefile or one of the stock datasets like chicago building footprints, I do see logs.

mhkeller commented 6 years ago

I put together a test repo to generate a csv containing a few million random points: https://github.com/mhkeller/gen-points

I get the same silent failing using the result from this script as I do with my file.

mhkeller commented 2 years ago

@sverhoeven any thoughts on this?

sverhoeven commented 2 years ago

No idea, I have not used Carto for a long time.