search

ghmo / google-refine

Automatically exported from code.google.com/p/google-refine
0 stars 0 forks source link

Progress feedback during upload #179

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
When creating a large-ish project from a data file, I don't get any feedback 
during the import, so I'm uncertain whether it's just spinning its wheels & 
thrashing, or whether it's making slow but steady progress.

It would also be great if there were a "cancel" button for the upload.  The 
only way I can see is to quit the background application & cancel the browser 
page load.

I'm using google-refine-2.0-r1760.dmg on OS X 10.6.4 with java 1.6.0_22.

Original issue reported on code.google.com by kena...@gmail.com on 4 Nov 2010 at 6:04

GoogleCodeExporter commented 8 years ago 
Just creating a project with ~1M rows. Taking 3 hours and I don't know if it 
will even finish today. An ETA would be really helpful for my workflow.

Original comment by kopo...@gmail.com on 14 Dec 2010 at 8:17

GoogleCodeExporter commented 8 years ago 
Not really relevant to the issue at hand, but since people attempting to load 
large projects are likely to end up here, for large projects you should:

1. Make sure that you've got sufficient real memory
2. Make sure that your Java heap size is set large enough to accommodate your 
project
3. Strongly consider turning off "guess data types" (which is on by default).  
It will make multiple guesses for each cell and for the common case (i.e. 
text), it will try them all before falling through to accept the value as 
ordinary text.
4. Consider turning off splitting and import as a single column.  There's a 
substantial amount of per-cell overhead which you may end up incurring 
unnecessarily for "uninteresting" columns that you'll just delete later.  
Better to import things as a single column and do a little more work to split 
things by hand afterwards for very large projects.

Original comment by tfmorris on 14 Dec 2010 at 8:49

GoogleCodeExporter commented 8 years ago 
The new importer UI framework in trunk/ should already show uploading progress.

Original comment by dfhu...@gmail.com on 11 Sep 2011 at 5:17

GoogleCodeExporter commented 8 years ago

Original comment by dfhu...@google.com on 9 Oct 2011 at 5:17