sosedoff / pgweb

Cross-platform client for PostgreSQL databases
https://sosedoff.github.io/pgweb
MIT License
8.63k stars 732 forks source link

Bytea fields download automatically when selecting tables #156

Closed benlowry closed 8 years ago

benlowry commented 8 years ago

If you select a table it automatically opens the rows view, when it has bytea data pgweb downloads it all and is unresponsive till it finishes, with no notification of what's going on or how long to remain.

I think bytea fields should have a link to download, ideally with the size if that's possible.

Second best would be a progress bar or something, that would also stop the browser messages about an unresponsive tab.

sosedoff commented 8 years ago

Please provide a sample SQL file so that i can test and fix the problem.

benlowry commented 8 years ago
create table tmp (
    blob bytea
);

insert into tmp(blob) values('\xABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789 ABCDEF0123456789')

I've been digging into the code, I don't know golang but I've managed to tentatively identify a bytea field and substitute its size, it's possible I'm not identifying it properly but maybe this gives you a head start. I was trying to look for the '\x' that byte data starts with (I think always but again not sure).

func (res *Result) PrepareBytea() {
        for i, row := range res.Rows {
                for j, col := range row {
            t := reflect.TypeOf(col).Kind()
            if(t != reflect.String) {
                continue
            }
            str := col.(string)
            if str[0] == 255 {
                res.Rows[i][j] = len(str)
            }
        }
    }
}
sosedoff commented 8 years ago

Thanks, i will check on this issues once i get some free time.

sosedoff commented 8 years ago

I just created a table from your example using pgweb 0.9.3 (latest) on OSX (chrome) and this is what i got so far: screenshot 2016-07-05 18 27 59

Nothing was downloaded automatically. Also tried Safari and Firefox, both work. What OS and browser are you using?

benlowry commented 8 years ago

That is what you downloaded, you just need to provide a lot more to see the effect, try some 10s of megabytes in the bytea value or reproduce the row 1000s of times and you will probably see your browser start straining. I should have mentioned this I just didn't want to put an incredibly massive example. In our case we have a table with about 2000 rows and a small amount of binary data for each row that totals 150 megabytes.

sosedoff commented 8 years ago

Ah i see what you mean now. What is the behavior you expect?

benlowry commented 8 years ago

Ideally clicking the tables wouldn't include the raw data at all, that could be solved by:

benlowry commented 8 years ago

This was pretty much fixed for me by limiting the number of rows returned when clicking a table.

Thanks for all your wonderful work @sosedoff!