nakagami / firebirdsql

Firebird RDBMS sql driver for Go (golang)
MIT License
227 stars 60 forks source link

Blobs returning incorrectly #46

Closed igorsousa-b closed 3 years ago

igorsousa-b commented 6 years ago

I'm actually unsure if this is a problem on my end or the packages.

When getting a blob via a query and trying to save it as a file, the file won't open. Printing the data as plain text shows that all characters are strangely encoded.

var mapa []byte qRow := conn.QueryRow("select A_PICTURE from A_TABLE where ID = 1") qRow.Scan(&mapa) err := ioutil.WriteFile("picture.png", mapa, 0644) if err != nil { log.Fatal(err) }

If I do this for a plain something.txt file, it works perfectly. Anything more complex than that results in a broken file.

The original intent behind this is to get a .sxc file from the database and parse it. File won't open. Tried using a .png file just to test it out with the same end result.

Not sure if this is an issue with the package or my method of saving the file doesn't work with the file types I'm working with. If it's on my end, apologies for submitting this.

juliusz-a commented 6 years ago

This works for me... var mapa []byte rows := conn.Query("select A_PICTURE from A_TABLE where ID = 1") for rows.Next() { row.Scan(&mapa) err := ioutil.WriteFile("/tmp/picture.png", mapa, 0660) if err != nil { log.Fatal(err) }} The binary in the blob must be not corrupted in the first place, what can easily happened during tests.

juliusz-a commented 6 years ago

correction: rows.Scan(&mapa)

nakagami commented 6 years ago

thanks @juliusz-a

This test https://github.com/nakagami/firebirdsql/blob/master/driver_test.go#L152 seems blob insert and select works.

igorsousa-b commented 6 years ago

Tried your example, @juliusz-a , but it still doesn't work.

The binary in the blob seems to be fine. If I download the blob directly from FlameRobin, it's perfectly fine.

There seems to be some inconsistency with the sizes, though. Getting the file directly from FlameRobins gets me a 14 KB file, which is the correct size of the picture, while both your and my examples get me a 21 KB file.

I'm using OS X 10.12.6

juliusz-a commented 6 years ago

igorsousa-b, Did you try to upgrade the Go compiler and the nakagami/firebird driver? The file size difference indicates that the returned binary content is treated as text. I had in the past very similar problem with one of the earlier version. I am ruining on Linux 32b, but I will try on OS X 10.11.6.

igorsousa-b commented 6 years ago

I was using 1.9. Just updated to 1.9.1 and also updated the firebird driver. Nothing changed.

Tried using HTML+PHP just to see if the file was corrupted or something but it shows the picture perfectly so the problem isn't on the file's side.

ghost commented 6 years ago

I saw someone with the same issue on another driver (mysql) and the person solved it by changing the byte to string. I know that internally they are the same thing (sort of), but for some reason it worked that way....

var mapa string qRow := conn.QueryRow("select A_PICTURE from A_TABLE where ID = 1") qRow.Scan(&mapa) err := ioutil.WriteFile("picture.png", []byte(mapa), 0644) if err != nil { log.Fatal(err) }

igorsousa-b commented 6 years ago

@giovannigaspar Thanks for the help, but same result.

juliusz-a commented 6 years ago

igorsousa-b, I works for me in the OSX 10.11.6/Go 1.9.1. However, I have confirmed the weird behavior (corrupted file) when only one param is used in the SELECT query and the param is for blob. Adding one more param needed or not needed seems to solve the problem.

var ID []byte var PIC []byte Row := conn.QueryRow("select ID, PIC from A_TABLE where ID = 1") Row.Scan(&ID, &PIC) err := ioutil.WriteFile(ID+".png", PIC, 0644) ....

igorsousa-b commented 6 years ago

I've had an example like that a few tries ago (wanted to see if the query was getting the correct row and such).

Unfortunately, it still fails in the same way.

Given all of your answers, I think this is some sort of deeper issue on my end.

pacp commented 6 years ago

I'm igorsousa-b's colleague, and was also looking into this issue. I found the cause to be a bad field definition in the database. In order to have binary blobs working properly, you need to define them as subtype 0, and this blob field was defined as subtype 1. I don't know if the database driver should be more flexible handling this (probably not); all other drivers I've ever used to access Firebird didn't care about this, and I was able to retrieve binary blobs correctly from subtype 1 blob fields just the same as form subtype 0 blob fields, but that is a bit sloppy and we certainly will be taking better notice of our blob subtypes henceforth! Thank you all for your help, and for the great work on the driver.