Closed travelliu closed 1 year ago
as I can see from comparison there is very big difference and probable answer for this, is the way each package dealing with lob in case of original C driver the row data collected as usual and each lob is returned from database as lob locator structure which is few bytes and delay reading the actual blob data until you try to read it on the other hand go-ora will read blob data for each rows (20,000) and I think this is the reason
godror's current default is to fetch as string/[]byte, which is much faster then getting a Lob locator . This works up to 1 GB.
Oracle Database LOBs in general also support 'LOB prefetching' (different to row prefetching), something the LOB team at Oracle like, but the fetch-as-string/buffer approach seems just as good in practice for ODPI-C -based drivers like godror. I did add lob prefetching to PHP OCI8 because it let me do minimal driver changes and not destabilize an old code base.
yes thanks @cjbj for your explanation I will review these Caps in your project and fix this issue
TKS。 Through the network packet capture, it was indeed lob prefet. database version 11.2.0.4
Gordor
go-ora
Any update?
i am working on it
fixed in v2.6.2
this changes I make mainly in the command.go file I take from C# driver
the speed is dramatically increase after that changes
but still I receive lob locator from the server after receiving data. the presentation as follow
1- send query request to the server
2- server will response with column definitions (no data)
3- send another request with define
4- server will return data
for lobs server will send <data><locator>
and so on
@cjbj you talk about 'LOB prefetching' and 'fetch-as-string/buffer approach' would you explain the sequence above is related to which of them
this approach is different from old one as follow: in old one data is sent in step 2 and no need to step 3 and 4
but data lob data will return as <locator>
only and you need to make a network call to get data and size
@travelliu could you test now and see the difference
@travelliu could you test now and see the difference
OK. i testing
@sijms
driver | select rows | time |
---|---|---|
go-ora | 20000 | 206ms |
godror | 20000 | 260ms |
0 rows exported 2023-03-18 12:26:26.720312976 +0800 CST m=+0.045739054
20000 rows exported 2023-03-18 12:26:26.92704968 +0800 CST m=+0.252475758
fetchSize: 4000 Rows: 20000 Time: 206ms Avg: 96726.45561031434/s
0 rows exported 2023-03-18 12:26:08.973837443 +0800 CST m=+0.035773967
20000 rows exported 2023-03-18 12:26:09.23424223 +0800 CST m=+0.296178764
fetchSize: 4000 Rows: 20000 Time: 260ms Avg: 76792.01451513443/s
Were you fetching as string, or as lob locators? The former will be a lot faster than the latter.
I think we can now close this issue
LOB field data content is small.
The query performance is lower than that of the godror driver, and the performance varies greatly
This code doesn't seem to take into account the inline storage scenario
Go Version
go-ora
godror
test code