Roadmap [0.3]

[Philio]

Goals for next release:

• Resolve all outstanding bugs, complete any outstanding enhancements.
• Change of library structure to be more "go-like".
  • Change MySQL struct to become Conn.
  • Merge New and Connect to form single Connect function.
  • Usage like: conn := mysql.Connect(params...)
  • Drop 'MySQL' in front of other structs.
• New packet handling process.
  • Read packet header.
  • Obtain packet length.
  • Read entire packet.
• New generic packet parser / processor.
  • Remove mass-duplicated existing packet processing code.

[WORK IN PROGRESS!]

[thomaslee]

Phil, if I can help with 0.3 please drop me a line.

[Philio]

I've not really done much, just putting thoughts together so far and coming up with a plan of action really but certainly your help would be welcomed :)

[Philio]

Any thoughts on the roadmap so far?

[thomaslee]

I'd like to see something that lets me read result sets one row at a time i.e. leave a "row" packet on the wire until I specifically ask for it.

This is similar to the difference between mysql_use_result and mysql_store_result: the latter loads the full result set into memory (like GoMySQL currently does), while the former merely initiates retrieval of results and does not store them all in memory. Further calls to mysql_fetch_row() do not differentiate between the two.

See http://dev.mysql.com/doc/refman/5.0/en/mysql-use-result.html and http://dev.mysql.com/doc/refman/5.0/en/mysql-store-result.html

Hope I'm making sense!

[thomaslee]

Also on that topic: should we consider wrapping the C API rather than implementing the MySQL protocol ourselves?

Purely hypothetical -- not sure of what we'd lose by doing so, nor if we'd see any obvious benefits. I do know that it would screw with my use of goinstall, but I'm willing to suffer that if we wind up with a better GoMySQL :)

[thomaslee]

Oh, oh, and tests.

[Philio]

I'm not actually sure how other libraries deal with result sets, if they simply remain in the buffer or are "stored" somewhere locally regardless. It would need to be read at some stage any to remove it from the buffer, wouldn't it?

Wrapping the C wrapper is cheating ;) (Also there are a number of libraries that already do this, the intention was always to make a pure Go library)

The protocol is really not that difficult to follow, plus it's 99% already covered just want to improve the implementation.

What I had in mind was to read the entire result from the buffer, then this could be processed separately, or even delayed until it was actually requested by the user, kinda like you said? Plus I think we will always need to read the header plus first byte regardless (at least of the first packet in a series).

Agree should implement everything with tests too :)

[thomaslee]

RE: C wrapper -> cool, no worries. Just throwing it out there. :)

RE: result sets -> yep, the idea is to give GoMySQL clients the option to pull back one row at a time, or load a full result set into memory in one go (e.g small result sets). Currently only the latter appears to be possible.

[thomaslee]

Regarding whether result sets are stored locally, my understanding of the use vs. store calls in the MySQL C API is that unprocessed rows are simply left on the wire in the case of the use call. Could be wrong.

Point is to give folks with huge result sets wanting to use GoMySQL an option to do so (e.g. batch processing of millions of records)

[Philio]

I've not done C programming in about a decade so I wouldn't get very far looking through C code, however it does make sense, we would just need to discard the data if it remains unused, as "on the wire" as such wouldn't happen it would be buffered by the OS until we chose to do something with it I assume?

[thomaslee]

Without reading the code myself: that would be my assumption too, yep.

[Philio]

Well I've just done a few updates to 0.2 branch which will make 0.3 more of a refactoring exercise than features/bugs. There are still a few small ones but the major ones (e.g. binary formats) are out the way.

[Philio]

I've added the basics for changing the library to function in a way that is much more like other Go packages. This also removes the rather cumbersome connect functions of 0.2.

I'm wondering if public properties should perhaps become private and should be accessed via getters and setters (this is how I would do it in other languages)

E.g. Error, Errno, Logging become private

func (client Client) GetErrno() string func (client Client) GetError() string func (client *Client) SetLogging(enabled bool, type int)

Tom, what do you think about this?

[thomaslee]

I'm +0.5 on leaving them as they are. Despite the "best practice" of accessors, I think we gain very little by wrapping this stuff up in the case of errno, error & logging. Further, I think most Unix-ites would expect errno to be a property. I'm not too fussed either way, though.

Priority 1 for me is eliminating duplication & simplifying what's there. I think your plans for the packet handling might help here, although I'd love to hear a detailed explanation of what you have in mind. If the code's easier to comprehend at the end of the cycle for 0.3, I'll be happy.

Priority 2 would be the result set streaming stuff I was rambling about earlier.

Priority 3 would be tests (I know some would argue this should be priority 1 :P)

On a bit of a tangent, Russ Cox waggled his finger at me for using the "Get" prefix on accessors in go-dbi :) Apparently it's more Goesque to simply omit the prefix (e.g. Errno(), Error())

[Philio]

Continued on the Google Group! https://groups.google.com/group/gomysql

[Philio]

This is now pretty old, hence closing.