loiclacombe / mybatisnet

Automatically exported from code.google.com/p/mybatisnet
0 stars 0 forks source link

Slow SQL map verification for databases with large table count #12

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What version of the MyBatis.NET are you using?

1.6.2 DataMapper and 1.9.2 DataAccess

Please describe the problem.  Unit tests are best!

We use (I/My)Batis.Net with Databases containing 200 - 300 tables. We have 
built a utility to auto-generate everything for .NET (domain objects, mappers, 
XML, etc., and no, you can't have it... yet :D). Initializing all 200 - 300 
maps is very quick (a few seconds). Upon the first query, MyBatis verifies all 
result maps against the domain objects they map to using reflection, and it can 
be a 20 - 30 second wait (depending on your horsepower) before the query is 
even executed. Once this happens, everything is good to go. Just to be clear, 
I'm not talking about the XML validation option on a SQL map.

What is the expected output? What do you see instead?

It would be awesome if MyBatis could be revamped to allow on-the-fly 
verification of domain objects against their result or parameter maps. I've 
looked through the source code and it's a bit more complicated than I had hoped 
or I might have tried to do this myself.

Please provide any additional information below.

One way we've tried to mitigate this is having whatever application is 
accessing our data framework fire off some queries on a separate thread while 
the user is starting their session. We can shave off a few seconds of loading 
time by getting the verification going straightaway, but it can still be an 
annoyance while the app or webpage sit their spinning until MyBatis can start 
returning results.

Are their any plans to address this and any other known performance issues in 
future versions? 

Original issue reported on code.google.com by consultc...@gmail.com on 2 Sep 2010 at 12:36

GoogleCodeExporter commented 9 years ago
To verify what I mean by "on-the-fly verification," I'm simply suggesting that 
result maps aren't verified against their mapped classes until they are 
actually used. In any one session, a very small percentage of the actual tables 
in a database are interacted with and it seems like a waste of CPU cycles.

I'm suggesting this because we've recently implemented some WCF services that 
use iBatis, and MyBatis has to reinitialize and verify its maps every time a 
new connection to the service is established.

Original comment by consultc...@gmail.com on 2 Sep 2010 at 12:42

GoogleCodeExporter commented 9 years ago
Maybe this problem shoul be a major priority. Applications that works with big 
databases "go on line" too slowly.

Original comment by valerio....@gmail.com on 2 Nov 2010 at 9:43

GoogleCodeExporter commented 9 years ago
Just an update here. You can force verification of all of the maps by simply 
looping through them; we put this function in our Mapper class and call it once 
the mapper is init'd:

    Public Overridable Sub PreLoad()
        ' The trick is to load all of the statements for the mapper, which can
        ' be done by simply accessing their collection properties.
        Dim loadStatements As IEnumerator = Me.Mapper.MappedStatements.GetEnumerator()
        Dim loadParameterMaps As IEnumerator = Me.Mapper.ParameterMaps.GetEnumerator()
        Dim loadResultMaps As IEnumerator = Me.Mapper.ResultMaps.GetEnumerator()
    End Sub

This doesn't contribute much to the topic but is slightly related.

Original comment by consultc...@gmail.com on 8 Nov 2010 at 7:06

GoogleCodeExporter commented 9 years ago
I agree, that this issue should be of higher priority. We have a long 
initialization time with a large number of sql maps too. All maps are loaded on 
startup and only a small number of them are really used. There should be a 
runtime mode which lazy loads and validates maps only when they are really 
executed.

Original comment by holger.j...@bluewin.ch on 16 Nov 2010 at 11:55

GoogleCodeExporter commented 9 years ago
I think the big issue with lazy-loading the maps when they are needed is that 
it might not be that trivial, all because you can reference maps and mappings 
from other SQL maps

Let's say you had <TableXYZMapper.xml>, and one of it's <resultMap>'s <result> 
attributes was <result ... resultMapping="TableABCMapper.SomeResultMapID" />.

Would MyBatis know at the time it needs to lazy-load TableXYZMapper that it 
also needs to load TableABCMapper? As it exists now, it loads everything all at 
once so it doesn't have to worry about these cross referenced setups.

This is the wall I hit when I described how "I've looked through the source 
code and it's a bit more complicated than I had hoped or I might have tried to 
do this myself" in my bug report.

Original comment by consultc...@gmail.com on 19 Nov 2010 at 4:34