search

APSIMInitiative / ApsimX

ApsimX is the next generation of APSIM
http://www.apsim.info
Other
130 stars 160 forks source link

Can we speed up deserialization? #2340

Closed zur003 closed 6 years ago

zur003 commented 6 years ago

Simulations can be rather slow to load initially. This is largely due to the time needed to deserialize the various objects in the simulation. We generate Models.XmlSerializers.dll during builds to accelerate this process, and that makes a substantial difference. However, if sgen is executed with the /verbose option, it can be seen that a number of classes cannot be readily serialized (commonly because of the lack of a parameterless constructor). It may be worth make the changes needed to allow serialization code for these classes to be generated.

hol353 commented 6 years ago

Should we instead move to JSON for our .apsimx file format. Would this remove the need to run sgen and also increase GUI startup times? I see there is a NewtonSoft JSON package that looks good. I've spoken to Neil Huth and Hamish Brown and they would be OK with a switch to JSON.

zur003 commented 6 years ago

I"m OK with JSON, but I think we'd want to do some performance benchmarking before making that change. There are some comparisons available at https://aloiskraus.wordpress.com/2017/04/23/the-definitive-serialization-performance-guide/ which suggest (if I'm interpreting correctly) that we might expect modest improvements in deserialization times with Json.NET, and little change in serialization time.

It may also be that much of the slowness is simply the time needed to locate and load Models.XmlSerializers.dll. It might help if we explicitly Load this dll at program startup.

hol353 commented 6 years ago

Yep good points. Are you able to look into this?

hol353 commented 6 years ago

It just occurred to me that the reason for the really slow loading of a .apsimx file in the GUI is related to the number of models that are all the same. For example, if you open Tests/Validation/Maize/Maize.apsimx there are many instances of the maize model (and hundreds of child models/organs/functions) that are all identical. I wonder if there is a way we could deserialise one maize model in memory and the simulations all point to it. Would be much quicker to load a .apsimx file.

hol353 commented 6 years ago

This is also kind of related to part of #2190 where Hamish wants to be able to see how the released models (e.g. maize) have been built (model and all child models). Perhaps we could optionally have a node in the GUI that shows released models and their child models but readonly and the simulations point to them.

As you can probably tell, I'm thinking out loud here.

hol430 commented 6 years ago

Duplicate of #2770