IO API should provide loading an asset from a stream to reduce byte array copies. The asset could also specify whether it wants the stream closed once the call to LoadAsset has finished, or continue loading the asset asynchronously.
A reference checking AssetLoader API where you can register asset usages and have them automatically unload when no one is using them. Compatibility with the old system can be achieved through a global usage.