micronode / mstor

Messaging store and archival tool
https://www.mstor.org
Other
11 stars 2 forks source link

Caching even though caching disabled? #4

Open oliviercailloux opened 8 years ago

oliviercailloux commented 8 years ago

Thanks for your work, it has been very useful to me.

I have a question concerning the opening of a mbox file. I think mstor starts by reading the whole file, if so, would it be possible to change that behavior? When only needing to read a small part of the file, that waiting time seems not useful.

Perhaps relatedly, I suspect the cache does not get disabled, contrary to asked. I get plenty of debug log output DEBUG net.fortuna.mstor.data.MboxFile - Found match at [115238873] (or other numbers), then DEBUG net.sf.ehcache.CacheManager - Attempting to create an existing singleton. Existing singleton returned. I also easily get OutOfMemoryErrors even though I do not retain references to the messages I obtain. (I also see INFO net.fortuna.mstor.util.Configurator - mstor.properties not found. at start, which I suppose is expected.)

Here is the code I use.

        final Properties props = new Properties();
        props.setProperty("mail.store.protocol", "mstor");
        props.setProperty("mstor.mbox.encoding", "utf-8");
        props.setProperty("mstor.cache.disabled", "true");
        props.setProperty("mstor.mbox.metadataStrategy", "none");
        final Session session = Session.getInstance(props);
        final Store store = session.getStore();
        store.connect();
        final Folder inbox = store.getFolder("myfile");
        inbox.open(Folder.READ_ONLY);
        /** > 6 min init time for a 3 GB file, 15sec for 200 MB. */
        LOGGER.info("Initializing.");
        inbox.getMessages(1, 1);
        LOGGER.info("Initialized.");

Here are other log statements printed while mstor initializes.

01:59:22.098 [main] DEBUG net.sf.ehcache.CacheManager - Creating new CacheManager with default config
01:59:22.099 [main] DEBUG net.sf.ehcache.CacheManager - Configuring ehcache from classpath.
01:59:22.102 [main] WARN  n.s.e.config.ConfigurationFactory - No configuration found. Configuring ehcache from ehcache-failsafe.xml  found in the classpath: jar:file:/home/olivier/.m2/repository/net/sf/ehcache/ehcache/1.4.1/ehcache-1.4.1.jar!/ehcache-failsafe.xml
01:59:22.102 [main] DEBUG n.s.e.config.ConfigurationFactory - Configuring ehcache from URL: jar:file:/home/olivier/.m2/repository/net/sf/ehcache/ehcache/1.4.1/ehcache-1.4.1.jar!/ehcache-failsafe.xml
01:59:22.103 [main] DEBUG n.s.e.config.ConfigurationFactory - Configuring ehcache from InputStream
01:59:22.113 [main] DEBUG n.s.e.config.DiskStoreConfiguration - Disk Store Path: /tmp
01:59:22.137 [main] DEBUG n.s.e.config.ConfigurationHelper - No CacheManagerEventListenerFactory class specified. Skipping...
01:59:22.138 [main] DEBUG n.s.e.config.ConfigurationHelper - No CachePeerListenerFactoryConfiguration specified. Not configuring a CacheManagerPeerListener.
01:59:22.138 [main] DEBUG n.s.e.config.ConfigurationHelper - No CachePeerProviderFactoryConfiguration specified. Not configuring a CacheManagerPeerProvider.
01:59:22.167 [main] DEBUG n.s.e.config.ConfigurationHelper - No BootstrapCacheLoaderFactory class specified. Skipping...
01:59:22.167 [main] DEBUG n.s.e.config.ConfigurationHelper - No CacheLoaderFactory class specified. Skipping...
01:59:22.167 [main] DEBUG n.s.e.config.ConfigurationHelper - No CacheExceptionHandlerFactory class specified. Skipping...
01:59:22.176 [main] DEBUG net.sf.ehcache.store.DiskStore - Deleting data file mstor.folder.-1166983863.data
01:59:22.179 [main] DEBUG net.sf.ehcache.store.MemoryStore - Initialized net.sf.ehcache.store.LruMemoryStore for mstor.folder.-1166983863
01:59:22.180 [main] DEBUG net.sf.ehcache.store.LruMemoryStore - mstor.folder.-1166983863 Cache: Using SpoolingLinkedHashMap implementation
01:59:22.180 [main] DEBUG net.sf.ehcache.Cache - Initialised cache: mstor.folder.-1166983863
benfortuna commented 7 years ago

I can see from your code that you are adding the 'mstor.cache.disabled' property to your javamail session properties. This will not work as you need to add it to your System properties. You can do this by adding an option at runtime (e.g. java -Dmstor.cache.disabled=true ), or programmatically:

System.setProperty("mstor.cache.disabled", "true");

regards, ben

oliviercailloux commented 7 years ago

Aaah ok. Thanks. I had not understood that when reading the doc. Perhaps you may consider making it work with the javamail session properties…

thackel commented 6 years ago

Thanks for the information in this issue. I had the same problem were i got EHCache serialization exceptions (other bug or usage problem) and could not deactivate the cache by the mailsession properties.

Using System properties to disable the cache solved this.

The scope of these configuration properties should not be system/vm wide, so the suggestion of @oliviercailloux is fine.