perrich / Hangfire.MemoryStorage

A memory storage for Hangfire.
Apache License 2.0
131 stars 43 forks source link

How to clean up the completed job regularly #38

Open davidchencloudexceed opened 3 years ago

davidchencloudexceed commented 3 years ago

Hello, I have a problem regarding cleanning up the completed job. I tried to use the attribute to define the expiration/life time.

public class HangfireJobRentionAttribute : JobFilterAttribute, IApplyStateFilter
    {
        public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
        {
            context.JobExpirationTimeout = TimeSpan.FromMinutes(1);
        }

        public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
        {
            context.JobExpirationTimeout = TimeSpan.FromMinutes(2);
        }
    }

After apply the attribute, I do see the hangfire pick it up in my dashboard. It said that it should be auto delted 28 min ago. Capture

However, I don't see the reduction of the memory footprint of my hangfire application. It keeps going up.

Does MemoryStorage has some build in clean up mechanism like SQL?

https://discuss.hangfire.io/t/does-hangfire-clear-down-old-job-data/2294/2

public SqlServerStorageOptions()
{
    TransactionIsolationLevel = null;
    QueuePollInterval = TimeSpan.FromSeconds(15);
    SlidingInvisibilityTimeout = null;
#pragma warning disable 618
    InvisibilityTimeout = TimeSpan.FromMinutes(30);
#pragma warning restore 618
    JobExpirationCheckInterval = TimeSpan.FromMinutes(30);
    CountersAggregateInterval = TimeSpan.FromMinutes(5);
    PrepareSchemaIfNecessary = true;
    DashboardJobListLimit = 10000;
    _schemaName = Constants.DefaultSchema;
    TransactionTimeout = TimeSpan.FromMinutes(1);
}
perrich commented 3 years ago

Hi,

The HangfireJobRentionAttribute is not used. The ExpirationManager uses the ExpireAt (IExpirable interface) property of the job.

davidchencloudexceed commented 3 years ago

thank you @perrich. How to set the job's ExpireAt property? I try to set the jobexpireationtimeout at the globalconfiguraiton. (I have to download the src code of hangfire and change the source to remove the 1 hour limit) However, I don't see InMemory storage will honor the change as my memory consumption same as before.

Hangfire.GlobalConfiguration.Configuration
                .SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
                .UseSimpleAssemblyNameTypeSerializer()
                .UseRecommendedSerializerSettings()
                .UseMemoryStorage(new MemoryStorageOptions
                {
                     JobExpirationCheckInterval = TimeSpan.FromMinutes(1)
                })
                .**WithJobExpirationTimeout(TimeSpan.FromMinutes(1))**
                .UseAutofacActivator(container);
davidchencloudexceed commented 3 years ago

find out why. InMemory did honor the setting. It is the one by one for loop detele slow the clean up.

perrich commented 3 years ago

The loop is not done one by one but by 1000 (of one of the managed type).