search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.08k stars 4.7k forks source link

Linux Platform All net version area-Extensions-Hosting `Microsoft.Extensions.Hosting.Systemd` #99406

Open 1009058470 opened 7 months ago

1009058470 commented 7 months ago

Description

I am follow this to build a unix service, but when i to kill them and start them sometimes, it also give a error like this Error: No space left on device, and then i try to increase the fs.inotify to a bigger number, it not show that error, but another thing appear that is the very slowly to exec Host.CreateDefaultBuilder(args) .ConfigureServices((hostContext, services) => { services.AddHostedService(); }).UseSystemd().StartAsync() I wirte a timer in StartAsync like this

    public override Task StartAsync(CancellationToken cancellationToken)
    {
        Console.WriteLine("call image worker start");
        Task.Factory.StartNew(CallImageAction, cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default);
        return base.StartAsync(cancellationToken);
    }

CallImageAction is a function to run a timer

I run them as service like this systemctl start xxx.service and then it run warn no space left on device and i find this in linux “By default, Linux only allocates 8192 watches for inotify, which is ridiculously low. And when it runs out, the error is also No space left on device, which may be confusing if you aren't explicitly looking for this issue.”

So i change the fs.inotify.max_user_watches = 8192 in /etc/sysctl.conf and then i rerun the service and then the worker do not run

Reproduction Steps

change the fs.inotify.max_user_watches = 8192 in /etc/sysctl.conf and then i rerun the service and then the worker do not run

Expected behavior

when change the fs.inotify.max_user_watches = 8192 in /etc/sysctl.conf and make the program as a linux service, it can run the service

Actual behavior

it seem it do not run the real worker in StartAsync in BackgroundService

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

steveharter commented 7 months ago

change the fs.inotify.max_user_watches = 8192 in /etc/sysctl.conf

Isn't this an environmental issue? In other words, not a bug in the library.

1009058470 commented 7 months ago

change the fs.inotify.max_user_watches = 8192 in /etc/sysctl.conf

Isn't this an environmental issue? In other words, not a bug in the library.

well, if change that, the program can not be run, it seem it will have a long time in running something in start the Microsoft.Extensions.Hosting.BackgroundService

1009058470 commented 7 months ago

it seems it run a long time on Host.CreateDefaultBuilder(args) .ConfigureServices((hostContext, services) => { services.AddHostedService<T>(); }).UseSystemd().StartAsync();

1009058470 commented 5 months ago

When i try to using a dotnet-stack to view them stack and i find this 

Thread (0x63D11):
  [Native Frames]
  System.Private.CoreLib!System.Threading.Thread.Sleep(int32)
  CallImageServer!CallImageServer.CallImageMain.Main(class System.String[])

**Thread (0x63D1D):
  [Native Frames]
  System.IO.FileSystem.Watcher!System.IO.FileSystemWatcher+RunningInstance.TryReadEvent(value class NotifyEvent&)
  System.IO.FileSystem.Watcher!System.IO.FileSystemWatcher+RunningInstance.ProcessEvents()
  System.IO.FileSystem.Watcher!System.IO.FileSystemWatcher+RunningInstance+<>c.<Start>b__14_0(class System.Object)
  System.Private.CoreLib!System.Threading.Tasks.Task.InnerInvoke()
  System.Private.CoreLib!System.Threading.Tasks.Task+<>c.<.cctor>b__277_0(class System.Object)
  System.Private.CoreLib!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)**
  System.Private.CoreLib!System.Threading.Tasks.Task.ExecuteWithThreadLocal(class System.Threading.Tasks.Task&,class System.Threading.Thread)
  System.Private.CoreLib!System.Threading.Tasks.Task.ExecuteEntryUnsafe(class System.Threading.Thread)
  System.Private.CoreLib!System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(class System.Object)
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart_Context(class System.Object)
  System.Private.CoreLib!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart(class System.Object)

Thread (0x63D1E):
  [Native Frames]
  System.Private.CoreLib!System.Threading.SemaphoreSlim.WaitUntilCountOrTimeout(int32,unsigned int32,value class System.Threading.CancellationToken)
  System.Private.CoreLib!System.Threading.SemaphoreSlim.Wait(int32,value class System.Threading.CancellationToken)
  System.Collections.Concurrent!System.Collections.Concurrent.BlockingCollection`1[Microsoft.Extensions.Logging.Console.LogMessageEntry].TryTakeWithNoTimeValidation(!0&,int32,value class System.Threading.CancellationToken,class System.Threading.CancellationTokenSource)
  System.Collections.Concurrent!System.Collections.Concurrent.BlockingCollection`1+<GetConsumingEnumerable>d__68[Microsoft.Extensions.Logging.Console.LogMessageEntry].MoveNext()
  Microsoft.Extensions.Logging.Console!Microsoft.Extensions.Logging.Console.ConsoleLoggerProcessor.ProcessLogQueue()
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart_Context(class System.Object)
  System.Private.CoreLib!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart()

Thread (0x63DFB):
  [Native Frames]
  System.Net.Sockets!System.Net.Sockets.SocketAsyncEngine.EventLoop()
  System.Net.Sockets!System.Net.Sockets.SocketAsyncEngine+<>c.<.ctor>b__14_0(class System.Object)
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart(class System.Object)

Thread (0x63DFC):
  [Native Frames]
  System.Net.Sockets!System.Net.Sockets.SocketPal.TryCompleteAccept(class System.Net.Sockets.SafeSocketHandle,unsigned int8[],int32&,int&,value class System.Net.Sockets.SocketError&)
  System.Net.Sockets!System.Net.Sockets.SocketAsyncContext.Accept(unsigned int8[],int32&,int&)
  System.Net.Sockets!System.Net.Sockets.SocketPal.Accept(class System.Net.Sockets.SafeSocketHandle,unsigned int8[],int32&,class System.Net.Sockets.SafeSocketHandle&)
  System.Net.Sockets!System.Net.Sockets.Socket.Accept()
  System.Net.Sockets!System.Net.Sockets.TcpListener.AcceptTcpClient()
  TcpServer!TcpServer.TcpFileServer.ConnectAndListenAndSendFiles()
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart_Context(class System.Object)
  System.Private.CoreLib!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib!System.Threading.ThreadHelper.ThreadStart()

Maybe is that FileSystemWatcher to cause this?

dotnet-policy-service[bot] commented 2 months ago

Tagging subscribers to this area: @dotnet/area-extensions-filesystem See info in area-owners.md if you want to be subscribed.