datalust / seq-tickets

Issues, design discussions and feature roadmap for the Seq log server
https://datalust.co/seq
96 stars 5 forks source link

Upgrade does not start Seq #2149

Closed DanAvni closed 3 months ago

DanAvni commented 5 months ago

I saw this on the last few versions: When upgrading Seq the installer at the end displays a checkbox "Install or start the Seq service on exit". The checkbox is checked but when I click finish the installer exists and Seq is not started. I then have to go to services and start the service manually.

liammclennan commented 5 months ago

Hi @DanAvni ,

That's odd. Anything in Seq's log?

DanAvni commented 5 months ago
{"@t":"2024-03-31T05:05:30.6667270Z","@mt":"Shutting down input channel; {AcceptedCount} queued events will be written","AcceptedCount":0,"SourceContext":"Seq.Server.Features.Apps.Inputs.InputChannel"}
{"@t":"2024-03-31T05:05:30.6673598Z","@mt":"Stopping broadcast channel","SourceContext":"Seq.Engine.Broadcast.BroadcastChannel"}
{"@t":"2024-03-31T05:05:30.8537503Z","@mt":"Shutting down broadcast channel with {AllowableFlush} flushable event batches (+{Tolerance} tolerance)","AllowableFlush":0,"Tolerance":15,"SourceContext":"Seq.Engine.Broadcast.BroadcastChannel"}
{"@t":"2024-03-31T05:05:30.8546692Z","@mt":"Stopping {AppCount} running apps","AppCount":3}
{"@t":"2024-03-31T05:05:32.9476957Z","@mt":"Closing event store","SourceContext":"Seq.Engine.Cluster.StorageNodeCoordinator"}
{"@t":"2024-03-31T05:05:33.4316399Z","@mt":"Closing metastore","SourceContext":"Quince.Storage.Native.NativeDocumentStore"}
{"@t":"2024-03-31T05:05:33.6556462Z","@mt":"Server stopped cleanly"}

...Upgraded seq and had to start it manually...the log continues with

{"@t":"2024-03-31T05:06:22.0990518Z","@mt":"Seq {SeqVersion} running on OS {OSCaption}","SeqVersion":"2024.2.11240","OSCaption":"Microsoft Windows Server 2019 Standard","SourceContext":"Seq.Server.Features.Diagnostics.EnvironmentInformationWriter"}
{"@t":"2024-03-31T05:06:22.1467895Z","@mt":"Running in Windows Service host"}
{"@t":"2024-03-31T05:06:22.1592367Z","@mt":"Configuring HTTP.sys web server","SourceContext":"Seq.Server.Web.Host.WebHostBuilderExtensions"}
{"@t":"2024-03-31T05:06:22.6276964Z","@mt":"Opening event store at {StreamPath}","StreamPath":"Z:\\Seq\\Stream\\stream.flare","SourceContext":"Seq.Engine.Cluster.StorageNodeCoordinator"}
{"@t":"2024-03-31T05:06:23.3683562Z","@mt":"Ingestion enabled","SourceContext":"Seq.Engine.Cluster.StorageNodeCoordinator"}
nblumhardt commented 5 months ago

Thanks for the follow-up, @DanAvni. Is your Seq instance a "named" one?

DanAvni commented 5 months ago

@nblumhardt , no they both are not named

nblumhardt commented 5 months ago

Looking into this again - when you click "Close" on the final page of the installer, does the small, console-styled "Seq Service Administration" window pop up? If so, are you able to capture the text it shows, or screenshot it?

If the little post-install window isn't popping up, my guess would be that it's an issue related to the installed .NET Framework version on the machine, group policy, or UAC, though I don't have a good starting point to investigate from. Trying to open "Seq Service Administration" from the Windows Start Menu might offer some clues.

DanAvni commented 5 months ago

The popup does not appear at all

Running Seq admin from the start menu works without any problem. Next upgrade I will try to use sysinternals process monitor to maybe catch the process activation attempt and see what happened

DanAvni commented 4 months ago

So I ran process monitor just before clicking finish on the upgrade and have also looked in our EDR. I can see msiexec.exe try to run "C:\Program Files\Seq\Admin\seq.administration.exe" and that is it (In the EDR it is displayed as create the file and modify it but the process is never started and I do not see it in the EDR)

nblumhardt commented 4 months ago

Thanks for the follow-up @DanAvni.

I wouldn't necessarily recommend upgrading with EDR/AV etc. switched off, but it seems like the issue may be at that level, we haven't observed the issue elsewhere so far, so something specific to the software/configuration on the machine is my best bet right now.

DanAvni commented 4 months ago

On my dev machine I have the same EDR/AV (Sentinel1) and the upgrade works and launches the admin at the end. It is only on the Seq licensed servers I have this issue. Also, If S1 would have blocked it I would have seen a notification and it does not show anything.

nblumhardt commented 4 months ago

Another possibility - is Serilog.dll in the GAC on the problem server? (Or any Serilog.* dll?)

DanAvni commented 4 months ago

Sorry but no. Not on my PC and not on the servers

nblumhardt commented 3 months ago

Hi Dani; unfortunately we haven't found any clues or received any similar reports at our end.

While we're unable to narrow down the cause, we'll convert this to a discussion to keep the tracker focused, and then reopen as an issue when there's something actionable to go forward with. Thanks!