Urgent / Critical Problem with 'christitus.com/win' Web Re-direct !!

og-mrk commented 2 months ago

First notice by the author of this issue ticket

I noticed something odd with an Issue Ticket recently.. it was reporting a problem with Numlock Toggle Tweak (Disable/Enable Numlock on Startup). ( here's a link to that issue )

Of course, when I saw this I said to myself "Well this is weird, this has been reported and fixed by Chris him self.. A WEEK AGO!"

then I looked into the Git Logs trying to find anything out of the ordinary.. nothing weird to see.

tried running WinUtil, and first thought ".. Old Version Maybe?", and indeed.. I was getting an older version (version 24.06.11) instead of the latest version found on GitHub (version 24.06.20, at the time of writing).

Screenshots (evidence to reference)

Additional Info

By looking at this.. it isn't something one person will have, as the issue I've referenced above is by another person, who's probably getting an older version (a version containing the Numlock Bug Issue).

This is a CRITICAL Issue, should be resolved as soon as possible. @ChrisTitusTech

og-mrk commented 2 months ago

Note for anyone reading this

Please try running WinUtil through the Admin PowerShell Terminal, command: irm christitus.com/win | iex, click the Gear Icon on the top-right corner (Next to the X Icon), and click the About Drop-down Item, see the Version Field, is it up-to-date to the one shown on GitHub Repo? ( link to version line(s) in 'winutil.ps1' file, line 11 & line 48 )

og-mrk commented 2 months ago

Looked into the HTTP Requests themselves, and it turns out it was permanently set to re-direct to the winutil.ps1 24.06.11 GitHub Release, here's a link to it

Screenshots

MyDrift-user commented 2 months ago

I thaught that this was intentional, because PRs now are made directly to the main branch and the Releases are autocreated when compiling. He once said, that dev users should use the raw link (old method), which is the same as the release version. These would not be Full Version Releases then, therefore the delay in changing the redirect. But I don't remember him actually talking/writing about the way he does/wants to do the redirects nowadays.

og-mrk commented 2 months ago

I thaught that this was intentional, because PRs now are made directly to the main branch and the Releases are autocreated when compiling. He once said, that dev users should use the raw link (old method), which is the same as the release version. These would not be Full Version Releases then, therefore the delay in changing the redirect. But I don't remember him actually talking/writing about the way he does/wants to do the redirects nowadays.

The reason behind delaying a release is fair enough, but wouldn't be better if the releases were made manual only? Of course I don't really know what Chris plan for the "deployment/release model", but testing the changes on the main branch, and reviewing what has changed as well is by far the best thing to do, even though there might be Human error in the process.. and the obvious time sinking it might take.

Just a humble & possible solution that might help

Now I don't know if the idea I'm proposing will work or not.. so take it with a grain of salt 😅

I've thought of a system that might satisfy every need there is.. but at the cost of an extra request (or two) made to GitHub, that being.. just don't download the winutil.ps1 file directly :/ What I mean by that is to have a File in the repo, lets call it winutil_launcher.ps1 for example, and this's the file the User will request, inside that script file you'll have a few lines of code, which figures out what's the latest stable release of WInUtil (like version 24.06.20), with again.. Release being Manual Only.

Now despite being a simple system.. its quite a flexible one, I mean if we want, we could implement a previous idea of mine ( explained in issue #2060 ), by making a new file, lets call it winutil_launcher_dev.ps1, which will do:

[Client Side] Request this winutil_launcher_dev.ps1 through a direct github link (like https://github.com/.../winutil_launcher_dev.ps1), or through a web re-direct (like christitus.com/windev), after getting this file downloaded, run it, then proceed to the next step.
check if git is installed
- Installed? -> Proceed to the next step
- Not Installed? -> Try Installing it using Invoke-WebRequest, Chocolatey Package Manager, or WinGet Package Manager (or even an entirely different approach to solving this problem)
do a Shallow Clone of winutil repo into the Temp Folder
[Optional/Recommend Doing] Check & Set Execution Policy, and the Language Mode (if possible)
run the Compile.ps1 script on the Client Computer, then run winutil.ps1 if the Compilation was successful.
Enjoy your Dev Build, which has been Built From Source Code, meaning no Backdoor to speak of, Jai Tan 😉

This of course.. for the Dev Branch, designed for people who want the latest changes/commits (and therefore, the most unstable), aim at testers using Virtual Machines (or just people want to see what's being worked on, and report any issues if encountered), or devs who want to live on the Bleed-Edge 🎸 🩸 , both cases are totally fine IMHO

And most importantly my fellow contributors.. Say Goodbye to Compiled Files (winutil.ps1) in PRs 🕶️ 😄 Because there'll be no need to ship with winutil.ps1 in the repo, and have Git ignore it completely (using .gitignore file).

Downsides of it (the one I could think of) is the extra time to get the winutil.ps1 file (from GitHub Releases, using their REST API), this is sadly un-avoidable, as more request you make, the request after depending on the previous to finish (blocking requests), the longer you'll have to wait.. this will depend on how close you're to the server (Geolocation relative to the server, at least the closest one), and is your internet stable or not (request might fail, and you'll have to request again, adding more wait time)

MyDrift-user commented 2 months ago

I don't have extensive knowledge about cloud environments, but I believe that eliminating the need for client-side setup would significantly enhance the user experience. WinUtil is designed to be modular, and requiring local installation and compilation would undermine the purpose of an invoked script. Ideally, we could implement a trigger that compiles WinUtil using a cloud-based solution like Cloudflare Workers, thereby reducing the burden on the client.

Also if we are using solutions like Workers we could also use them to identify the newest release and use that one for regular users.

Those changes would remove the regional issues that people have (eg. India) because the client does not have to reach github anymore.

Upsides:

Devs compile the newest version without the need of any setup/Client ressources.
Github Domain Blocking won't be a big problem anymore
better Management of Releases
Less API reqests to GitHub

Downsides:

Potential additional costs (Cloudflare is known for generous pricing)
Could reduce/increase loading time, but Cloudflare is known to be fast.

Marterich commented 2 months ago

Another different idea I proposed in a issue (I don't exactly remember when) would be to implement a model like Microsoft with the winget repo where new commits are automatically generated and marked as pre release Then the compilation of a new stable release could be triggered manually

Real-MullaC commented 2 months ago

Another idea - have a winutildev github where people can do PR's on and that is a testing ground then when it is tested and good it gets transfered to the main winutil page for stable release.

MyDrift-user commented 2 months ago

Another idea - have a winutildev github where people can do PR's on and that is a testing ground then when it is tested and good it gets transfered to the main winutil page for stable release.

That would be similar to the old method of using a testbranch. I don't know how well liked that would be thinking about we "just" switched from a similar setup.

I personally would also prefer to have a project on one repository, to keep maintainability and reduce confusion.

But it does also have its ups, so I think the best way would be to merge ideas from this issue to come up with a solution.

I propose following combination of Ideas:

The solution from Marerich is in my opinion almost a must have at this point.
We could combine that with a Cloudflare worker/Github Action ..., to always get the newest release/prerelease and link it to the url (christitus.com/win & christitus.com/windev), so that this doesn't have to be done manually and users can use WinUtil even with the blocked GitHub url in some countries.
Pre-Releases will still get released automatically at compilation
Full releases have to be done manually

og-mrk commented 2 months ago

Another idea - have a winutildev github where people can do PR's on and that is a testing ground then when it is tested and good it gets transfered to the main winutil page for stable release.

That would be similar to the old method of using a testbranch. I don't know how well liked that would be thinking about we "just" switched from a similar setup.

I personally would also prefer to have a project on one repository, to keep maintainability and reduce confusion.

But it does also have its ups, so I think the best way would be to merge ideas from this issue to come up with a solution.

I propose following combination of Ideas:

The solution from Marerich is in my opinion almost a must have at this point.

We could combine that with a Cloudflare worker, to always get the newest release/prerelease and link it to the url (christitus.com/win & christitus.com/windev), so that this doesn't have to be done manually and users can use WinUtil even with the blocked GitHub url in some countries.

Pre-Releases will still get released automatically at compilation

Full releases have to be done manually

Nice summary @MyDrift-user 👍 All the points you provided are well done, and of course, even if we couldn't do the Cloudflare part, we could do a workaround to achieve a similar result.

EDIT: Trying to do a similar fix for Region locking bit without Cloudflare might be.. difficult? I'm not that into Web Dev/Technology, so I can't speak on this part...

MyDrift-user commented 2 months ago

Thanks @og-mrk :)

If we want to fix that, we'll have to make it so the user never actually has to make a request to github himself, which is as you said difficult without another provider/temp. hoster/...

The Idea I had with the worker is following:

user makes request to domain
cloudflare worker gets newest release/prerelease and hosts it himself.
user gets access to that "self" hosted file, not having to make a request himself to github.com/githubusercontent.com

The Idea came because I once heard (or thaught i heard) Chris talking about his website, that is apparently also hosted on Cloudflare using Cloudflare Pages, so the domain would already be linked to cloudflare making that the easiest setup when needing another provider than GitHub.

That was also just a random idea I had, but I'm not a Web Dev either ...

Marterich commented 2 months ago

I would actually argue, that using a CDN like Cloudflare is not really necessary at all. Chris already hosts his own website (the hosting provider really doesn't matter) and winutil itself (at least at the moment) is quite small, so nothing would prevent a script or GitHub Action to run, that simply copies winutil onto the Webserver so that it could be queried from there. Of course like this, Chris's Webserver would need to handle the load and the traffic of the users downloading the script which directly correlates to potentially higher prices (which are not necessary when fetching from GitHub)

Edit: the discussion drifted from "how to fix an url" over, "how to setup the releases better" to "how to fully revamp the whole script distribution". I think those are all valid points that can be discussed, but it should probably be done in separate threads (e.g. separate discussion topics) and not all jumbled together in this issue

og-mrk commented 2 months ago

I would actually argue, that using a CDN like Cloudflare is not really necessary at all. Chris already hosts his own website (the hosting provider really doesn't matter) and winutil itself (at least at the moment) is quite small, so nothing would prevent a script or GitHub Action to run, that simply copies winutil onto the Webserver so that it could be queried from there. Of course like this, Chris's Webserver would need to handle the load and the traffic of the users downloading the script which directly correlates to potentially higher prices (which are not necessary when fetching from GitHub)

Well the handling and responsibility of making it work is quite daunting, as according to Chris, every day, more than 10 thousand requests are made to christitus.com/win.. which will make changing things a bit difficult + depending on what web service Chris uses for hosting, DDOS attacks will be annoying to handle (not if, but when they happen)

Edit: the discussion drifted from "how to fix an url" over, "how to setup the releases better" to "how to fully revamp the whole script distribution". I think those are all valid points that can be discussed, but it should probably be done in separate threads (e.g. separate discussion topics) and not all jumbled together in this issue

True 😂 Overall, I had fun discussing the different ideas and possible solutions to use to make the Distribution process more streamlined and better on both the Devs and Users.. but it's better to discuss it in a separate issue ticket (we shouldn't drift further from the main topic at hand).

ChrisTitusTech commented 2 months ago

@og-mrk @Marterich @MyDrift-user Great suggestions all! I really like the latest release and pre-release approach with a manual full release.

Website is actually hosted on cloudflare using github... which is pretty much how winutil is handled hehe. So I don't have a "host" per se because it is distributed like a netlify static site using Cloudflare CDNs that are copied from the website repo. This makes my website have no attack surface and fast.

Winutil link from christitus.com/win is because I'm lazy and wanted a smaller url to type. I love the idea of just linking this to latest and dev making another windev url.

Thank you!

Marterich commented 2 months ago

@ChrisTitusTech Take a look at #2060 Regarding the dev url (together with pre-releases) there @og-mrk and I discussed this a bit more in detail already :)

ChrisTitusTech commented 2 months ago

@Marterich merged this in and I really like it. One thing I need a bit of clarity on is there a global "pre-release" tag like there is on latest? I switched the christitus.com/win link to push to latest, but I wanted to make a christitus.com/windev link to the "latest pre-release"

Real-MullaC commented 2 months ago

@Marterich merged this in and I really like it. One thing I need a bit of clarity on is there a global "pre-release" tag like there is on latest? I switched the christitus.com/win link to push to latest, but I wanted to make a christitus.com/windev link to the "latest pre-release"

Yes there is. I think this is what you mean.

Marterich commented 2 months ago

@Marterich merged this in and I really like it. One thing I need a bit of clarity on is there a global "pre-release" tag like there is on latest? I switched the christitus.com/win link to push to latest, but I wanted to make a christitus.com/windev link to the "latest pre-release"

@ChrisTitusTech Annoyingly I think not really.

HTML site that queries the Latest dev Release via JS

```html Redirecting to the latest pre-release version... ```

This will still not solve the issue because afaik there's no option in PS to Download and Run a Website including JS

Idea 1: Create a job that periodically downloads the script onto your webserver or create a Github Action that uploads the file via FTP

Idea 2: Check if you can set your DNS Entries via API. That way you could create a GitHub Action that dynamically updates the CloudFlare DNS Record.

Idea 3 (Solution): I'll see if it's possible to create a small PowerShell Script that can be embedded as text which will act as a loader to fetch the newest Version. I'll update the commend if I manage to create it :)

Edit: Create the following static page on your windev link. If this is invoked via irm https:/[...] | iex it'll query the newest pre-release, and run it.

We'll have to see how much AV solution like these multistage IEX calls, but for the moment it should work

# Function to fetch the latest release tag from the GitHub API
function Get-LatestRelease {
    try {
        $releases = Invoke-RestMethod -Uri 'https://api.github.com/repos/ChrisTitusTech/winutil/releases'
        $latestRelease = $releases | Select-Object -First 1
        return $latestRelease.tag_name
    } catch {
        Write-Host "Error fetching release data: $_" -ForegroundColor Red
        return $null
    }
}

# Function to redirect to the latest pre-release version
function RedirectToLatestPreRelease {
    $latestRelease = Get-LatestRelease
    if ($latestRelease) {
        $url = "https://raw.githubusercontent.com/ChrisTitusTech/winutil/$latestRelease/winutil.ps1"
        Invoke-RestMethod $url | Invoke-Expression
    } else {
        Write-Host 'Unable to determine latest pre-release version.' -ForegroundColor Red
    }
}

# Call the redirect function

RedirectToLatestPreRelease

MyDrift-user commented 2 months ago

Edit: Create the following static page on your windev link. If this is invoked via irm https:/[...] | iex it'll query the newest pre-release, and run it.

Very cool and works super fast.

I have one short debug suggestion:

If there is no network the dev still gets a error because we are using write-error:

Get-LatestRelease : Error fetching release data: The remote name could not be resolved: 'api.github.com'
At C:\Users\SOMEONE\Desktop\test.ps1:15 char:22
+     $latestRelease = Get-LatestRelease
+                      ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Get-LatestRelease

I think these don't really help in this case, so I would personally suggest a write-host maybe with -ForegroundColor Red to keep it clean and still see the most important infos if there is an issue.

Marterich commented 2 months ago

@MyDrift-user Even though I think this is really a case that should pretty much never occur, I updated my code snippet in the comment. Also, I added logic to pull the latest full release if no pre-releases should exist. (I added no error handling regarding no Releases at all because if that's the case the script doesn't matter anyways anymore so fuck it xD)

MyDrift-user commented 2 months ago

@Marterich If I see it correctly we don't need that fallback, this solution rather makes other Issues:

If the latest release is a Full release and not a pre-release, it still gets the older pre-release instead of the newest.
If it can't take the newest pre-release it directly goes to the full release instead of the next available release.
A DEV-/Pre-Release is made to test & find issues, if there is an issue and we just take the next release issues may not get detected as fast.

The Solution would be to just take the newest release no matter what kind it is. (I don't mean the tag "latest")

$preRelease = $releases | Select-Object -First 1

Marterich commented 2 months ago

@ChrisTitusTech @MyDrift-user I created a linked PR that initializes the file at the base of the repo and describes what link needs to be done to get https://christitus.com/windev to work

ChrisTitusTech / winutil