search

thilojaeggi / WinGetty

An open source REST Backend for creating a private WinGet Repo without having to rely on cloud dependencies.
https://wingetty.dev
GNU Affero General Public License v3.0
168 stars 15 forks source link

Support for installer url instead of file upload #23

Closed kdpuvvadi closed 8 months ago

kdpuvvadi commented 8 months ago

Is your feature request related to a problem? Please describe. Instead selecting file and uploading it, why not select a fully qualified url on a file server. Some of our installers are around 4 to 7 gigs. Hosted on a file server readily available.

Describe the solution you'd like or how it could be implemented

  1. Adding an option to select between file or url.
  2. Download the file, calculate the hashes and purge files or Option to add file hash

Additional context Large installer are already taking ton of storage on file servers. Uploading same files makes it redundent and costs storage.

thilojaeggi commented 8 months ago

v.0.0.7 will have this!

thilojaeggi commented 8 months ago

This is now in #27 v0.0.6

kdpuvvadi commented 8 months ago

Installing application is erroring out.

Winget output

$ winget install --id Hashicorp.Terraform -s puvvadi
Found Terraform [Hashicorp.Terraform] Version 1.6.3
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://winget.local.puvvadi.net/api/download/Hashicorp.Terraform/1.6.3/x64/both
An unexpected error occurred while executing the command:
Download request status is not success.
0x801901f4 : unknown error

Winget Log

'C:\Users\kd\AppData\Local\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\puvvadi\installed.db'
2023-11-04 10:10:11.024 [SQL ] Opening SQLite connection #2: 'C:\Users\kd\AppData\Local\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\puvvadi\installed.db' [2, 0]
2023-11-04 10:10:11.026 [REPO] Opened SQLite Index with version [1.6], last write [2023-11-04 09:40:31.000]
2023-11-04 10:10:11.084 [REPO] Sending http GET request to: https://winget.local.puvvadi.net/wg/packageManifests/Hashicorp.Terraform
2023-11-04 10:10:11.170 [REPO] Response status: 200
2023-11-04 10:10:11.173 [REPO] Finding installed package from available package using system reference search: Query:[none] Include:NormalizedNameAndPublisher='terraform'+'hashicorp'[Exact]
2023-11-04 10:10:11.186 [REPO] Opening pinning index
2023-11-04 10:10:11.186 [REPO] Opening existing pinning index
2023-11-04 10:10:11.186 [REPO] Opening SQLite Index for ReadWrite at 'C:\Users\kd\AppData\Local\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\pinning.db'
2023-11-04 10:10:11.186 [SQL ] Opening SQLite connection #3: 'C:\Users\kd\AppData\Local\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\pinning.db' [2, 0]
2023-11-04 10:10:11.187 [REPO] Opened Pinning Index with version [1.0], last write [2023-09-30 12:34:38.000]
2023-11-04 10:10:11.188 [CLI ] Found one app. App id: Hashicorp.Terraform App name: Terraform
2023-11-04 10:10:11.191 [CLI ] Starting installer selection.
2023-11-04 10:10:11.192 [CLI ] Completed installer selection.
2023-11-04 10:10:11.192 [CLI ] Manifest fields: Name [Terraform], Version [1.6.3]
2023-11-04 10:10:11.211 [CLI ] Found existing installer file at 'C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d'. Verifying file hash.
2023-11-04 10:10:11.212 [CLI ] Hash does not match. Removing existing installer file C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d
2023-11-04 10:10:11.213 [CLI ] Failed to remove installer file. Reason: remove: unknown error: "C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d"
2023-11-04 10:10:11.215 [CLI ] Generated temp download path: C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d
2023-11-04 10:10:11.216 [CORE] Downloading to path: C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d
2023-11-04 10:10:11.216 [CORE] DeliveryOptimization downloading from url: https://winget.local.puvvadi.net/api/download/Hashicorp.Terraform/1.6.3/x64/both
2023-11-04 10:10:14.370 [CORE] DeliveryOptimization error: 0x801901f4, extended error: 0x00000000
2023-11-04 10:10:14.371 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FF921E7A465: (caller: 00007FF921E7AD01) Exception(1) tid(1148) 801901F4 Internal server error (500).

2023-11-04 10:10:14.389 [CORE] Started applying motw to C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d with zone: 3
2023-11-04 10:10:14.395 [CORE] Finished applying motw
2023-11-04 10:10:14.395 [CORE] WinINet downloading from url: https://winget.local.puvvadi.net/api/download/Hashicorp.Terraform/1.6.3/x64/both
2023-11-04 10:10:14.706 [CORE] Download request failed. Returned status: 500
2023-11-04 10:10:14.706 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!00007FF921E743D6: (caller: 00007FF921E75C49) Exception(2) tid(1148) 801901F4 Internal server error (500).
    Msg:[Download request status is not success.] 

2023-11-04 10:10:14.852 [CLI ] Failed to download, waiting a bit and retry. Url: https://winget.local.puvvadi.net/api/download/Hashicorp.Terraform/1.6.3/x64/both
2023-11-04 10:10:15.353 [CORE] Downloading to path: C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d
2023-11-04 10:10:15.354 [CORE] DeliveryOptimization downloading from url: https://winget.local.puvvadi.net/api/download/Hashicorp.Terraform/1.6.3/x64/both
2023-11-04 10:10:18.461 [CORE] DeliveryOptimization error: 0x801901f4, extended error: 0x00000000
2023-11-04 10:10:18.461 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FF921E7A465: (caller: 00007FF921E7AD01) Exception(3) tid(1148) 801901F4 Internal server error (500).

2023-11-04 10:10:18.463 [CORE] Started applying motw to C:\Users\kd\AppData\Local\Temp\WinGet\Hashicorp.Terraform.1.6.3\e61d94fd72d13dd19f9bf0480be3abb8e8ec432400155c0f0c0ca47b6233750d with zone: 3
2023-11-04 10:10:18.465 [CORE] Finished applying motw
2023-11-04 10:10:18.465 [CORE] WinINet downloading from url: https://winget.local.puvvadi.net/api/download/Hashicorp.Terraform/1.6.3/x64/both
2023-11-04 10:10:18.672 [CORE] Download request failed. Returned status: 500
2023-11-04 10:10:18.672 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!00007FF921E743D6: (caller: 00007FF921E75C49) Exception(4) tid(1148) 801901F4 Internal server error (500).
    Msg:[Download request status is not success.] 

2023-11-04 10:10:18.740 [CLI ] Caught wil::ResultException: D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!00007FF921E743D6: (caller: 00007FF921E75C49) Exception(4) tid(1148) 801901F4 Internal server error (500).
    Msg:[Download request status is not success.]

Wingetty ouput

[2023-11-04 10:10:19,127] ERROR in app: Exception on /api/download/Hashicorp.Terraform/1.6.3/x64/both [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/app/app/api_routes.py", line 407, in download
    debugPrint(f"Installer path: {installer_path + '/' + installer.file_name}")
TypeError: can only concatenate str (not "NoneType") to str

On every retry from winget, it's same thing.

thilojaeggi commented 8 months ago

Is this by any chance an external installer?

thilojaeggi commented 8 months ago

Totally found the error just now, will be fixed in the next version

thilojaeggi commented 8 months ago

Should be fixed in v0.0.7 now