snipe / snipe-it

A free open source IT asset/license management system
https://snipeitapp.com
GNU Affero General Public License v3.0
10.88k stars 3.14k forks source link

Failure to show private files in s3 #10097

Open jorgefm87 opened 3 years ago

jorgefm87 commented 3 years ago

Debug mode

Describe the bug

When saving files in S3, private files then can't be found because the code tries to load them from local.

Reproduction steps

  1. Configure env

PRIVATE_FILESYSTEM_DISK=s3_private PUBLIC_FILESYSTEM_DISK=s3_public

  1. User sign Asset delivery confirmation

  2. The file (signature) is saved in s3

Captura de pantalla de 2021-09-17 16-04-17

  1. Same file saved in s3 is then undefined when I view the signature column

[2021-09-17 16:03:27] production.ERROR: file_get_contents(/var/www/snipeit/storage/private_uploads/signatures/siglog-aacb5585-80b0-4b46-a91b-93585e79aaf3-2021-09-17-032031.png): failed to open stream: No such file or directory

Captura de pantalla de 2021-09-17 15-57-53

Expected behavior

I expect the files hosted in s3 to be showed.

Screenshots

Captura de pantalla de 2021-09-17 15-57-53 Captura de pantalla de 2021-09-17 16-04-17

Snipe-IT Version

v5.2.0 - build 6339 (master)

Operating System

Ubuntu

Web Server

Apache

PHP Version

PHP 7.4.23

Operating System

No response

Browser

No response

Version

No response

Device

No response

Operating System

No response

Browser

No response

Version

No response

Error messages

No response

Additional context

No response

welcome[bot] commented 3 years ago

πŸ‘‹ Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can.

nubpro commented 3 years ago

Perhaps show more of your env vars related to s3 config

snipe commented 3 years ago

Make sure you have

FILESYSTEM_DISK=s3

in your config as well

jorgefm87 commented 3 years ago

I have added FILESYSTEM_DISK and it follows the bug:

PRIVATE_FILESYSTEM_DISK=s3_private PUBLIC_FILESYSTEM_DISK=s3_public FILESYSTEM_DISK=s3

PUBLIC_AWS_SECRET_ACCESS_KEY=null PUBLIC_AWS_ACCESS_KEY_ID=null PUBLIC_AWS_DEFAULT_REGION=eu-west-1 PUBLIC_AWS_BUCKET=xxxx PUBLIC_AWS_URL=https://xxxx.s3.eu-west-1.amazonaws.com PUBLIC_AWS_BUCKET_ROOT=public

PRIVATE_AWS_ACCESS_KEY_ID=null PRIVATE_AWS_SECRET_ACCESS_KEY=null PRIVATE_AWS_DEFAULT_REGION=eu-west-1 PRIVATE_AWS_BUCKET=xxxx PRIVATE_AWS_URL=https://xxxx.s3.eu-west-1.amazonaws.com PRIVATE_AWS_BUCKET_ROOT=private

GregS commented 2 years ago

Having the same problem. I see a FIXME comment in the controller right where this happens. Is S3 support not yet complete, or is this a configuration issue? The docs aren't super clear on how to get S3 working, but I was able to use the artisan command to move files into S3, so it seems my config is at least very close to right.

snipe commented 2 years ago

The required settings are:

# --------------------------------------------
# REQUIRED: UPLOADED FILE STORAGE SETTINGS
# --------------------------------------------

PRIVATE_FILESYSTEM_DISK=s3_private
PUBLIC_FILESYSTEM_DISK=s3_public

# --------------------------------------------
# PUBLIC S3 Settings
# --------------------------------------------
PUBLIC_AWS_SECRET_ACCESS_KEY='XXXXXXXX'
PUBLIC_AWS_ACCESS_KEY_ID=XXXXXXXX
PUBLIC_AWS_DEFAULT_REGION='us-west-2'
PUBLIC_AWS_BUCKET=snipe-flysystem-public-test
PUBLIC_AWS_URL='https:/XXXXXXXXXt.s3-us-west-2.amazonaws.com/'

# --------------------------------------------
# PRIVATE S3 Settings
# --------------------------------------------
PRIVATE_AWS_SECRET_ACCESS_KEY='XXXXXXXXXX'
PRIVATE_AWS_ACCESS_KEY_ID=XXXXXXXXX
PRIVATE_AWS_DEFAULT_REGION='us-west-2'
PRIVATE_AWS_BUCKET=snipe-flysystem-private-test

(The bucket name obviously changes based on your own setup)

GregS commented 2 years ago

Yep, that's exactly what I had, and like I said artisan was able to push the documents there. But when I removed them from the local filesystem, they could no longer be viewed through the UI, the path it was generating for them in UserFilesController::show was still /var/www/...., nothing related to AWS at all.

GregS commented 2 years ago

P.S. There doesn't seem to be any rhyme or reason to which settings need quotes around them and which don't? null shouldn't be quoted, obviously, and I get why numbers might not be (I'm a PHP developer myself), but why does the region have quotes in your example but not the bucket?

urbaned121 commented 1 year ago

snipe/snipe-it:v6.0.14 - docker version and the issue still exists, the same problem. The file (signature) is saved in s3 but not shown in UI
image

STACK TRACE:

`[2023-02-14 13:48:45] production.ERROR: Call to undefined method Monolog\Logger::warn() {"userId":1,"exception":"[object] (Error(code: 0): Call to undefined method Monolog\Logger::warn() at /var/www/html/vendor/laravel/framework/src/Illuminate/Log/Logger.php:308) [stacktrace]

0 /var/www/html/vendor/laravel/framework/src/Illuminate/Log/LogManager.php(668): Illuminate\Log\Logger->__call()

1 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(261): Illuminate\Log\LogManager->__call()

2 /var/www/html/app/Http/Controllers/ActionlogController.php(23): Illuminate\Support\Facades\Facade::__callStatic()

3 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\Http\Controllers\ActionlogController->displaySig()

4 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\Routing\Controller->callAction()

5 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(262): Illuminate\Routing\ControllerDispatcher->dispatch()

6 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(205): Illuminate\Routing\Route->runController()

7 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(721): Illuminate\Routing\Route->run()

8 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Routing\Router->Illuminate\Routing\{closure}()

9 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/AuthenticateSession.php(58): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

10 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Session\Middleware\AuthenticateSession->handle()

11 /var/www/html/vendor/laravel/framework/src/Illuminate/Auth/Middleware/Authenticate.php(44): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

12 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Auth\Middleware\Authenticate->handle()

13 /var/www/html/app/Http/Middleware/AssetCountForSidebar.php(55): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

14 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\AssetCountForSidebar->handle()

15 /var/www/html/vendor/laravel/passport/src/Http/Middleware/CreateFreshApiToken.php(50): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Laravel\Passport\Http\Middleware\CreateFreshApiToken->handle()

17 /var/www/html/app/Http/Middleware/CheckForTwoFactor.php(59): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

18 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\CheckForTwoFactor->handle()

19 /var/www/html/app/Http/Middleware/CheckUserIsActivated.php(47): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

20 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\CheckUserIsActivated->handle()

21 /var/www/html/app/Http/Middleware/CheckLocale.php(37): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

22 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\CheckLocale->handle()

23 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

24 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\VerifyCsrfToken->handle()

25 /var/www/html/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

26 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse->handle()

27 /var/www/html/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

28 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Cookie\Middleware\EncryptCookies->handle()

29 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

30 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(723): Illuminate\Pipeline\Pipeline->then()

31 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(698): Illuminate\Routing\Router->runRouteWithinStack()

32 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(662): Illuminate\Routing\Router->runRoute()

33 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(651): Illuminate\Routing\Router->dispatchToRoute()

34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(167): Illuminate\Routing\Router->dispatch()

35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}()

36 /var/www/html/vendor/livewire/livewire/src/DisableBrowserCache.php(19): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

37 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Livewire\DisableBrowserCache->handle()

38 /var/www/html/vendor/barryvdh/laravel-debugbar/src/Middleware/InjectDebugbar.php(66): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

39 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Barryvdh\Debugbar\Middleware\InjectDebugbar->handle()

40 /var/www/html/vendor/fruitcake/laravel-cors/src/HandleCors.php(38): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

41 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Fruitcake\Cors\HandleCors->handle()

42 /var/www/html/app/Http/Middleware/PreventBackHistory.php(23): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

43 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\PreventBackHistory->handle()

44 /var/www/html/app/Http/Middleware/SecurityHeaders.php(26): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

45 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\SecurityHeaders->handle()

46 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

47 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()

48 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull->handle()

49 /var/www/html/app/Http/Middleware/CheckForDebug.php(25): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

50 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\CheckForDebug->handle()

51 /var/www/html/app/Http/Middleware/CheckForSetup.php(25): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

52 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\CheckForSetup->handle()

53 /var/www/html/vendor/fideloper/proxy/src/TrustProxies.php(57): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

54 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Fideloper\Proxy\TrustProxies->handle()

55 /var/www/html/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

56 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\View\Middleware\ShareErrorsFromSession->handle()

57 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

58 /var/www/html/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(64): Illuminate\Session\Middleware\StartSession->handleStatefulRequest()

59 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Session\Middleware\StartSession->handle()

60 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

61 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance->handle()

62 /var/www/html/app/Http/Middleware/NoSessionStore.php(28): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

63 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\Http\Middleware\NoSessionStore->handle()

64 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

65 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(142): Illuminate\Pipeline\Pipeline->then()

66 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(111): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter()

67 /var/www/html/public/index.php(52): Illuminate\Foundation\Http\Kernel->handle()

68 {main}

"}`

fe80 commented 3 months ago

Hello @snipe

I've the same issue, I think they is 2 separated issues; The first one is an error about the warning methods name cc my patch.

But after that still not work, this function manage only local files. I guess it's suppose to have a case with config("filesystems.disks.$disk.driver") for redirect the link to the s3 bucket location, but I don't know how.

Do you want I create a PR with my first patch ?

Edit: It's work when I force the s3 (patch):

        $file = 'private_uploads/signatures/'.$filename;
        $disk = config('filesystems.default');
        return redirect()->away(Storage::disk($disk)->temporaryUrl($file, now()->addMinutes(5)));