Content deployment error

[johnchoicl]

Reproduction

Bug summary

My settings are Umbraco on premise v11 - latest Umbraco Deploy on premise - yearly support subscription As the Storage Provider(media files) we use [https://github.com/adam-werner/Our.Umbraco.StorageProviders.AWSS3] to host the files on AWS S3.

With Umbraco Deploy, the schema updates works perfectly. However, the contents push fails if there's medai file in the source content even with the schemas are in sync across the source and the target environments. If there's no media file/images, then the content push works well.

The error details can be found in the Community Forum post where I originally reported this issue on.

[https://our.umbraco.com/forum/using-umbraco-and-getting-started/111767-umbraco-deploy-error]

I've also created an official support ticket (support.umbraco.com/hc/requests/78329).

After a few exhcnages, I've been advised to file a bug report here and the support team advised that this may require changes in the Umbraco Deploy to support the S3 buckets after the support guy consulted the Umbraco dev team.

I will leave the investigations to you but it seems like the StorageProvider I use is not compatible with Umbraco Deploy. If that's the case, is there any other AWS S3 bucket storage provider that Umbraco Deploy supports?

By the way, moving away from S3 bucket is not an option for our team although I am happy to swap out the provider to another s3 butcket storage provider.

Specifics

full error details are found here [https://our.umbraco.com/forum/using-umbraco-and-getting-started/111767-umbraco-deploy-error]

Steps to reproduce

• make sure source and target schemas are all in sync
• push some contents that contains a media file
• Then I get "Invalid path" error (error details are here [https://our.umbraco.com/forum/using-umbraco-and-getting-started/111767-umbraco-deploy-error])

If there's no media file/images in the source content, then the content push works well. This bug only happens when there's a media file to push to the target.

Expected result

Expected to successfully push the content to the target env.

Actual result

error details are here [https://our.umbraco.com/forum/using-umbraco-and-getting-started/111767-umbraco-deploy-error]

[AndyButland]

Hi John - I was looking at this case with support the other week, and seems my last response may have been missed. It was two-fold - partly an effort to first set up using S3 media storage myself (that I wasn't able to get working with the package you used) and secondly a test case I was looking to put together to determine if the issue lies with Deploy, the S3 package or the CMS.

I'll paste here and maybe you can advise on a couple of things.

1. Can you help me get the S3 package set up and working by seeing what I've done and what I may have missed/is different on your side?
2. Can you have a look at the test case I've provided and see what result you get in your setup?

Thanks - I'll now copy the response below.

---

I've been having a look today to see if I could get this small test case working. Unfortunately I haven't quite... in the sense that I haven't been able to get S3 media storage working at all. I'm getting credentials errors, which is probably something I'm doing wrong.

But maybe between us and the customer we can make a bit of progress anyway... so here are two requests/suggestions.

1. I've not worked with AWS in years so I'm likely missing something obvious, but I'll share what I've tried to get the S3 package they are using setup, and maybe the customer can advise what step I might be missing:

• Added package reference: <PackageReference Include="Our.Umbraco.StorageProviders.AWSS3" Version="1.1.1" />
• Created an AWS account, an S3 bucket and a user with S3 permissions and an access key.
• Have added .AddAWSS3MediaFileSystem()and u.UseInstallerEndpoints(); to Startup.cs
• Have also added the following in StartUp.ConfigureServices()

AWSOptions awsOptions = new AWSOptions
{
    Profile = "local-test-profile",
    Credentials = new BasicAWSCredentials("...", "...")
};
services.AddDefaultAWSOptions(awsOptions);

• And these to my appSettings.config:

  "Umbraco": {
    "CMS": {
    },
    "Storage": {
      "AWSS3": {
        "Media": {
          "BucketName": "umbraco-test-media"
        }
      }
    }
  },
  "AWS": {
    "Profile": "local-test-profile",
    "Region": "eu-north-1"
  },

However with this, when I try to create a media item in the back office I get this error:

Unable to get IAM security credentials from EC2 Instance Metadata Service

Maybe that can see something I've missed here to get it working?

2. What I was looking to do was to create a short code snippet that verifies the "shadow file system" in the CMS with S3, which I've done with this code.

using Microsoft.AspNetCore.Mvc;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.IO;
using Umbraco.Cms.Core.PropertyEditors;
using Umbraco.Cms.Core.Services;
using Umbraco.Cms.Core.Strings;
using Umbraco.Cms.Web.Common.Controllers;
using IScopeProvider = Umbraco.Cms.Infrastructure.Scoping.IScopeProvider;

namespace TestS3.Controllers
{
    public class TestController : UmbracoApiController
    {
        private readonly IScopeProvider _scopeProvider;
        private readonly IMediaService _mediaService;
        private readonly IContentTypeBaseServiceProvider _contentTypeBaseServiceProvider;
        private readonly MediaFileManager _mediaFileManager;
        private readonly MediaUrlGeneratorCollection _mediaUrlGeneratorCollection;
        private readonly IShortStringHelper _shortStringHelper;

        public TestController(
            IScopeProvider scopeProvider,
            IMediaService mediaService,
            IContentTypeBaseServiceProvider contentTypeBaseServiceProvider,
            MediaFileManager mediaFileManager,
            MediaUrlGeneratorCollection mediaUrlGeneratorCollection,
            IShortStringHelper shortStringHelper)
        {
            _scopeProvider = scopeProvider;
            _mediaService = mediaService;
            _contentTypeBaseServiceProvider = contentTypeBaseServiceProvider;
            _mediaFileManager = mediaFileManager;
            _mediaUrlGeneratorCollection = mediaUrlGeneratorCollection;
            _shortStringHelper = shortStringHelper;
        }

        public IActionResult CreateMedia()
        {
            using var scope = _scopeProvider.CreateScope(scopeFileSystems: true);
            using var stream = System.IO.File.OpenRead("c:\\temp\\test-image.jpg");

            var mediaItem = _mediaService.CreateMedia("Test Image", -1, "Image");
            mediaItem.SetValue(
                _mediaFileManager,
                _mediaUrlGeneratorCollection,
                _shortStringHelper,
                _contentTypeBaseServiceProvider,
                Constants.Conventions.Media.File,
                "test.jpg",
                stream);

            _mediaService.Save(mediaItem);

            scope.Complete();
            return Ok("Done");
        }
    }
}

This works with the default disk based media storage, and it works when I use Azure and the Umbraco.StorageProviders.AzureBlob package.

I'd like to know if it works with S3. As I haven't been able to get S3 working at all, I can't verify this. But perhaps the customer can given they have it working for other media operations?

Please let me know if that helps or if you get any further feedback.

[johnchoicl]

Hi Andy, Thanks for looking into the issue. Yes, I think I might have missed your last message completely.

Below is my Umbraco section in the asspSettings. Note the configs under Storage->AWSS3.

The other thing I’ve noticed is below is missing. i.e. UseAWSS3MediaFileSystem()

Something I haven’t mentioned earlier is that

• My dev env is Mac using VS Studio
• The media pushing(via Umbraco Deploy) works well on my local dev env because I’ve configured it to use my local drive when it runs on local
• When it’s deployed to TEST/STG or even LIVE, it uses AWS S3.
• We host our Umbraco dokerized (with linux image) and use AWS ECS Fargate to host it.
• All media and everything works well with the S3 Storage provider. The issue only happens when it comes to pushing media using Umbraco Deploy

I also use profile on my Mac for connection to the AWS account, but slightly differently to yours. I don’t put that in the appSettings or anywhere in the code but keep the connection profile in my Mac’s home folder’s .aws folder.

When it’s deployed to the server, because the S3 and the ECS(where it’s hosted) are on the same AWS account, it just works without the profile being available.

Hope it helps ironing out your issues for recreating the environment.

[AndyButland]

I'm afraid what you've tried to send above hasn't come through correctly @johnchoicl (see above).

Would be interested to hear your findings from part 2 of my comment above if you have chance too please.

[johnchoicl]

Hi Andy (@AndyButland), I simply replied the email wishing it would render fine here, my bad. I corrected them above. The image links should work now.

I haven't tried part 2 of your comment above but is it something to test the media file upload itself? If so, that's for sure to work. We've been using Umbraco in production for years now and AWS S3 has always been the media storage and that's using the StorageProviders.AWSS3 as the middleware. Therefore no doubt the code would work. Sorry, as that test itself requires a deployment to one of our busy servers(QA's going on and etc), so I wanted to make sure the purpose of the test you've requested before investing the time/efforts and avoid it if I can.

To me, the error you are having looks like the AWS configurations. I would want to help out as much as possible configuring the AWS environment for you to reproduce the bug I am experiencing but I haven't configured our hosting environment myself, it was done by AWS consultancy company who helped with all DevOps, I may not be the best person to help out unfortunately. I wished you have someone at Umbraco HQ who would have tested Umbraco Deploy on various environments including AWS S3 as the media storage.

Please let me know what I can further assist to get the Umbraco Deploy works for the company I work for. Let me know if verifying our Umbraco Deploy support license would speed up the support process as we are keen to use this for our production environment ASAP.

Regards, John Choi

[AndyButland]

Thanks @johnchoicl - I have managed now to get the S3 package working locally so media can now be created and saved to S3. So that's good, and allowed me to run my test code above, which I can confirm doesn't work and reported the same error - "Invalid path" - you are seeing with Deploy.

What this code is showing is that when you create media within a scope using a scoped file system, we get this error when using S3 - but it works as expected with media on the local disk, or when using Azure blob storage. The crux of this "scoped file system" is to provide transaction support - media files are first created in a temporary location, and when the scope completes, the transaction is committed and the files are copied to the right place. Deploy uses this transaction feature.

So now we need to figure out if this is caused by something the S3 package isn't handling correctly, or something in the CMS and it's support for scoped file systems. I'll see what more I can find out.

[AndyButland]

I've created a PR for the S3 package you are using, which should fix up this issue. As such I'll close this now, as it appears that the resolution needs to be in the S3 package. It isn't currently fully supporting all the CMS operations, including one that Deploy makes use of and exposes the problem.

[johnchoicl]

Thanks @AndyButland . Now the PR is merged and a new version released by the author through Nuget. Version number is 1.2.1 and it works perfectly for me. This is awesome and thanks again!

[AndyButland]

Great, glad to hear, and thanks for letting me know,