umbraco / Umbraco.Deploy.Issues

1 stars 0 forks source link

PathTooLongException when transfer content to remote environment #200

Closed bjarnef closed 7 months ago

bjarnef commented 7 months ago

While migrating a project from Umbraco v8 -> v10 > v12 -> v13 I have a some dependency errors.

I had at look at pickers on thee documents, but I couldn't easily from what caused the error. Could it e.g. log the specific property?

image

For now I have ignored dependencies: https://docs.umbraco.com/umbraco-deploy/getting-started/deploy-settings#ignorebrokendependencies

"Deploy": {
  "Settings": {
    "IgnoreBrokenDependencies": true,
    "IgnoreBrokenDependenciesBehavior": "All",
    "PreferLocalDbConnectionString": true,
    "DatabaseCommandTimeout": "0.0:30:00",
    "HttpClientTimeout": "0.0:30:00",
    "SessionTimeout": "0.0:30:00",
    "SourceDeployTimeout": "0.0:30:00"
  }
}

A new transfer causes PathTooLongException instead.

image

Would it be possible to ignore those files or another workaround for this?

Umbraco v13.0.3 Deploy v13.0.3


This item has been added to our backlog AB#36935

AndyButland commented 7 months ago

Yes, there's a setting you can use to ignore such files.

You may also be able to carry out the steps described on this page to enable long path support on Windows.

bjarnef commented 7 months ago

@AndyButland I have this configuration of Deploy now, but the transferfrom local to development still fails with "The source environment has thrown a System.IO.PathTooLongException with message".

"Deploy": {
  "Settings": {
    "ContinueOnMediaFilePathTooLongException": true,
    "IgnoreBrokenDependencies": true,
    "IgnoreBrokenDependenciesBehavior": "All",
    "PreferLocalDbConnectionString": true,
    "DatabaseCommandTimeout": "0.0:30:00",
    "HttpClientTimeout": "0.0:30:00",
    "SessionTimeout": "0.0:30:00",
    "SourceDeployTimeout": "0.0:30:00"
  }
} 
AndyButland commented 7 months ago

Hmm... maybe this exception is being triggered in more than one place. Could you share the full stack trace please?

bjarnef commented 7 months ago

This is what I can find in log:

System.IO.PathTooLongException: Path C:\VSProjects\my-project\UmbracoCloud\src\UmbracoProject\wwwroot\media\wybhiygh\riga-er-trods-sine-800-år-en-ung-by-her-bor-mange-studerende-unge-turister-fra-ikke-mindst-norden-og-tyskland-rejser-hertil-og-nyder-godt-af-prisvenligheden-foto-askoldsb-dreamstime-com.jpg is too long.
   at Umbraco.Cms.Core.IO.PhysicalFileSystem.GetFullPath(String path)
   at Umbraco.Deploy.Infrastructure.Connectors.ValueConnectors.ImageCropperValueConnector.ToArtifact(Object value, IPropertyType propertyType, ICollection`1 dependencies, IContextCache contextCache)
   at Umbraco.Deploy.Infrastructure.Connectors.ServiceConnectors.ContentConnector.ContentConnectorBase`2.AddProperty(IProperty property, ICollection`1 dependencies, Boolean published, IDictionary`2 propertyCollection, IContextCache contextCache)
   at Umbraco.Deploy.Infrastructure.Connectors.ServiceConnectors.ContentConnector.ContentConnectorBase`2.MapArtifactProperties(IPropertyCollection properties, ICollection`1 dependencies, Boolean published, IContextCache contextCache)
   at Umbraco.Deploy.Infrastructure.Connectors.ServiceConnectors.ContentConnector.MediaConnector.GetArtifact(GuidUdi udi, IContextCache contextCache)
   at Umbraco.Deploy.Infrastructure.Connectors.ServiceConnectors.ServiceConnectorBase`3.Umbraco.Cms.Core.Deploy.IServiceConnector.GetArtifact(Udi udi, IContextCache contextCache)
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.GetDependencyArtifact(ArtifactDependency dep, DependentArtifactState& dependentArtifactState, String& dependencyName, String& dependencyErrorMessage, IContextCache contextCache)
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.AddEntries(Manifest manifest, List`1 sigs, IEnumerable`1 items, HashSet`1 exclude, IWorkItem workItem, CancellationToken token)
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.UpdateManifest(Guid sessionId, Manifest manifest, IEnumerable`1 exclude)
   at Umbraco.Deploy.Infrastructure.Environments.CurrentEnvironment.UpdateManifestAsync(Guid sessionId, Manifest manifest, IEnumerable`1 exclude, CancellationToken token)
   at Umbraco.Deploy.Infrastructure.Work.WorkItems.DeployRestoreWorkItemBase.NegotiateManifest(Manifest manifest, IEnumerable`1 exclude, Nullable`1 batchIndex, Nullable`1 numberOfBatches)
   at Umbraco.Deploy.Infrastructure.Work.WorkItems.SourceDeployWorkItem.ProceedToPackage(IEnumerable`1 items, Nullable`1 batchIndex, Nullable`1 numberOfBatches)
   at Umbraco.Deploy.Infrastructure.Work.WorkItems.SourceDeployWorkItem.ExecuteAsyncSub(IEnumerable`1 items, Nullable`1 batchIndex, Nullable`1 numberOfBatches, CancellationToken token)
bjarnef commented 7 months ago

@AndyButland do you know what the maximum length or path allowed is on Umbraco Cloud?

Then I can maybe find media items with a long path using SQL.. shorter these and the physical file as well. Or maybe a SQL to list media with longest path first.

AndyButland commented 7 months ago

I misread earlier that you were pushing to Cloud rather than pulling from Cloud. Of course in the push to Cloud situation you can't amend the "long path" registry settings in Windows.

For media, it shouldn't be a problem on Cloud, as it uses blob storage which has a much higher limit.. But I guess the image cropper is also using the local disk for some part of it's save operation.

If that's right, then I expect the limit is 256 characters, which is what I read is the default limit on Windows.

Your example is 270.

bjarnef commented 7 months ago

Hi @AndyButland

Okay, the currently database and media is on Azure and in Blob Storage, so I would have assumed it was just same limit since they database was a database backup restored locally from Azure.

image

I guess we could use a SQL script to update and replace the existing path.

Maybe there is a nice way to list longest path first to get an overview.

AndyButland commented 7 months ago

We've updated such that the ContinueOnMediaFilePathTooLongException setting will be respected in the image cropper value connector now too. This will be part of the next round of patch releases.

AndyButland commented 7 months ago

Maybe there is a nice way to list longest path first to get an overview.

This should do it:

SELECT * FROM umbracoPropertyData
ORDER BY LEN(Convert(nvarchar(max), textValue)) DESC
bjarnef commented 7 months ago

Maybe there is a nice way to list longest path first to get an overview.

This should do it:

SELECT * FROM umbracoPropertyData
ORDER BY LEN(Convert(nvarchar(max), textValue)) DESC

It was quite slow ~3 minutes to execute and sort of course Block Grid and Block List content first.

This was much faster :)

SELECT * FROM [BLL-Umbraco-prod].[dbo].[umbracoPropertyData]
WHERE propertyTypeId IN(6, 24) AND textValue LIKE '%/media/%'
ORDER BY LEN(Convert(nvarchar(max), textValue)) DESC 

But not really longest path first, but entire value on property.

image