Azure Datalake Gen2 Rename File Issue

[marilyndd]

Describe the bug:

Utilizing the DataLakeServiceClient, I am trying to rename files after they have been unzip in Azure Synapse notebooks, but I have been successful up until the point of renaming (code is failing at the last line for rename_file method).

I receive the following error:

HttpResponseError: (OutOfRangeInput) The specified resource name length is not within the permissible limits. RequestId:6ef9e79f-a01f-0067-5cb0-62b105000000 Time:2023-03-30T02:34:49.8798540Z Code: OutOfRangeInput Message: The specified resource name length is not within the permissible limits. RequestId:6ef9e79f-a01f-0067-5cb0-62b105000000 Time:2023-03-30T02:34:49.8798540Z

Code Snippet:

    extraction_path = f"/synfs/{mssparkutils.env.getJobId()}/sink/{self.sink_path}"
    with zipfile.ZipFile(compressed_data) as zip_file:
        zip_file.extractall(extraction_path)

    dl_service_client_source = DataLakeServiceClient(f"https://{lake}.dfs.core.windows.net", credential=credential)
    dl_file_system_client_source = dl_service_client_source.get_file_system_client("refined")
    dl_directory_client_source = dl_file_system_client_source.get_file_client(self.solution_folder) 

    files1 = (self.solution_folder + '/' + self.sink_path)

    files = dl_file_system_client_source.get_paths(path=files1)

    for file in files:
        if not file.is_directory:
            file_client = dl_file_system_client_source.get_file_client(file.name)
            dir_path, file_name1 = os.path.split(file.name)
            file_name, file_extension = os.path.splitext(file.name)
            new_name = 'test'
            new_file_path = os.path.join(dir_path, new_name)
            file_client.rename_file(new_file_path)

All of my values are printing out correctly, but I am still receiving the length error.

Output of the values:

_filename: IT/SpatioTemporalAssetCatalog/0_unzipped/1511111156_DodgeDeploy/Manufacturing/ORTHOMOSAIC/TKC/gmwc/TKC_Orthomosaic_export_TueAug31183858312496 files.name: IT/SpatioTemporalAssetCatalog/0_unzipped/1511111156_DodgeDeploy/Manufacturing/ORTHOMOSAIC/TKC/gmwc/TKC_Orthomosaic_export_TueAug31183858312496.kml _newname: test _new_filepath: IT/SpatioTemporalAssetCatalog/0_unzipped/1511111156_DodgeDeploy/Manufacturing/ORTHOMOSAIC/TKC/gmwc/test

[pvaneck]

Thanks for the feedback. I'll tag some folks who can perhaps provide some insight.

[jalauzon-msft]

Hi @marilyndd, it seems you are trying to work with a resource where the file path is above the maximum limit imposed by the Storage service. The service limits paths to 1024 characters (including /s). See documentation here.

Are the paths you shared in your post the full file paths that you are using? If so, they do seem to be below the limit so perhaps there is something else going on.

[github-actions[bot]]

Hi @marilyndd. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[marilyndd]

Hi there, I posted this on stack overflow and one of the Microsoft support people replied stating it was my size of my container IT producing the error.

[jalauzon-msft]

Hi @marilyndd, is IT actually your "container" name or is it just the first part of your path? Your container name in this case would actually be your Filesystem name (i.e. whatever you pass to get_file_system_client). From your samples it looks like IT is just part of your file path, in which case that should be ok I think, especially if the source file already exists (meaning it was able to be created).

StackOverflow for reference.

[github-actions[bot]]

Hi @marilyndd. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[github-actions[bot]]

Hi @marilyndd, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!