Issue with Creating Nested Folders and Files Using SMBLibrary in AWS Lambda

I'm utilizing SMBLibrary within AWS Lambda to read S3 objects and transfer them to FSX. I've diligently followed all the best practices for using SMBLibrary, but I'm encountering difficulties when it comes to creating folders. While I can successfully create a folder, the subsequent subfolder isn't being created underneath it, and the same issue occurs with files.

Here is my current code: ` ISMBFileStore fileStore = client.TreeConnect("share", out status);

if (status == NTStatus.STATUS_SUCCESS) //Connection to share successful { string folder = "/TestBucket/First; string file = "sample.txt";

  // Split the path into individual components
              string[] folderComponents = folder.Split('/'); // Assuming a Unix-style path separator here

              // Initialize a variable to keep track of the current folder path
              string currentFolder = "";

              // Loop through each component and process it to create folder.
              for (int i = 0; i < folderComponents.Length; i++)
              {
                  currentFolder = folderComponents[i];

                  //create folder
                  status = fileStore.CreateFile(out fileHandle, out fileStatus, currentFolder, AccessMask.GENERIC_READ | AccessMask.SYNCHRONIZE, SMBLibrary.FileAttributes.Directory, ShareAccess.Write | ShareAccess.Read, CreateDisposition.FILE_OPEN_IF, CreateOptions.FILE_DIRECTORY_FILE | CreateOptions.FILE_SYNCHRONOUS_IO_ALERT, null);

                  //status
                  if (status == NTStatus.STATUS_SUCCESS)
                  {
                      context.Logger.LogInformation($"Created folder: {currentFolder}");

                      status = fileStore.QueryDirectory(out List<QueryDirectoryFileInformation> files, fileHandle, "*", FileInformationClass.FileDirectoryInformation);

                  }
              }

       //create file
        status = fileStore.CreateFile(out fileHandle, out fileStatus, file, AccessMask.GENERIC_WRITE | AccessMask.SYNCHRONIZE, SMBLibrary.FileAttributes.Normal, ShareAccess.None, CreateDisposition.FILE_OPEN_IF, CreateOptions.FILE_NON_DIRECTORY_FILE | CreateOptions.FILE_SYNCHRONOUS_IO_ALERT, null);

}

The above code creates folder "TestBucket", "First" and file "Sample.txt" under Share. While I'm looking it as "Share\TestBucket\First\Sample.txt"

Appreciate assistance in resolving this issue.

Unable to provide detailed reply as I'm on vacation, but I'm sure you'll figure it out. You'll need to recursively create the subdirectories. You'll need to provide the full path relative to the share when creating a directory or file.

On Tuesday, September 19, 2023, spp789 @.***> wrote:

I'm utilizing SMBLibrary within AWS Lambda to read S3 objects and transfer them to FSX. I've diligently followed all the best practices for using SMBLibrary, but I'm encountering difficulties when it comes to creating folders. While I can successfully create a folder, the subsequent subfolder isn't being created underneath it, and the same issue occurs with files.

Here is my current code: ` ISMBFileStore fileStore = client.TreeConnect("share", out status);

if (status == NTStatus.STATUS_SUCCESS) //Connection to share successful { string folder = "/TestBucket/First; string file = "sample.txt";

// Split the path into individual components string[] folderComponents = folder.Split('/'); // Assuming a Unix-style path separator here
          // Initialize a variable to keep track of the current folder path
          string currentFolder = "";

          // Loop through each component and process it to create folder.
          for (int i = 0; i < folderComponents.Length; i++)
          {
              currentFolder = folderComponents[i];

              //create folder
              status = fileStore.CreateFile(out fileHandle, out fileStatus, currentFolder, AccessMask.GENERIC_READ | AccessMask.SYNCHRONIZE, SMBLibrary.FileAttributes.Directory, ShareAccess.Write | ShareAccess.Read, CreateDisposition.FILE_OPEN_IF, CreateOptions.FILE_DIRECTORY_FILE | CreateOptions.FILE_SYNCHRONOUS_IO_ALERT, null);

              //status
              if (status == NTStatus.STATUS_SUCCESS)
              {
                  context.Logger.LogInformation($"Created folder: {currentFolder}");

                  status = fileStore.QueryDirectory(out List<QueryDirectoryFileInformation> files, fileHandle, "*", FileInformationClass.FileDirectoryInformation);

              }
          }

   //create file
    status = fileStore.CreateFile(out fileHandle, out fileStatus, file, AccessMask.GENERIC_WRITE | AccessMask.SYNCHRONIZE, SMBLibrary.FileAttributes.Normal, ShareAccess.None, CreateDisposition.FILE_OPEN_IF, CreateOptions.FILE_NON_DIRECTORY_FILE | CreateOptions.FILE_SYNCHRONOUS_IO_ALERT, null);
}

`

The above code creates folder "TestBucket", "First" and file "Sample.txt" under Share. While I'm looking it as "Share\TestBucket\First\Sample.txt"

Appreciate assistance in resolving this issue.

— Reply to this email directly, view it on GitHub https://github.com/TalAloni/SMBLibrary/issues/221, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHPWEXYI5LG4KC6BS6R54LX3HQD7ANCNFSM6AAAAAA46XJCY4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Thanks @TalAloni for your response.

After conducting additional testing, I successfully resolved the issue. Here are the pertinent details: The problem occurred when attempting to create nested folders. For the initial folder, it's necessary to provide only the name without a "\". For example, if the desired path is "a\b\c," you should first pass "a," followed by "a\b" for the next folder creation, and finally "a\b\c" for the complete path.

The steps to create nested folders and file and write to the file are as below.

// I followed below steps using the SMBclient to create nested folders and file. Step 1 - Folder Creation:

For a single folder, it is created as "a".
For nested folders like "a\b\c", they are sequentially created as "a", "a\b", "a\b\c". Step 2 - File Creation:
Once the folder structure is established, files are created.
If a file is under a nested folder, its path will be like "a\b\c\Sample.txt". Step 3 - Writing to File:
After creating a file and obtaining a handle, data is written to the file using a stream.

I'm using following values in my example.

folderPath = a\b\c file = Sample.txt fullpath = a\b\c\Sample.txt

Step 1: The following code will create nested folders.

        // Split the path into individual components
        string[] folders = folderPath.Split('/'); // Assuming a Unix-style path separator here

        // The following loop generates folders and files.
        foreach (string folder in folders)
        {
            //validate folder has value
            if (folder != null)
            {
                currentFolder = currentFolder + folder.Trim();

                //create folder
                status = fileStore.CreateFile(out fileHandle, out fileStatus, currentFolder,
                            AccessMask.SYNCHRONIZE | (AccessMask)DirectoryAccessMask.GENERIC_READ | (AccessMask)DirectoryAccessMask.DELETE, 0,
                            ShareAccess.Read | ShareAccess.Delete, CreateDisposition.FILE_OPEN | CreateDisposition.FILE_CREATE,
                            CreateOptions.FILE_SYNCHRONOUS_IO_NONALERT | CreateOptions.FILE_DIRECTORY_FILE, null);

                //append "\" to the folder for next folder to get created.
                currentFolder = currentFolder + @"\";
            }
        }

Step 2 - File Creation

   // if there is a folderstructure available, append it with file so, it can create under nested folder.
    if (currentFolder != "")
    {
        file = currentFolder + file;
    }

    //create file
    status = fileStore.CreateFile(out fileHandle, out fileStatus, file,
        AccessMask.GENERIC_WRITE | AccessMask.SYNCHRONIZE, SMBLibrary.FileAttributes.Normal, ShareAccess.None,
        CreateDisposition.FILE_OPEN_IF, CreateOptions.FILE_NON_DIRECTORY_FILE | CreateOptions.FILE_SYNCHRONOUS_IO_ALERT, null);

Step 3 - Writing to File

    //writing file
    if (status == NTStatus.STATUS_SUCCESS)
    {
        // Write the data from the S3 response stream to the SMB file store.
        byte[] buffer = new byte[1024];
        int bytesRead;
        int writeOffset = 0;

        //reading the object though s3objectasync, if your usecase if to read from different source you can update below line to read through different stream
        while ((bytesRead = s3Response.ResponseStream.Read(buffer, 0, buffer.Length)) > 0)
        {
            // Create a new buffer if the current buffer is too small.
            if (bytesRead > buffer.Length)
            {
                buffer = new byte[bytesRead];
            }

            //write file
            status = fileStore.WriteFile(out bytesRead, fileHandle, writeOffset, buffer);

            //failed to write to file and throw exception.
            if (status != NTStatus.STATUS_SUCCESS)
            {
                throw new Exception($"Failed to write to file {file}.");
            }
            writeOffset += bytesRead;
        }

        // Close the file.
        status = fileStore.CloseFile(fileHandle);
    }
    else  //failed to create to file and throw exception.
    {
        throw new Exception($"Failed to create to file {file}.");
    }

TalAloni / SMBLibrary

Issue with Creating Nested Folders and Files Using SMBLibrary in AWS Lambda #221