search

spring-projects / spring-integration-extensions

The Spring Integration Extensions project provides extension components for Spring Integration
http://www.springintegration.org/
284 stars 266 forks source link

SMB: FileTransferringMessageHandler not creating folders #250

Open icstreispe opened 2 years ago

icstreispe commented 2 years ago

I've created a class like this:

public class SmbMessageHandler extends FileTransferringMessageHandler<SmbFile>{
  public SmbMessageHandler (SmbRemoteFileTemplate s, FileExistsMode m){
    super (s, m);
  }
  public SmbMessageHandler (SessionFactory<SmbFile> f){
    this(new SmbRemoteFileTemplate(f));
  }
  public SmbMessageHandler (SmbRemoteFileTemplate s{
    super (s);
  }
  public boolean isChmodCapable(){ return false; }
}

Then used it in a spring integration flow like this:

@Bean("smbMessageHandler")
public MessageHandle  smbMessageHandler(FileNameGenerator g){
  SmbMessageHandler h = new SmbMessageHandler(smbRemoteFileTemplate, REPLACE);
  h.setAutoCreateDirectory(true);
  h.setRemoteDirectoryExpression("dir1/dir2/dir3/dir4/");
  h.setFileNameGenerator (g);
  return h;
}

And when I run the flow everything is fine as long as at least the folders dir1, dir2, dir3 are already created, it managed to create dir4 if is not available. But if dir3 (or below, like dir1 or dir2) are not already created, the component fails with:

jcifs.smb.SmbException: The network name cannot be found

What can I do in order to make it create all the necessary folders, no matter how many?

artembilan commented 2 years ago

Show, please, the whole stack trace for such an error. The RemoteFileUtils.makeDirectories() has a logic like this:

for (String pathToCreate : pathsToCreate) {
    if (logger.isDebugEnabled()) {
        logger.debug("Creating '" + pathToCreate + "'");
    }
    session.mkdir(pathToCreate);
}

So, if you add a DEBUG logging level for the org.springframework.integration.smb.session category, you would see what is going to be created against remote session.

Tell us also, please, how it works if you don't have that / in the end of your path.

icstreispe commented 2 years ago

full path from log: int2/PRJ/data/20220303/provider/IN/ int2/PRJ/data/ exists already 20220303/provider/IN/ - do not exist yet

  1. stack trace is below:
  2. the logs as you said (obfuscated host and user) are at the middle of this message
  3. removing / at the end has no effect, the log is exactly the same (at first I was trying without / and I thought it's my mistake)
  4. if delete all the folders in the share it manages to create the first folder (int2) and then dies with the same exception and the logs you can see at the end:

1

jcifs.smb.SmbException: The network name cannot be found. at jcifs.smb.SmbTransportImpl.checkStatus2(SmbTransportImpl.java:1467) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTransportImpl.checkStatus(SmbTransportImpl.java:1578) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:1027) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1549) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:409) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:347) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTreeImpl.treeConnect(SmbTreeImpl.java:611) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTreeConnection.connectTree(SmbTreeConnection.java:614) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:568) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTreeConnection.connectHost(SmbTreeConnection.java:489) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTreeConnection.connect(SmbTreeConnection.java:465) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbTreeConnection.connectWrapException(SmbTreeConnection.java:426) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbFile.ensureTreeConnected(SmbFile.java:573) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbFile.exists(SmbFile.java:873) ~[jcifs-2.1.28.jar:na] at jcifs.smb.SmbFile.mkdirs(SmbFile.java:1702) ~[jcifs-2.1.28.jar:na] at org.springframework.integration.smb.session.SmbSession.mkdir(SmbSession.java:276) ~[spring-integration-smb-1.2.1.RELEASE.jar:na] at org.springframework.integration.file.remote.RemoteFileUtils.tryCreateRemoteDirectory(RemoteFileUtils.java:83) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileUtils.makeDirectories(RemoteFileUtils.java:73) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.sendFileToRemoteDirectory(RemoteFileTemplate.java:561) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.doSend(RemoteFileTemplate.java:353) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.lambda$send$0(RemoteFileTemplate.java:314) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:452) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.send(RemoteFileTemplate.java:314) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.send(RemoteFileTemplate.java:302) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.RemoteFileTemplate.send(RemoteFileTemplate.java:294) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.file.remote.handler.FileTransferringMessageHandler.handleMessageInternal(FileTransferringMessageHandler.java:207) ~[spring-integration-file-5.5.7.jar:5.5.7] at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:56) ~[spring-integration-core-5.5.7.jar:5.5.7] ....

2

22:20:54.086 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - New org.springframework.integration.smb.session.SmbSession created. 22:20:54.092 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/provider/IN/] for path [int2/PRJ/data/20220303/provider/IN/]. 22:20:54.120 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/provider/IN] for path [int2/PRJ/data/20220303/provider/IN]. 22:20:54.132 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/provider] for path [int2/PRJ/data/20220303/provider]. 22:20:54.142 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303] for path [int2/PRJ/data/20220303]. 22:20:54.152 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/] for path [int2/PRJ/data/]. 22:20:54.152 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/] for path [int2/PRJ/data]. 22:20:54.172 [http-nio-9990-exec-2] DEBUG o.s.i.s.s.SmbRemoteFileTemplate - Creating 'int2/PRJ/data/20220303' 22:20:54.182 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/] for path [int2/PRJ/data/20220303/]. 22:20:54.182 [http-nio-9990-exec-2] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/] for path [int2/PRJ/data/20220303].

... 4

22:51:15.756 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - New org.springframework.integration.smb.session.SmbSession created. 22:51:15.756 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/provider/IN/] for path [int2/PRJ/data/20220303/provider/IN/]. 22:51:15.839 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/provider/IN] for path [int2/PRJ/data/20220303/provider/IN]. 22:51:15.889 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303/provider] for path [int2/PRJ/data/20220303/provider]. 22:51:15.950 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data/20220303] for path [int2/PRJ/data/20220303]. 22:51:15.999 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/data] for path [int2/PRJ/data]. 22:51:16.009 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ] for path [int2/PRJ]. 22:51:16.059 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2] for path [int2]. 22:51:16.059 [http-nio-9990-exec-6] DEBUG o.s.i.s.s.SmbRemoteFileTemplate - Creating 'int2' 22:51:16.109 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/] for path [int2/]. 22:51:16.109 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/] for path [int2]. 22:51:16.219 [http-nio-9990-exec-6] INFO o.s.i.smb.session.SmbSession - Successfully created remote directory [int2] in share [smb://user:Password123%40@host:0/vfs_dev_00009]. 22:51:16.219 [http-nio-9990-exec-6] DEBUG o.s.i.s.s.SmbRemoteFileTemplate - Creating 'int2/PRJ' 22:51:16.229 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/] for path [int2/PRJ/]. 22:51:16.229 [http-nio-9990-exec-6] DEBUG o.s.i.smb.session.SmbSession - Created new jcifs.smb.SmbFile[smb://user:Password123%40@host:0/int2/PRJ/] for path [int2/PRJ]. 22:51:16.295 [http-nio-9990-exec-6] DEBUG o.s.web.servlet.DispatcherServlet - Failed to complete request: org.springframework.messaging.MessageDeliveryException:

artembilan commented 2 years ago

This is the part of JCIFS where that exception is thrown:

        default:
            if ( log.isDebugEnabled() ) {
                log.debug("Error code: 0x" + Hexdump.toHexString(resp.getErrorCode(), 8) + " for " + req.getClass().getSimpleName());
            }
            throw new SmbException(resp.getErrorCode(), null);
        }

Doesn't looks like there is something we can do with this problem in this project. I suggest you to raise the concern with JCIFS community or your SMB vendor.

Perhaps you got some limits for your requests?..

icstreispe commented 2 years ago

seems that I found out the problem in the end: smbRemoteFileTemplate should be initialized with a SmbSessionFactory object whose value of its shareAndDir property should contain an / at the end

artembilan commented 2 years ago

Hm. We have a logic in there in the SmbConfig like this:

        if (!path.startsWith("/")) {
            path = "/" + path;
        }

According to your findings it looks like we also have to add / into the end.

If that is what you are thinking, too, feel free to contribute the fix.

Thank you!

GregBragg commented 2 years ago

Hm. We have a logic in there in the SmbConfig like this:

      if (!path.startsWith("/")) {
          path = "/" + path;
      }

According to your findings it looks like we also have to add / into the end.

If that is what you are thinking, too, feel free to contribute the fix.

Thank you!

Not sure if this is a good idea to always force a forward slash at the end... I'd like to understand all the use cases before we allow this in the code.

As stated this is mitigated anyways by the configuration setting in the SmbSessionFactory initialization.

artembilan commented 2 years ago

Yes. We can live with that workaround and give it some flexibility when same configuration without "forward slash at the end" works in other environment.