apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.73k stars 4.58k forks source link

[Bug] [seatunnel] buildOptions with getResourceName may be error #16445

Open zhuang1125 opened 1 month ago

zhuang1125 commented 1 month ago

Search before asking

What happened

this code may be error when get ResourceName:

 protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                args.add(resourceInfo.getResourceName().replaceFirst(".*:", ""));
            });
        }
        return args;
    }

should be change to below to resolve:

 protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                // args.add(resourceInfo.getResourceName().substring(1));
                String path = resourceInfo.getResourceName();

                String absolutePath= taskExecutionContext.getResourceContext().getResourceItem(path)
                        .getResourceAbsolutePathInLocal();
                args.add(absolutePath);                
                // args.add("/" + resourceInfo.getResourceName().replaceFirst(".*:", ""));
            });
        }
        return args;
    }

What you expected to happen

this code may be error when get ResourceName:

 protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                args.add(resourceInfo.getResourceName().replaceFirst(".*:", ""));
            });
        }
        return args;
    }

should be change to below to resolve:

 protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                // args.add(resourceInfo.getResourceName().substring(1));
                String path = resourceInfo.getResourceName();

                String absolutePath= taskExecutionContext.getResourceContext().getResourceItem(path)
                        .getResourceAbsolutePathInLocal();
                args.add(absolutePath);                
                // args.add("/" + resourceInfo.getResourceName().replaceFirst(".*:", ""));
            });
        }
        return args;
    }

How to reproduce

this code may be error when get ResourceName:

 protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                args.add(resourceInfo.getResourceName().replaceFirst(".*:", ""));
            });
        }
        return args;
    }

should be change to below to resolve:

 protected List<String> buildOptions() throws Exception {
        List<String> args = new ArrayList<>();
        if (BooleanUtils.isTrue(seatunnelParameters.getUseCustom())) {
            args.add(CONFIG_OPTIONS);
            args.add(buildCustomConfigCommand());
        } else {
            seatunnelParameters.getResourceList().forEach(resourceInfo -> {
                args.add(CONFIG_OPTIONS);
                // TODO: Need further check for refactored resource center
                // TODO Currently resourceName is `/xxx.sh`, it has more `/` and needs to be optimized
                // args.add(resourceInfo.getResourceName().substring(1));
                String path = resourceInfo.getResourceName();

                String absolutePath= taskExecutionContext.getResourceContext().getResourceItem(path)
                        .getResourceAbsolutePathInLocal();
                args.add(absolutePath);                
                // args.add("/" + resourceInfo.getResourceName().replaceFirst(".*:", ""));
            });
        }
        return args;
    }

Anything else

my env is docker with minio as resourcemanager to get resource

Version

3.2.x

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.