apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.88k stars 4.63k forks source link

[Bug] [dolphinscheduler-storage-hdfs] When you create a new task, the resource file cannot be displayed hierarchically #16331

Closed JohnZp closed 3 days ago

JohnZp commented 4 months ago

Search before asking

What happened

image When creating a new task, the resource file cannot be displayed in a hierarchical manner, making it difficult to distinguish which file it is

What you expected to happen

image Ability to display files hierarchically

How to reproduce

IN HdfsStorageOperator,

    public List<StorageEntity> listFileStorageEntityRecursively(String resourceAbsolutePath) {
        exceptionIfPathEmpty(resourceAbsolutePath);

        List<StorageEntity> result = new ArrayList<>();

        LinkedList<String> foldersToFetch = new LinkedList<>();
        foldersToFetch.addLast(resourceAbsolutePath);

        while (!foldersToFetch.isEmpty()) {
            String absolutePath = foldersToFetch.pollFirst();
            RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(new Path(absolutePath), true);
            while (remoteIterator.hasNext()) {
                LocatedFileStatus locatedFileStatus = remoteIterator.next();
                result.add(transformFileStatusToResourceMetadata(locatedFileStatus));
            }
        }
        return result;
    }

Anything else

No response

Version

dev

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 2 months ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.