deepjavalibrary / djl

An Engine-Agnostic Deep Learning Framework in Java
https://djl.ai
Apache License 2.0
4.12k stars 654 forks source link

Error in image folder loading #1147

Closed zluo01 closed 3 years ago

zluo01 commented 3 years ago

in the source code,

 private void loadSynset() {
        File root = new File(mrl.getRepository().getBaseUri());
        File[] dir = root.listFiles(f -> f.isDirectory() && !f.getName().startsWith("."));
        if (dir == null || dir.length == 0) {
            throw new IllegalArgumentException(root + " not found or didn't have any file in it");
        }
        Arrays.sort(dir);
        for (File file : dir) {
            synset.add(file.getName());
        }
    }

Should the condition inside listFiels be !f.isDirectory() && !f.getName().startsWith(".") since we want not directory or hidden files.

Thanks.

frankfliu commented 3 years ago

@zluo01 We use the folder name as class name (synset), we are looking for folders only. Let's say we have the following folder structure:

root |

The loadSyncet() function will create a list: [cat, dog, goat, sheep]

zluo01 commented 3 years ago

Hi, I wonder how does DJL detect GPU or what are the requirements ? I am on Fedora 34 with a 3080 with following driver and cuda version. But I am able to use CPU when I train the model.

NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4  

Thanks.

frankfliu commented 3 years ago

@zluo01 We don't support cuda 11.4, none of the engine released official support for cuda 11.4

In DJL, we load libcudart.so and use this library to detect the version of CUDA and compute compatible version.

Then we search for engines that support the CUDA + sm version. If not found we fallback to CPU:

Currently pytorch support: cu102, cu111 MXNet support: cu102, cu110 TensorFlow support: cu102

See: http://docs.djl.ai/docs/development/dependency_management.html