Bug in image loading

[LethalSnake1337]

I run into a weird problem that. If my images are lets say on my X SSD and DeepLabel on my Y SSD it will throw an error that images couldn't be loaded. If its on the same Hard drive it works just fine.

[jveitchmichaelis]

Hi, yes due to the way the database stores relative image paths, files can't be stored on different drives. In an upcoming version of the app this will (probably) be fixed but currently you need to store the label database on the same drive letter.

To give some more detail about why this is non-trivial, and why Deeplabel works this way: this is a design decision as it allows users to easily transfer label projects between machines, since only the relative path from the database is stored. If you stored the absolute path to your data, then it wouldn't work on another machine without having to specify the updated paths (which gets complicated if your data are spread out over e.g. multiple drives or root folders). The solution is probably to allow users to specify if they want absolute or relative paths in a project, with the understanding that using absolute paths reduces portability.

If you're using Linux, an option might be to symlink your external drive to a folder on the drive you want your labels to be stored in. Then you provide a path to the symlink, and deeplabel should treat it like a local path. You can do the same on Windows and it should work.

[LethalSnake1337]

Hey thanks for the fast answere that totaly makes sense.

I am encountering a new problem related to export. I tried to export my images to darknet but I keep getting the error 1.Export output folder doesn't exist 2.Couldn't find this label in the names file: "head\r" my labels in deeplabel are Head, enemies and the same in my obj.names. Then I tried to export to another module all exports doesnt work. Am I missing any steps here?

[jveitchmichaelis]

Heya, I pushed a couple of fixes to export code recently (there was a path bug). Although darknet should be OK. Odd that your label has a carriage return at the end though, that probably shouldn't happen.

The export folder should be created as part of the process, so you can probably ignore that. Otherwise make sure the labels in your project exactly match the labels in your names file (e.g. the case is the same).

Are you building deeplabel from source, or are you using a released version?

[LethalSnake1337]

Well I build it for my Linux system from source. The labeling works just fine but the export doesn't work for some reason. The Labels are matching Its just weird that its head\r even though in deeplabel I don't have any class of this name. The two class names in Deeplabel are Head and enemies same as in my obj.name. Is it just fine having this two names in it? for example: Head enemies

or do I need this structure { item { name: Head id: 1 displayname: Head }

I also tried the Windows version the problem there is that the automatic annotation isn't working. Its processing it correctly but I don't see any annotation on the image in deeplabel.

[jveitchmichaelis]

The names file should be a list of newline-separated classes, e.g.

head
enemies

Does the export folder get created? (ignore the warning for now, that might just be a message telling you that deeplabel is going to run mkdir)

[LethalSnake1337]

No it will instantly put it into the directory for example I select Desktop and it will create two folders named train and valid onto the Desktop. I don't know if u mean by export folder a folder which has this structure export->train and valid. If its that structure then no it isn't creating any export folder. Is there any difference if the first letter in my obj.names is an capital letter e.g Head

[LethalSnake1337]

Its creating the valid and train folder that works just fine and its exporting the enemy label and all the images just not the Head.

[jveitchmichaelis]

Ok I'm just looking up how deeplabel handles classnames. Probably a bug there somewhere. e.g. the code that loads the name file is:

void DarknetExporter::generateLabelIds(const QString names_file){
    id_map.clear();

    // Force sort list
    QStringList class_list;
    QFile fh(names_file);

    if (fh.open(QIODevice::ReadOnly)) {

        while (!fh.atEnd()) {
            // Darknet name file is just a newline delimited list of classes
            QByteArray line = fh.readLine();
            class_list.append(line);
        }
    }

    if(class_list.size() == 0){
        qWarning() << "No classes found in names file.";
        return;
    }

    int i = 0;
    for(auto &name : class_list){
        auto cleaned_name = name.simplified().toLower();
        id_map[cleaned_name] = i++;
        qDebug() << "Adding: " << cleaned_name << " (" << i << ")";
    }
}

so each line in the names file should be stripped and lowercased. Maybe simplified isn't stripping the carriage return. In theory the class list shouldn't be case-sensitive.

Edit: should be OK - https://doc.qt.io/qt-5/qstring.html#simplified. It might be the class name in your project then. You can try fixing this manually using e..g DB Browser to modify the class name (maybe it got a rogue \r somewhere). Look in the class table and update the entry.

[LethalSnake1337]

Hey thank I'll edit the db thanks for the tipp :) I'll let you know if I run into any problems thanks again

[jveitchmichaelis]

I just pushed a fix that should clean classnames being entered into the database. That should fix your problem, but as I mentioned you may need to fixup the class name in your project manually, sorry about that - there's currently no way to do that in deeplabel.

Edit: Actually can you try this: pull and rebuild deeplabel, create a new (empty) project and then merge your existing one into it? (File->merge) That should fix the labels when entering into the new project.

[LethalSnake1337]

Already fixed it with DB Browser there was indeed a carriage return in the DB I dont know how that even happend but will pull it. I do have a question enemy is class 1 now and Head class 2 which isn't want I wanted it should be the other way. Can I just swap the class name in the classes table and swap the class_id in the labels table? Would that fix it? I am not sure if there will be any inconsistency if changing those two columns.

[jveitchmichaelis]

The class ID in deeplabel is arbitrary - when you output things, the ID is derived from the names file. This way if you have two projects where e.g. in (1) you added an enemy class and then head, and then in (2) you added head then enemy the output will be consistent. TL;DR don't worry about it, the order in the names file is the important thing.

To be specific, class ID in the project file is just a SQL auto-increment key. It doesn't reflect the class ID in your model.

If it doesn't work that way let me know, definitely a bug ;)

You mentioned you also had an issue with auto labelling - do you mean from a pre-trained model? I can also take a look at that.

[LethalSnake1337]

You are right just the names file is important just checked working.

About the auto labelling thats just on Windows. The Linux version works perfectly now :). Yes its on the pre-trained model. It takes the input fine and the processing output looks also good seems like he is realy working and labelling but I dont see any labelling after he is done on the picture.

[jveitchmichaelis]

OK I'll take a look on my machine and see if something got broken - possible this is a related issue.

On Wed, Sep 22, 2021 at 1:15 PM LethalSnake1337 @.***> wrote:

You are right just the names file is important just checked working.

About the auto labelling thats just on Windows. The Linux version works perfectly now :). Yes its on the pre-trained model. It takes the input fine and the processing output looks also good seems like he is realy working and labelling but I dont see any labelling after he is done on the picture.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jveitchmichaelis/deeplabel/issues/25#issuecomment-924920383, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYDMJ47VUUMULBTMOU4SR3UDHJIZANCNFSM5EOSJEZQ .

[jveitchmichaelis]

Hmm I can't replicate this. I've tested using a bunch of images from the MS-COO val2017 dataset and then running yolov4-tiny:

and I get something like:

maybe I can add some more debug output/logging to say if the detector is actually finding objects.

[LethalSnake1337]

Thats weird for me it isn't working. If you dont mind adding some debug output that would be awesome. I dont know why it isnt showing anything. I might redownload the release version

[jveitchmichaelis]

The release is pretty out of date, I'll try and push a new version shortly...

[LethalSnake1337]

That would be awesome. Thanks :)

[LethalSnake1337]

Hey buddy I am facing a new problem. I am done with annotating 50k images. A lot of them are negative samples. But only the labeled images are exported. I also tried checking the "Export unlabelled images" checkbox but nothing changed. Could you help me out

[LethalSnake1337]

I just realized you pushed a fix working now fetched the new repo thx :)

[jveitchmichaelis]

Great, so to confirm the latest push worked?

On Wed, Sep 29, 2021, 06:45 LethalSnake1337 @.***> wrote:

I just realized you pushed a fix working now fetched the new repo thx :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jveitchmichaelis/deeplabel/issues/25#issuecomment-929484362, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYDMJ6S5MX4G3ZCHNKULTTUEH5LFANCNFSM5EOSJEZQ .

[lukaswozniak]

Hi, Im having same problem. I try to check your latest commit, but I see that all recent windows builds are failed on github actions. Do you plan to fix the github actions soon or upload new build on github releases?

[jveitchmichaelis]

@lukaswozniak try this one: https://github.com/jveitchmichaelis/deeplabel/releases/tag/0.16.1

Building (for windows) in Github's CI is a pain and I'm still trying to get it working. It seems to be a pathing issue somewhere (compilation is fine, but the linker isn't finding the 3rd party dependencies).

[lukaswozniak]

tyvm! Im going to check it later today

[lukaswozniak]

Seems that my issue may be a different problem. I will create separate PR