Closed DHG-Dav closed 2 years ago
KichangKim
commented
2 years ago You should double back-slash for the database path. (or use slash instead.)
"database_path": "A:\NN\DeepTag\dataset\test21.sqlite",
to
"database_path": "A:/NN/DeepTag/dataset/test21.sqlite",
DHG-Dav
commented
2 years ago Yes i do it, otherwise i get an error message, however on github when i copypaste, the double back-slash is replaced with a single one. Maybe because i didn't use the "code" thingy ?
{
"image_width": 299,
"image_height": 299,
"database_path": "A:\\NN\\DeepTag\\test1\\test21.sqlite",
"minimum_tag_count": 15,
"model": "resnet_custom_v3",
"minibatch_size": 2,
"epoch_count": 10,
"export_model_per_epoch": 10,
"checkpoint_frequency_mb": 200,
"console_logging_frequency_mb": 10,
"optimizer": "adam",
"learning_rate": 0.001,
"rotation_range": [
0.0,
360.0
],
"scale_range": [
0.9,
1.1
],
"shift_range": [
-0.1,
0.1
],
"mixed_precision": false
}In other word the problem doesn't come from this Example, here i remove the double backslash :
Traceback (most recent call last):
File "A:\ANACONDA\envs\deeptag\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "A:\ANACONDA\envs\deeptag\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "A:\ANACONDA\envs\deeptag\Scripts\deepdanbooru.exe\__main__.py", line 7, in <module>
File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1053, in main
rv = self.invoke(ctx)
File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\__main__.py", line 52, in train_project
dd.commands.train_project(project_path, source_model)
File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\commands\train_project.py", line 30, in train_project
project_context = dd.io.deserialize_from_json(project_context_path)
File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\io\__init__.py", line 13, in deserialize_from_json
return json.loads(stream.read())
File "A:\ANACONDA\envs\deeptag\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "A:\ANACONDA\envs\deeptag\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "A:\ANACONDA\envs\deeptag\lib\json\decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid \escape: line 4 column 25 (char 75)And i get an error message.
But my problem doesn't raise any error, it just doesn't train, doesn't use the images... I just edited my first message with the "code" tags (using ctrl+e) so you don't get confused further.
KichangKim
commented
2 years ago Can you share test21.sqlite?
DHG-Dav
commented
2 years ago shader ? you mean share ? here is a picture : https://i.postimg.cc/mgW7hZwB/004331.png And here is the file itself https://we.tl/t-RO5ALQmiHd
KichangKim
commented
2 years ago I got it. Current DeepDanbooru filters image by its extension, then use only png or jpeg or jpg (its lower-case). But your database contains PNG or JPG.
Change ext to lower case, or modify the source code of DeepDanbooru directly on your local: https://github.com/KichangKim/DeepDanbooru/blob/master/deepdanbooru/data/dataset.py
DHG-Dav
commented
2 years ago OMG that was so trivial and i lost my mind for days on it. Thank you so much, i'll try it right away ! Edit : it works ! after so much frustration and effort it finally work, thank you so much, i can't tell how happy i am right now !
Hello ! It's been a while i'm trying to make this run, i had some external help to solve most problems, but i can't find a solution to this :
All seems to go well, except it doesn't train... is there a verbose mode or a way to find out what's the problem ? the directories have been checked :
├─dataset │ │ test21.sqlite │ │ │ └─images │ └─00 │ 00000747f1d8598b622bd0d7ad364075.png │ [...] └─test17 project.json tags.txt
And my json :
The tags.txt contains approximately 180 tags that are all in the images (there is no "orphan" tags), all images contains 10~30 tags, and there is 7,7k images (i'm testing it on a reduced dataset first).
Thank you for any help.
PS : here are all the packages and version i have installed in this environment :