Is this the expected behavior of --out_dir?

[vdet]

Hello again,

I have to process dozens of image datasets. For each one I set --out_dir to specific values, say output_001, but then data fail to load when I connect to http://localhost:5000/output_001. I get:

[...]
::ffff:127.0.0.1 - - [22/Apr/2021 18:55:50] "GET /output-001/output-001/data/imagelists/imagelist-26bf6650-a387-11eb-98e9-38c9861de4fc.json HTTP/1.1" 404 -

File /output-001/output-001/data/imagelists/imagelist-26bf6650-a387-11eb-98e9-38c9861de4fc.json does not exist indeed. File output-001/data/imagelists/imagelist-26bf6650-a387-11eb-98e9-38c9861de4fc.json does.

Am I missing something in the way I hangle the --our_dir option?

Cheers,

Vincent

[duhaime]

Whoops! Sorry @vdet . We just pushed a commit that should resolve this issue. If you try that out, it'd be good to hear how it goes!

If you need to salvage a collection of plots you've already generated, please just let us know and we can help.

[vdet]

Thanks Douglas! The fix raises an error in pixplot.pl which is unrelated to setting --out_dir to a new value:

$ python3 pixplot/pixplot.py --image "../../../my_data/tiles/res-224/*png" --metadata ../../../my_data/pixplot_metadata.csv 
2021-04-23 09:16:39.887249: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-04-23 09:16:39.887519: I tensorflow/core/common_runtime/direct_session.cc:361] Device mapping: no known devices.
Traceback (most recent call last):
  File "/Users/detours/ai/ciga-resnet18/code/python/pix-plot-master/pixplot/pixplot.py", line 1381, in <module>
    parse()
  File "/Users/detours/ai/ciga-resnet18/code/python/pix-plot-master/pixplot/pixplot.py", line 1378, in parse
    process_images(**config)
  File "/Users/detours/ai/ciga-resnet18/code/python/pix-plot-master/pixplot/pixplot.py", line 130, in process_images
    kwargs['image_paths'], kwargs['metadata'] = filter_images(**kwargs)
  File "/Users/detours/ai/ciga-resnet18/code/python/pix-plot-master/pixplot/pixplot.py", line 212, in filter_images
    l = get_metadata_list(**kwargs)
  File "/Users/detours/ai/ciga-resnet18/code/python/pix-plot-master/pixplot/pixplot.py", line 306, in get_metadata_list
    with open(kwargs['metadata']) as f:
FileNotFoundError: [Errno 2] No such file or directory: '../../../my_datamy_data/pixplot_metadata.csv'

Somehow the path ../../../my_data/pixplot_metadata.csv got changed to ../../../my_datamy_data/pixplot_metadata.csv.

If you need to salvage a collection of plots you've already generated, please just let us know and we can help.

Thanks for the offer but I just generates test images at this point.

Vincent

[duhaime]

@vdet this is curious indeed. The metadata path processing logic didn't change in that commit I pushed yesterday (I don't think it's change in a while). Does adding quotes around the metadata path help?

[vdet]

I just tried what you suggested, but unfortunately it does not fix the problem.

Cheers,

Vincent

[duhaime]

@vdet hmm. I'm a little nonplussed. I'm happy to do a little 5 minute screenshare, in which I'm sure we can get this sorted.

If you wanted, you could start a Python terminal (or Jupyter notebook) and:

import pixplot
print(pixplot.__file__)

That will tell you where the pixplot source is on your machine. Then if you were to edit pixplot.py so as to add a print(kwargs) between lines 128 and 129 and then between lines 300 and 301. We expect to see the exact path to the metadata.csv file you provided displayed in both instances. If you could paste the full output from the command issuance through the stacktrace (including those outputs) that would be helpful!

[vdet]

Oooup! Another human error: I realized while implementing the flags that I got mixed that I got confused with the argument. Your fix does work. My apology! And thanks for your responsiveness and patience.

Vincent

[duhaime]

Ah praise be to God I was afraid there was a ghost in the machine! 👻