tomMoral / dicodile

Experiments for "Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals"

https://tommoral.github.io/dicodile/

BSD 3-Clause "New" or "Revised" License

18 stars 10 forks source link

Dicodile text example #21

Closed hndgzkn closed 3 years ago

hndgzkn commented 3 years ago

Add a draft example on using dicodile on a text image generated with letters PAMI of text_size=5000.

Creates 10 patches.
Uses 10 jobs.
Takes ~45 min. to run dicodile.

As the job runs too long, the artifacts can be viewed from https://github.com/hndgzkn/dicodile/actions/runs/624302432

codecov[bot] commented 3 years ago

Codecov Report

Merging #21 (a9baef0) into main (85a6310) will increase coverage by 0.12%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #21      +/-   ##
==========================================
+ Coverage   74.15%   74.28%   +0.12%     
==========================================
  Files          41       41              
  Lines        2573     2586      +13     
==========================================
+ Hits         1908     1921      +13     
  Misses        665      665

Flag	Coverage Δ
unittests	`74.28% <100.00%> (+0.12%)`	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
dicodile/data/_text.py	`0.00% <ø> (ø)`
dicodile/data/images.py	`76.92% <100.00%> (+12.21%)`	:arrow_up:
dicodile/data/tests/test_images.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 85a6310...a9baef0. Read the comment docs.

agramfort commented 3 years ago

@hndgzkn can you share an image of the results? thx

hndgzkn commented 3 years ago

@hndgzkn can you share an image of the results? thx

dict_image

text_image

agramfort commented 3 years ago

looks great !

hndgzkn commented 3 years ago

I'd like to document text image generation dependencies:

pandoc
texlive
texlive-extra-utils
texlive-fonts-extra
ImageMagic

Shall I document it in Installation section of README.rst?

agramfort commented 3 years ago

I think this image was produced once for the paper and for illustration purposes.

I don't think it's really meant to be advertised. I would expect to just have one image downloaded for illustration.

hndgzkn commented 3 years ago

In that case:

Instead of generating the image, shall I save the image under data or data/images directory and return this image (may be images of different sizes)?
Shall I keep the code to generate text?

agramfort commented 3 years ago

I would keep the code somewhere to replicate the image and yes store the image available for download.

basically like we do it for mandrill

hndgzkn commented 3 years ago

I created the file text_4_5000_PAMI.npz (~18M) and saved it under dicodile/data/samples directory.
I added a method load_pami inside dicodile/data/images.py to load PAMI image X and associated D.

I am trying to use gitlfs to push the file, however it is not possible due to pushing-large-files-to-forks.

@agramfort Do you have any suggestion?

I would keep the code somewhere to replicate the image and yes store the image available for download. basically like we do it for mandrill …

agramfort commented 3 years ago

@hndgzkn are you suggesting to add the file to the main github repo? I would not add any big binary file to the source code repo.

I would put the file somewhere on the web eg in https://figshare.com/ https://zenodo.org/

hndgzkn commented 3 years ago

this looks good to me. A few nitpicks.

Also, maybe add a print for the full size of the image when loading it?

Initially I had it, but removed as you can hardly see anything.

whole_image

If you prefer, I can add it.

May be you didn't mean that?

tomMoral commented 3 years ago

May be you didn't mean that?

Yeah, I like the zoomed one, it is much clearer. I was mentionning adding print(X_orginal.shape), so the user see the scale of the problem.

hndgzkn commented 3 years ago

I was mentionning adding print(X_orginal.shape), so the user see the scale of the problem.

I print the shape for X after reshaping and also I print the shape of dictionary D and D_init as well thinking that it would be also good to see the expected shape for dicodile. If you prefer X_original, I'll change it.

tomMoral commented 3 years ago

I print the shape for X after reshaping and also I print the shape of dictionary D and D_init as well thinking that it would be also good to see the expected shape for dicodile. If you prefer X_original, I'll change it.

that is perfect like this. Merging! thanks @hndgzkn :tada:

agramfort commented 3 years ago

🎉