Huy-Ngo / skin-crawler

A web crawler for crawling dermoscopic images of skin cancer.
MIT License
4 stars 2 forks source link

Shared folder for downloaded image #11

Closed Huy-Ngo closed 3 years ago

Huy-Ngo commented 3 years ago

Since two of our API sources (Kaggle and ISIC) require downloading to the backend, let's move those images to a common folder backend/images/ for ease of hosting. They should be named in a way to avoid conflict (e.g. prefixing with ISIC- and Kaggle-). As far as I'm aware ISIC images are already prefixed, not sure about Kaggle.

Huy-Ngo commented 3 years ago

I will link those in the route /images manually then.

Huy-Ngo commented 3 years ago

Reopen this issue.

I just learned that files in the ./static/ are served automatically by flask where . is the same directory as app.py. Please move the downloaded images folder to backend/static/isic/ and backend/static/kaggle/ @dinhanhx @casker2209

Like this:

image

Huy-Ngo commented 3 years ago

If possible, please move the config file inside the project folder as well @dinhanhx

dinhanhx commented 3 years ago

If possible, please move the config file inside the project folder as well @dinhanhx

The kaggle api token file's path is fixed and defined by module kaggle-api. Therefore it's not possible for me to move it into project folder.

Huy-Ngo commented 3 years ago

OK then. Until this issue is resolved, I'll finish my PR as it it already has been.

dinhanhx commented 3 years ago

I just learned that files in the ./static/ are served automatically by flask where . is the same directory as app.py. Please move the downloaded images folder to backend/static/isic/ and backend/static/kaggle/ @dinhanhx @casker2209

According to my own kaggle doc download func, you can specify download path by yourself.

For example, I assume you have this block of code in app.py,

from pathlib import Path
import kaggle

download_path = './static/kaggle/'
download_path = Path(download_path)
dataset_path = kaggle.download('dinhanhx/skin-cancer-mnist-ham10000', download_path)
scm = kaggle.SkinCancerMnist(dataset_path)
Huy-Ngo commented 3 years ago

Oh I see.