search

dividiti / ck-caffe

Collective Knowledge workflow for Caffe to automate installation across diverse platforms and to collaboratively evaluate and optimize Caffe-based workloads across diverse hardware, software and data sets (compilers, libraries, tools, models, inputs):
http://cKnowledge.org
BSD 3-Clause "New" or "Revised" License
193 stars 40 forks source link

Remove duplicate files #90

Open psyhtest opened 7 years ago

psyhtest commented 7 years ago

The CK-Caffe repository contains several duplicate files:

$ find $CK_REPOS/ck-caffe -name *binaryproto -exec ls -la {} \;
-rw-r--r-- 1 anton dvdt 602126 Apr  7 19:09 /home/anton/CK/ck-caffe/package/caffemodel-resnet101/ResNet_mean.binaryproto
-rw-r--r-- 1 anton dvdt 602126 Apr  7 19:09 /home/anton/CK/ck-caffe/package/caffemodel-resnet50/ResNet_mean.binaryproto
-rw-r--r-- 1 anton dvdt 602126 Apr  7 19:09 /home/anton/CK/ck-caffe/package/caffemodel-resnet152/ResNet_mean.binaryproto
-rw-r--r-- 1 anton dvdt 786446 Jan 13 09:00 /home/anton/CK/ck-caffe/program/caffe-classification-cuda/imagenet_mean.binaryproto
-rw-r--r-- 1 anton dvdt 786446 Jan 23 14:15 /home/anton/CK/ck-caffe/program/caffe/for_classification_demo/imagenet_mean.binaryproto
-rw-r--r-- 1 anton dvdt 786446 Jan 23 14:15 /home/anton/CK/ck-caffe/program/caffe-classification-opencl/imagenet_mean.binaryproto
-rw-r--r-- 1 anton dvdt 786446 Nov  8 16:34 /home/anton/CK/ck-caffe/program/caffe-classification/imagenet_mean.binaryproto
$ find $CK_REPOS/ck-caffe -name synset_words.txt -exec ls -la {} \;
-rw-r--r-- 1 anton dvdt 31675 Jan 13 09:00 /home/anton/CK/ck-caffe/program/caffe-classification-cuda/synset_words.txt
-rw-r--r-- 1 anton dvdt 31675 Jan 23 14:15 /home/anton/CK/ck-caffe/program/caffe/for_classification_demo/synset_words.txt
-rw-r--r-- 1 anton dvdt 31675 Jan 23 14:15 /home/anton/CK/ck-caffe/program/caffe-classification-opencl/synset_words.txt
-rw-r--r-- 1 anton dvdt 31675 Nov  8 16:34 /home/anton/CK/ck-caffe/program/caffe-classification/synset_words.txt

These should be moved into separate entries, with dependencies set up on them.

psyhtest commented 7 years ago

This should also solve the current problem e.g. with program:caffe-classification-cuda:

$ ck run program:caffe-classification-cuda
...
0) default ($#BIN_FILE#$ $<<CK_CAFFE_MODEL_FILE>>$ $<<CK_CAFFE_MODEL_WEIGHTS>>$ $#up_dir#$imagenet_mean.binaryproto $#up_dir#$synset_words.txt $#dataset_path#$$#dataset_filename#$)
1) use_continuous ($#BIN_FILE#$ $<<CK_CAFFE_MODEL_FILE>>$ $<<CK_CAFFE_MODEL_WEIGHTS>>$ $<<CK_CAFFE_IMAGENET_MEAN_BIN>>$ $<<CK_CAFFE_IMAGENET_SYNSET_WORDS_TXT>>$ --continuous $<<CK_CAFFE_IMAGENET_VAL>>$ $<<CK_CAFFE_IMAGENET_VAL_TXT>>$)
2) use_external_image ($#BIN_FILE#$ $<<CK_CAFFE_MODEL_FILE>>$ $<<CK_CAFFE_MODEL_WEIGHTS>>$ $#up_dir#$imagenet_mean.binaryproto $#up_dir#$synset_words.txt $<<CK_CAFFE_PATH_TO_IMAGE>>$)

where the commands 0 and 2 refer to the up_dir which is not correct for the ResNet's...