karpathy / neuraltalk2

Efficient Image Captioning code in Torch, runs on GPU
5.49k stars 1.26k forks source link

NeuralkTalk2: Stuck at "listing all images in directory" after changing DataLoaderRaw.lua file #183

Open texzone opened 7 years ago

texzone commented 7 years ago

I have installed NeuralTalk2 and, to the best of my knowledge, all dependencies. I am running Ubuntu 14.04 on VMware Workstation 12. I have around 9.7GB of RAM allocated to the VM. When I run free, this is my output:

tex@ubuntu:~/neuraltalk2$ free
             total       used       free     shared    buffers     cached
Mem:       9906788    2806640    7100148      11660      54184    1250844
-/+ buffers/cache:    1501612    8405176
Swap:      1045500          0    1045500

This is the command I'm running (and the output that is displayed afterwards):

tex@ubuntu:~/neuraltalk2$ th eval.lua -model ~/Desktop/model_id1-501-1448236541.t7_cpu.t7 -image_folder ~/catkin_ws/video_images/Best\ Melodic/ -num_images -1 -gpuid -1

tex@ubuntu:~/neuraltalk2$ th eval.lua -model ~/Desktop/model_id1-501-1448236541.t7_cpu.t7 -image_folder ~/catkin_ws/video_images/Best\ Melodic/ -num_images -1 -gpuid -1
DataLoaderRaw loading images from folder:   /home/tex/catkin_ws/video_images/Best Melodic/  
listing all images in directory /home/tex/catkin_ws/video_images/Best Melodic/  

Its just stuck there. Been stuck there for maybe 30 minutes. There are only 45 jpeg files.

I have tried changin the number of images, deleting images, making only one image available.... I dont know what to do. Its JPEG, do I don't think there should be a problem because its supported, right?

IMPORTANT: I changed the DataLoaderRaw.lua file to this:

--[[
Same as DataLoader but only requires a folder of images.
Does not have an h5 dependency.
Only used at test time.
]]--

local utils = require 'misc.utils'
require 'lfs'
require 'image'

local DataLoaderRaw = torch.class('DataLoaderRaw')

function DataLoaderRaw:__init(opt)
  local coco_json = utils.getopt(opt, 'coco_json', '')

  -- load the json file which contains additional information about the dataset
  print('DataLoaderRaw loading images from folder: ', opt.folder_path)

  self.files = {}
  self.ids = {}
  if string.len(opt.coco_json) > 0 then
    print('reading from ' .. opt.coco_json)
    -- read in filenames from the coco-style json file
    self.coco_annotation = utils.read_json(opt.coco_json)
    for k,v in pairs(self.coco_annotation.images) do
      local fullpath = path.join(opt.folder_path, v.file_name)
      table.insert(self.files, fullpath)
      table.insert(self.ids, v.id)
    end
  else
    -- read in all the filenames from the folder
    print('listing all images in directory ' .. opt.folder_path)
    local function isImage(f)
      local supportedExt = {'.jpg','.JPG','.jpeg','.JPEG','.png','.PNG','.ppm','.PPM'}
      for _,ext in pairs(supportedExt) do
        local _, end_idx =  f:find(ext)
        if end_idx and end_idx == f:len() then
          return true
        end
      end
      return false
    end
    local n = 1
    local m = 0
    local counter = 1
    while counter <= #paths.dir(opt.folder_path) - 2 do -- two directories "." and ".." returned by dir that are not images
      local file_added = false
      for file in paths.files(opt.folder_path, isImage) do
        if tonumber(string.sub(file, 7, 10)) == n then -- insert files into self.files in order
          if tonumber(string.sub(file, 12, 12)) == m then
        local fullpath = path.join(opt.folder_path, file)
        table.insert(self.files, fullpath)
        table.insert(self.ids, tostring(n)) -- just order them sequentially
        m=m+1
        counter = counter + 1
        file_added = true
        break
      end
        end
      end
      if file_added == false then
        n = n + 1
        m = 0
      end
    end
    print(self.files)
  end

  self.N = #self.files
  print('DataLoaderRaw found ' .. self.N .. ' images')

  self.iterator = 1
end

function DataLoaderRaw:resetIterator()
  self.iterator = 1
end

--[[
  Returns a batch of data:
  - X (N,3,256,256) containing the images as uint8 ByteTensor
  - info table of length N, containing additional information
  The data is iterated linearly in order
--]]
function DataLoaderRaw:getBatch(opt)
  local batch_size = utils.getopt(opt, 'batch_size', 5) -- how many images get returned at one time (to go through CNN)
  -- pick an index of the datapoint to load next
  local img_batch_raw = torch.ByteTensor(batch_size, 3, 256, 256)
  local max_index = self.N
  local wrapped = false
  local infos = {}
  for i=1,batch_size do
    local ri = self.iterator
    local ri_next = ri + 1 -- increment iterator
    if ri_next > max_index then ri_next = 1; wrapped = true end -- wrap back around
    self.iterator = ri_next

    -- load the image
    local img = image.load(self.files[ri], 3, 'byte')
    img_batch_raw[i] = image.scale(img, 256, 256)

    -- and record associated info as well
    local info_struct = {}
    info_struct.id = self.ids[ri]
    info_struct.file_path = self.files[ri]
    table.insert(infos, info_struct)
  end

  local data = {}
  data.images = img_batch_raw
  data.bounds = {it_pos_now = self.iterator, it_max = self.N, wrapped = wrapped}
  data.infos = infos
  return data
end

Is there an obvious error somewhere? I changed the function DataLoaderRaw:__init(opt) in this script to sort the files loaded by the DataLoader before appending them to self.files(line 52). Without this functionality, captions are generated for images in a random order and the progression of a scene is impossible to determine.