kikito / md5.lua

MD5 sum in pure Lua, with no C and no external dependencies
MIT License
326 stars 151 forks source link

Incremental md5 #10

Closed ghost closed 8 years ago

ghost commented 8 years ago

Implements #8.

Tested with a 47 Gb file, with LuaJIT only, using the following program:

local md5 = require 'md5'

local f,x = io.open(arg[1], 'rb')
print(f, x)

local state = md5.new()

repeat
  local s = f:read(math.random(1, 800))
  if not s then break end
  state:update(s)
until false
f:close()

print(md5.tohex(state:finish()))

(took 3+ hours; the result was compared to that of md5sum which took ~8 minutes). The updates were sent with random lengths to exercise different buffer fills.

Couldn't test with Lua as it refused to open the file.

It is based on the commits of PR #9, so they are included too. I can rebase if necessary.

kikito commented 8 years ago

I have not forgotten about this (hectic weekend). I still have to give it a more thorough review.

ghost commented 8 years ago

Thanks for the heads-up. Take your time, there's no hurry. Meanwhile I'll push my rebased version.

kikito commented 8 years ago

I finally was able to review this (sorry about the delay).

I made a small change: update returns self. This way, you can chain commands:

local x = md5.new():update('foo'):update('bar'):finish()

I will now create a changelog and release md5 1.1.0

EDIT: updated and uploaded to Luarocks!