Double duplicate keys causes Segmentation Fault

[sadleb]

If I process a file with the following contents, it seg faults with the below exception. Notice the duplicate medium_with_weekday and long_with_weekday keys.

en:
  date:
    formats:
      medium_with_weekday: "%a %b %-d, %Y"
      long_with_weekday: "%A, %B %-d"
      long: "%B %-d, %Y"
      long_with_weekday: "%A, %B %-d"
      medium: "%b %-d, %Y"
      medium_month: "%b %Y"
      medium_with_weekday: "%a %b %-d, %Y"
    days:
      today_lower: "today"

Exception:

/var/canvas/releases/20160623161622/gems/i18n_tasks/lib/tasks/i18n.rake:102: [BUG] Segmentation fault at 0x007fd6c7993770
ruby 2.1.8p440 (2015-12-16 revision 53160) [x86_64-linux-gnu]

-- Control frame information -----------------------------------------------
c:0023 p:---- s:0086 e:000085 CFUNC  :direct_lookup
c:0022 p:0126 s:0083 e:000082 BLOCK  /var/canvas/releases/20160623161622/gems/i18n_tasks/lib/tasks/i18n.rake:102 [FINISH]
c:0021 p:---- s:0075 e:000074 CFUNC  :call
c:0020 p:0028 s:0070 e:000069 BLOCK  /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240 [FINISH]
c:0019 p:---- s:0067 e:000066 CFUNC  :each
c:0018 p:0113 s:0064 e:000063 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235
c:0017 p:0075 s:0060 e:000059 BLOCK  /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:179
c:0016 p:0014 s:0058 e:000057 METHOD /usr/lib/ruby/2.1.0/monitor.rb:211
c:0015 p:0025 s:0055 e:000054 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:172
c:0014 p:0036 s:0048 e:000047 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:165
c:0013 p:0033 s:0043 e:000042 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:150
c:0012 p:0009 s:0036 e:000035 BLOCK  /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106 [FINISH]
c:0011 p:---- s:0033 e:000032 CFUNC  :each
c:0010 p:0039 s:0030 e:000029 BLOCK  /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106
c:0009 p:0025 s:0028 e:000027 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:115
c:0008 p:0007 s:0024 e:000023 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:100
c:0007 p:0019 s:0021 e:000020 BLOCK  /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:78
c:0006 p:0006 s:0019 e:000018 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:176
c:0005 p:0007 s:0015 e:000014 METHOD /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:75
c:0004 p:0040 s:0012 e:000011 TOP    /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/bin/rake:33 [FINISH]
c:0003 p:---- s:0010 e:000009 CFUNC  :load
c:0002 p:0135 s:0006 E:0019e8 EVAL   /var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/bin/rake:23 [FINISH]
c:0001 p:0000 s:0002 E:001dc8 TOP    [FINISH]

-- Ruby level backtrace information ----------------------------------------
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/bin/rake:23:in `<main>'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/bin/rake:23:in `load'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:100:in `top_level'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
/usr/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
/var/canvas/releases/20160623161622/vendor/bundle/ruby/2.1.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
/var/canvas/releases/20160623161622/gems/i18n_tasks/lib/tasks/i18n.rake:102:in `block (2 levels) in <top (required)>'
/var/canvas/releases/20160623161622/gems/i18n_tasks/lib/tasks/i18n.rake:102:in `direct_lookup'

-- C level backtrace information -------------------------------------------

[jenseng]

Good catch, I'll take a look

[surki]

Ran into similar kind of issue (duplicate entry and as well as hash merge). For time being I am using the following fix: https://github.com/surki/i18nema/commit/e05b151cf4d687edd82d46a2e26101355f5586b4

Ran it under valgrind with all my locales (about 10 langs, with ~250K objects), seems to hold up okay.

[jenseng]

My apologies, I haven't had time to look into this, and Instructure doesn't actually use this gem anymore in canvas. The GC improvements in newer versions of Ruby make i18nema's wins less significant. We can leave this open if someone wants to submit a pull request, but it's pretty unlikely that I (or someone at Instructure) will have time to fix this.