Converting to Markdown fails with: NoMethodError: undefined method `identifier' for nil:NilClass

[benibr]

Hi and thanks very much for this plugin! I'd like to move to Markdown but unfortunately the conversion is failing with:

...
Wiki versions: converting 914/914 historic content revisions
rake aborted!
NoMethodError: undefined method `identifier' for nil:NilClass
...

I tried it with Redmine 4.0.0 and 4.2.3, the error is the same. I suspect a project with a kind of bad setting but I cannot figure out a way to display more information or to run to conversion just for one specific project. Any help appreciated

[benibr]

Full Log:

redmine@dev-redmine:~$ bundle exec rake reformat:convert to_formatting=common_mark dryrun=1 --trace RAILS_ENV=production
** Invoke reformat:convert (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute reformat:convert
Running with setup:
{:converters_json=>"(use default converters)",
 :to_formatting=>"common_mark",
 :workers=>1,
 :dryrun=>true,
 :from_formatting=>"(based on your Setting.text_formatting: textile)"}
All 1/1 workers started.
Settings
Comment: converting 0/0 non-blank [:content] occurences of 0 total
Document: converting 0/0 non-blank [:description] occurences of 2 total
Issue: converting 4718/4718 non-blank [:description] occurences of 5790 total
Issue: 1000 / 4718
Issue: 2000 / 4718
Issue: 3000 / 4718
Issue: 4000 / 4718
Issue: 4718 / 4718
JournalDetail[Issue.description]: converting 478/478 non-blank [:value, :old_value] occurences of 478 total
JournalDetail[Issue.description]: 478 / 478
Journal: converting 5467/5467 non-blank [:notes] occurences of 15671 total
Journal: 1000 / 5467
[WARNING] Journal#5369: /issues/2578#change-5369 - placeholder '«qtag»%' usage is 4 at the end
Journal: 2000 / 5467
Journal: 3000 / 5467
Journal: 4000 / 5467
Journal: 5000 / 5467
Journal: 5467 / 5467
Message: converting 0/0 non-blank [:content] occurences of 0 total
News: converting 0/0 non-blank [:description] occurences of 0 total
Project: converting 7/7 non-blank [:description] occurences of 26 total
Project: 7 / 7
WikiContent: converting 109/109 non-blank [:text] occurences of 109 total
WikiContent: 109 / 109
Wiki versions: converting 914/914 historic content revisions
rake aborted!
NoMethodError: undefined method `identifier' for nil:NilClass
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:184:in `block in migrate_wiki_versions'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:70:in `each'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:70:in `block in find_each'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:222:in `loop'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:222:in `in_batches'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:182:in `migrate_wiki_versions'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:132:in `do_migrate'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:118:in `block (3 levels) in call'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/context.rb:40:in `with_cached_projects'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:117:in `block (2 levels) in call'
/opt/redmine/app/models/mailer.rb:634:in `with_deliveries'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:116:in `block in call'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/usr/local/rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/usr/local/rvm/gems/ruby-2.6.3/gems/activerecord-5.2.6/lib/active_record/transactions.rb:212:in `transaction'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:115:in `call'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/convert_redmine.rb:110:in `call'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/invoker.rb:36:in `convert_redmine'
/opt/redmine/plugins/redmine_reformat/lib/redmine_reformat/invoker.rb:24:in `run'
/opt/redmine/plugins/redmine_reformat/lib/tasks/reformat.rake:13:in `block (2 levels) in <top (required)>'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/usr/local/rvm/gems/ruby-2.6.3/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/rvm/gems/ruby-2.6.3/bin/rake:23:in `load'
/usr/local/rvm/gems/ruby-2.6.3/bin/rake:23:in `<main>'
/usr/local/rvm/gems/ruby-2.6.3/bin/ruby_executable_hooks:24:in `eval'
/usr/local/rvm/gems/ruby-2.6.3/bin/ruby_executable_hooks:24:in `<main>'
Tasks: TOP => reformat:convert

[martincizek]

Hi @benibr, seems like a referential integrity issue. Some wiki data perhaps reference a deleted project. Is it possible?

Can you try to comment out the line calling migrate_wiki_versions (see below) and repeat the dryrun?

iff --git a/lib/redmine_reformat/convert_redmine.rb b/lib/redmine_reformat/convert_redmine.rb
index 07ba76e..2897ed1 100644
--- a/lib/redmine_reformat/convert_redmine.rb
+++ b/lib/redmine_reformat/convert_redmine.rb
@@ -129,7 +129,7 @@ module RedmineReformat
       @exn.start
       migrate_settings if @exn.master?
       migrate_objects
-      migrate_wiki_versions
+      # migrate_wiki_versions
       migrate_custom_values
       Setting.text_formatting = @to_formatting if @exn.master? && !@exn.dryrun
       unless @exn.finish(true)

If this solves the issue, I'll prepare a fix. It seems that Redmine works around this condition as a side effect of wiki move feature, which might have hidden the issue that caused this.

[benibr]

Thanks for the fast answer! Your fix works like a charm.

redmine@dev-redmine:~$ bundle exec rake reformat:convert to_formatting=common_mark dryrun=1 --trace RAILS_ENV=production
** Invoke reformat:convert (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute reformat:convert
Running with setup:
{:converters_json=>"(use default converters)",
 :to_formatting=>"common_mark",
 :workers=>1,
 :dryrun=>true,
 :from_formatting=>"(based on your Setting.text_formatting: textile)"}
All 1/1 workers started.
Settings
Comment: converting 0/0 non-blank [:content] occurences of 0 total
Document: converting 0/0 non-blank [:description] occurences of 2 total
Issue: converting 4718/4718 non-blank [:description] occurences of 5790 total
Issue: 1000 / 4718
Issue: 2000 / 4718
Issue: 3000 / 4718
Issue: 4000 / 4718
Issue: 4718 / 4718
JournalDetail[Issue.description]: converting 478/478 non-blank [:value, :old_value] occurences of 478 total
JournalDetail[Issue.description]: 478 / 478
Journal: converting 5467/5467 non-blank [:notes] occurences of 15671 total
Journal: 1000 / 5467
[WARNING] Journal#5369: /issues/2578#change-5369 - placeholder '«qtag»%' usage is 4 at the end
Journal: 2000 / 5467
Journal: 3000 / 5467
Journal: 4000 / 5467
Journal: 5000 / 5467
Journal: 5467 / 5467
Message: converting 0/0 non-blank [:content] occurences of 0 total
News: converting 0/0 non-blank [:description] occurences of 0 total
Project: converting 7/7 non-blank [:description] occurences of 26 total
Project: 7 / 7
WikiContent: converting 109/109 non-blank [:text] occurences of 109 total
WikiContent: 109 / 109
1/1 workers finished successfuly.
1/1 workers confirmed completion.
Progress monitoring finished.

[martincizek]

Your fix works like a charm.

Well skipping wiki version migration is actually not a fix. But if historic versions are not needed, then it's a workaround.

We would need to examine the data in-depth to find the cause.

[benibr]

Well skipping wiki version migration is actually not a fix. But if historic versions are not needed, then it's a workaround. Yeah, well ... words ;-)

After tracking down the culprit wikiversion, I found a database entries which pointed to a non-existing project ID. Thanks again for the instant help!

If anyone else has the same trouble, I've just added the following line

diff --git a/lib/redmine_reformat/convert_redmine.rb b/lib/redmine_reformat/convert_redmine.rb
index adb0209..8cd1730 100644
--- a/lib/redmine_reformat/convert_redmine.rb
+++ b/lib/redmine_reformat/convert_redmine.rb
@@ -180,6 +180,7 @@ module RedmineReformat

       finished = 0
       scope.includes(:page => { :wiki => :project }).find_each do |version|
+        puts version.attributes
         ctx.ref = "WikiContentVersion\##{version.id}: "\
           "/projects/#{version.project.identifier}"\
           "/wiki/#{version.page.title}/#{version.version}"

to show which database row triggers the error and deleted in manually.