Bootstrap fails, setup.xqy no longer handles special characters in [env].properties files

[tdiepenbrock]

Thank you for submitting an issue!

Bug Template

The issue

Short description of the problem: deploy/lib/setup.xqy has a syntax error at line 6369 and will not run.

What are the steps to reproduce the problem?

1. do a bootstrap or anything else that runs setup.xqy

Tech Specs

Which Operating System are you using? All

Which version of MarkLogic are you using? 9.0-2

Which version of Roxy are you using (see version.txt)? 1.7.7

[grtjn]

@tdiepenbrock I see you changed the title. What kind of special chars are you talking about? I can imagine special chars could break ml-config.xml when appended to setup.xqy. There were changes related to this.

[tdiepenbrock]

Hi Geert,

I not only changed the title but also the full description. Originally I though the issue was with setup.xqy, and it is—but only in the form that eventually gets eval’d by the server. Properties with special characters are not being escaped for XQuery, so characters like ‘{‘ in property values completely break it. This frequently happens with the randomly-generated app user password, for example. We also send small hunks of JSON as property values, which also break it.

The steps to reproduce show how it breaks with the random password that Roxy generated for the app-user.

I also found a fix: the issue is actually in ServerConfig.properties_map() in server_config.rb. This fixes it (change in bold):

def properties_map
  entries = []
  @properties.each do |k, v|
    entries.push %Q{map:entry("#{k}", "#{v.xquery_safe}")}
  end
  "map:new((\n" + entries.join(",\n  ")+ "))"
end

Thanks, Tom

[grtjn]

Looks like your description edits don't show up, so thanks for the extra detail. I'll try to take a closer look at this, since I likely touched the code that caused this..

[grtjn]

Oh, and the bold doesn't show up on github.. :)

[tdiepenbrock]

lol—just need to append “.xquery_safe” to the values in that function.

[grtjn]

Our internal Jenkins was down unfortunately. Now that it is back up, it confirms issues with bootstrap. I suspect it succeeds or fails depending on generated appuser-password..

[tdiepenbrock]

It is not calling .xquery_safe on the values when it creates the properties map. So any property with special characters may result in either invalid xquery or invalid entity references, etc.

[grtjn]

Fixed in dev

[grtjn]

A hotfix was also applied to master..