Mixed syntax scanner

[acorcutt]

I'm not sure if you want to pull this in - but it includes a new scanner that supports mixed syntax:

It will parse both <r:tag></r:tag> and {r:tag}{/r:tag} block styles and also a {tag} style without a prefix or end-tag.

So you can choose the standard <xml:tag> syntax to hide tags from wysiwyg editors, and switch to {block:tags} when you want to see something, also the {tag} style looks better inside attributes <a href="{url}">{Title}</a>

It also gets close to supporting tumblr style templates{block:Post}{Title}{/block:Post}

[saturnflyer]

Interesting. I took a quick look through but didn't see a test like this (perhaps I missed it). What happens with this scanner if I try:

<r:something>some content{/r:something}

Will that raise an error or just quietly pass?

[acorcutt]

It will allow different start/end syntax, but if this is something that should not be allowed I'm not sure if this should be checked inside the scanner or if it should be something that is passed through to the Parser which is already doing the start/end tag checks.

[jocubeit]

It seems the MixedScanner won't allow:

{tag}some content{/tag}
or
<tag>some content</tag>

How would the regex be changed to support an optional prefix?

[acorcutt]

No it will match single tags {tag} but for blocks you need the prefix {block:tag}{/block:tag}

To support {tag}content{/tag} you need something like this:

%r{\{([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)\}|\{\/([\w:]+?)\s*\}}

But to support <tag> you will also match any html tags, so you would need to restrict it to your own tags somehow.

[jocubeit]

Thanks for the fast reply.

I actually don't require the html/xml style tags at all, so having to restrict matches to my own tags isn't really an issue.

I changed the regex in the MixedScanner to the following, using your suggestion, but adding the #{prefix} in place of the \w:

%r{<#{prefix}:([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)>|<\/#{prefix}:([\w:]+?)\s*>|\{([#{prefix}:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)\}|\{\/([#{prefix}:]+?)\s*\}|\{\s*([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)\}}

I'm trying to use the following tag combination:

{truncate length="50"}{widget:description}{/truncate}

The scanner is now confused and fails. The output is the un-truncated widget#description with {/truncate} appended.

When using the original regex, and prefixes in my truncate tag, the output is perfect:

{r:truncate length="50"}{widget:description}{/r:truncate}

I feel your answer is close, but the lack of an end tag for the inner tag confuses the scanner. Adding a self-closing tag however doesn't fix the issue.

I suck at regex. Your help is much appreciated.

[saturnflyer]

@MeetDom wouldn't your example need a self-closing tag {widget:description/}? (I've not tested to see if this fixes your problem)

[jocubeit]

@saturnflyer Yes I did try that, but it doesn't fix the issue.

I'm on Mac and so downloaded Reggie and RegExhibit. Both tools match well, and RegExhibit actually shows the match details.

I'm currently using this regex:

<([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)>|<\/([\w:]+?)\s*>|\{([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)\}|\{\/([\w:]+?)\s*\}|\{\s*([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)\}

And the following text:

<tag>Content</tag>
<tag attr="1">Content</tag>
<prefix:tag>Content</prefix:tag>
<prefix:tag attr="1">Content</prefix:tag>
{tag}Content{/tag}
{tag attr="1"}Content{/tag}
{tag}{content}{/tag}
{tag attr="1"}{content}{/tag}
{tag attr="1"}{content attr2="something"}{/tag}
{prefix:tag}Content{/prefix:tag}
{prefix:tag attr="1"}Content{/prefix:tag}
{prefix:tag}{content}{/prefix:tag}
{prefix:tag attr="1"}{content attr2="something"}{/prefix:tag}
{prefix:tag}{widget:description}{/prefix:tag}
{tag}{widget:description}{/tag}
{tag}{widget:description truncate="50"}{/tag}

All variations are matching. The problem doesn't seem to be the regex, just the ruby statement when interpolating the #{prefix}. I get {/truncate} at the end of the output, and the truncate fails.

The latest variation of the regex I'm using in the MixedScanner is:

%r{<([#{prefix}:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)>|<\/([#{prefix}:]+?)\s*>|\{([#{prefix}:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)(\/?)\}|\{\/([#{prefix}:]+?)\s*\}|\{\s*([\w:]+?)(\s+(?:\w+\s*=\s*(?:"[^"]*?"|'[^']*?')\s*)*|)\}}

[acorcutt]

Yes, sorry the regex will match but I forgot the scanner also modifies operate() to handle the non-closed {tag} style.

If you use the operate() from the default scanner you will be able to do {tag}{another:here /}{/tag} or {block:style}{tag /}{/block:style} but you will need to close all tags.

To support the mixing of closed/non-closed tags it will need a change to the parser that injects any missing end tags rather than raise an error. I'll give it a try but would this be something that could be pulled into radius without changing how existing templates work?