Filter invalid characters from URL slug

[cebe]

What steps will reproduce the problem?

When creating a new page, the URL slug is filled from the page title.

If I fill How and Why?, the URL defaults to how-and-why?, when URL Path segment contains a ? the page can not be loaed.

What is the expected result?

Invalid characters should be replaced/filtered, or validation should apply to warn users about it.

What do you get instead? (A Screenshot can help us a lot!)

Additional infos

Q	A
LUYA Version	core 1.0.18, cms 1.0.9
PHP Version	7.1.7
Platform	Apache
Operating system	Linux Server

[nadar]

If you say when URL Path segment contains a ? the page can not be loaed. do you mean frontend? or do you get a javascript error?

There where a few discussions regarding url paths/slugs - because it seems that transliteration is not helpfull in url context when talking about search engines:

https://github.com/luyadev/luya/issues/1557

[cebe]

Frontend can not load the page. The generated URL is http://exampl.com/de/features/how-and-why? which will result in the request being http://exampl.com/de/features/how-and-why (because ? is the query param delimiter) which then redirects to http://exampl.com/de/features.

[nadar]

Ok sure ? should not be a valid char in this context, otherwise it should %3F instead of ?.

[hbugdoll]

Ok sure ? should not be a valid char in this context, otherwise it should %3F instead of ?.

% is currently stripped afterwards, so URL-encoded characters are currently not possible. http://exampl.com/de/features/how-and-why3f will generated in this case.

But there are some more problems:

• # also results in a 404 error.
• /, \ and _ raise an "URL path segment is invalid" error on input.
  • This is ok, but not user-friendly for two reasons: The user is not informed why the error occurs and must correct the input manually. These characters should be removed automatically during input.
• Some characters are stripped afterwards Edit: ... or replaced by - in core/helpers/Inflector::slug().
  • This is ok, but not user-friendly. The user should receive feedback immediately.
• RFC 3986 listed reserved characters in URIs/URLs.
  • They should better be removed.

I have created an overview...

Character	Resolve Error (404)	Error on Input	Stripped	Replaced		Reserved (RFC 3986)
?	X				X
#	X				X
/		X	(X)		X
\		X	(X)
_		X
%			X
.			X
< >			X
{ }			X
\|			X
`			X
=			X	(X)	X
+			X		X
—				X
–				X
whitespace				X
!					X
$					X
&					X
'					X
( )					X
[ ]					X
*					X
,					X
:					X
;					X
@					X

[nadar]

I am going to close this issue because:

• Its a question (Please post questions at LUYA discussions)
• Missing information's
• No more activity
• No one picked up the task (no Pull Request received)

If you think this is still important or there are more/new informations. Please reopen the issue and let us know.

If this is a problem or a feature you like to discuss, join our discussion: LUYA discussions. We try to keep our issue tracker as clean as possible in order to focus on active issues.

[hbugdoll]

core/helpers/Inflector::slug() needs some improvements:

• avoid at least the 404 errors caused by ? and #
• . | + should better replaced by - instead of stripped
  • to avoid distortion of meaning in regard of SEO, e.g. 2.5 years→ 25-years, 1+1→ 11 or pine|apple→ pineapple
• = cannot currently replaced by - because it's accidentally stripped before

[nadar]

https://github.com/luyadev/luya-module-cms/blob/master/src/models/NavItem.php#L168

if transliteration would be enabled, we would have much much less problems, it would follow the original yii2 inflector rules. Maybe the cms should just have a module config option whether transliteration should be enabled or not.

https://github.com/luyadev/yii-helpers/blob/master/src/helpers/Inflector.php#L38

[hbugdoll]

Maybe the cms should just have a module config option whether transliteration should be enabled or not.

This would be added value. My suggestion: https://github.com/hbugdoll/luya-module-cms/commit/423ccaf921909b7b54467c9b7ca8e12e2faaa85e