Character encoding not declared results in 403 on using editor in intro locations.

GQTool commented 4 years ago

Current behavior / Resultado actual / Résultat actuel

When adding text on the course or learning path page specifically and add or edit the intro using the inbuilt editor wysiwyg on saving it results in a 403.

The page location on the editor view is:

/main/course_home/course_home.php /main/lp/lp_controller.php

And all those text editors saving to table [c_tool_intro]

The console error on submit:

"The character encoding of the HTML document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the page must be declared in the document or in the transfer protocol."

Note: error You can open the to editor in html view and add a single line of text <p>hello world</p> and it will save, however it will not save any other edits (another <p></p>) using html and results in the same 403 error.

You can edit the database directory and add the html and it will show, but if you attempt to edit in these locations using the editor it fails.

The editor and wysywig works in main course creations pages etc, only appears to be in these page intro areas.

All files are correctly set to 755 and the database is set to utf-8 and installation is as recommended.

Expected behavior / Resultado esperado / Résultat attendu

Should save the text and the formatting and correctly encode the char charset as utf-8 and save to the database.

Steps to reproduce / Pasos para reproducir / Étapes pour reproduire

Create a course
Add a learning path
Attempt to add an intro text in the course page
Attempt to add some text in the learning path intro

Results in 403 and encoding error in these location.

/main/course_home/course_home.php /main/lp/lp_controller.php

Chamilo Version / Versión de Chamilo / Version de Chamilo

1.11.10

ywarnier commented 4 years ago

@GQTool Just a quick question: is your computer/browser configured in any specific character encoding?

caos30 commented 4 years ago

[SOLVED!!!] Hi @ywarnier , i finally found how to solve this and which is the problem most of the cases. I've read this problem in other issues and it probably will continue happening if you don't apply this solution in .htaccess (or a solution like this=. let me explain what i've discovered.

Problem

The problem arise when you're submitting some HTML forms containing specific HTML/CSS/JS that are detected by modSecurity module of Apache as a false positive of XSS atack !! You can check it on the apache LOG file.

For example, when i submit the form to CREATE DOCUMENT in a lesson of a course, inserting an embeded VIDEO, i get this 403 Forbidden access response by server in my web browser after submit form, and if i see in the Apache LOG error file i see something like this:

[Fri Jul 24 04:30:55.100772 2020] [:error] [pid 19164:tid 140500090664704] [client 177.228.72.123:40348] 
[client 177.228.72.123] ModSecurity: Access denied with code 403 (phase 2). Pattern match 
"<link.{0,}?href\\\\/{0,}=" at ARGS_POST:content_lp. [file "/etc/modsecurity/07_XSS_XSS.conf"] 
[line "131"] [id "212980"] [rev "4"] [msg "WAF: IE XSS Filters - Attack Detected.||aula.mydomain.org|F|2"]
 [data "Matched Data: <linkhref= found within ARGS_POST:content_lp: <!doctypehtml><html><head>
<linkhref=\\x22https://aula.mydomain.org/web/assets/bootstrap/dist/css/bootstrap.min.css\\x22media=\\x22screen\\x22rel=\\x22stylesheet\\x22type=\\x22text/css\\x22/><linkhref=\\x22https://aula.mydomain.org/web/assets/fontawesome/css/font-awesome.min.css\\x22media=\\x22screen\\x22rel=\\x22stylesheet\\x22type=\\x22text/css\\x22/><linkhref=\\x22https://aula.mydomain.org/web/css/document.css\\x22media=\\x22screen\\x22rel=\\x22s...
"] [severity "CRITICAL"] [tag "CWAF"] [tag "XSS"] [hostname "aula.mydomain.org"] 
[uri "/main/lp/lp_controller.php"] [unique_id "XxpH31JihFgAAErcbdwAAABW"], 
referer: https://aula.mydomain.org/main/lp/lp_controller.php?cidReq=CONECTA01&id_session=0&gidReq=0&gradebook=0&origin=&action=add_item&type=step&lp_id=17&isStudentView=false

Solution

So easy as include these lines at the top of the .htaccess on the root directory of our Chamilo:

# Add exception for: /home/.../www/main/lp/******.php 
# we need it because this form generate false positive of XSS Atack

<IfModule mod_security.c>
    <If "%{REQUEST_URI} =~ m#main/lp#">
        SecRuleRemoveById 212000-212999
    </If>
</IfModule> 

<IfModule mod_security2.c>
    <If "%{REQUEST_URI} =~ m#main/lp#">
        SecRuleRemoveById 212000-212999
    </If>
</IfModule>

A few important comments:

include these 2 blocks (mod_security & mod_security2.c) because i've read on internet a lot of confusion among useres who has one or another version of mod_security in their servers. So with both blocks we get sure that in any case it will run
It's very important apply the condition for the directory, because we not want to full disable the modScurity module on our server!
We can choose this tool on .htaccess to tisable certain rules. I applied an interval around 212000 because this form was firing the rules 212740 & 212980, so i have disabled all that period for avoid other possible rules in the same "range".

Sincerely: i'm not an expert on mod_security, in fact i discovered this smart use of the .htaccess thanks to a chat with the technical support team of my hosting provider (!). And i'm a newbie too on Chamilo, so maybe this solution i've implemented should consider another "directories" and maybe another ranges of rules. So please, i would ask you to continue my initial work on this solution way ;-)

Indeed, don't hesitate thinking about if this is a Chamilo problem. In this sense, i've read today in other forum threads that this problem is typical in Wordpress and other CMS. As i said, the problem arise when you try to send some "advanced" CSS or JS from a web form of the admin panel to your server, and this is easily confused with an XSS atack. I suspect that avoid this alarm firing would means to remove some interesting functionalities of Chamilo, for example the embedding of video players. And i would say that this is not the solution.

But well, you're the boss.

Cheers!!

Note: this is my first month using seriously Chamilo for a customer of mine. And this is my first collaboration with the project here on Github. I've seen some other "bugs" or odd behaviours. So you i will be commenting here during the next weeks, i hope that contributing to improve this great "piece of software". Thanks!

ywarnier commented 4 years ago

Hi @caos30 Your contribution has been added in time for Chamilo 1.11.12. Thanks!

ywarnier commented 4 years ago

Well apparently this breaks some of our systems: .htaccess: SecRuleRemoveById not allowed in <If> context

It seems like you sent us a code that you didn't test, @caos30 ...?

lonesomewalker commented 4 years ago

As a pen tester, i suggest NOT to implement any rules like that in the .htaccess nor in the Apache config.

Best practice is to fix the error which causes this issue. In this case it would be a good idea to find out exactly WHAT has been submitted and how can this input be sanitized so there is no "false positive" anymore.

ywarnier commented 4 years ago

Agreed

ywarnier commented 4 years ago

I'm re-packaging 1.11.12 right now :-/

caos30 commented 4 years ago

As a pen tester, i suggest NOT to implement any rules like that in the .htaccess nor in the Apache config.

Best practice is to fix the error which causes this issue. In this case it would be a good idea to find out exactly WHAT has been submitted and how can this input be sanitized so there is no "false positive" anymore.

Of course @lonesomewalker , just because i understand that it's not the best solution to "hack" the rules of the mod_security module of the server, i wasted a few hours to find a way to restrict the "cancellation of only those rules for certain path of files" thanks to the condition:

<If "%{REQUEST_URI} =~ m#main/lp#">

Said this, i also agree with your recommendation to find the real problem on the app code and fix it, of course. But, regarding this recommendation:

1) in the real world, i need to give solution to my customers and all their users of Chamilo today, not several months later when app code got fixed. Believe me: that error make NOT-USABLE Chamilo in my mod_security apache configuration! so i decided that my hack of the .htaccess file could be a "temporally" good way to avoid the problem (not a solution, you're true).

2) in this specific case, as i explained several weeks above when i shared my "hack", the root problem seems be not easy to fix. I didn't dedicated very much time to search the problem and its cause, but the few i got to understand by read error apache logs and searching on internet, it seems that the problem is that the WYSIWIG editor in that part of the "lesson edition" POST to the server certain combination of HTML & javascript & CSS (as value of some of the POSTED fields in the form) which shoot the mod_security rules... ie. generating a false positive.

Sincerely, i don't know enough about Chamilo to understand it and to fix it. If you or @ywarnier ask to me about any suggestion, i would say that this form SHOULD NOT to send HTML headers (like <html> and <body> tags !!? If the use of this tasks is necessary to render after the content submitted Chamilo should build/add those tags in real time, not stored in database.

Yannick, i'm developing my own CMS since more than 10 years ago, and i know that this kind of things are really a stone in your shoes. If it help to you, let me say that the software you has built (are building) -Chamilo- is very very great. So i hope my comments here are considered as sincere contributions :smiley:

And as you see, if i have any "good" suggestion i'm here available to help!

lonesomewalker commented 4 years ago

@caos30 think about what you were doing :-( You are actively disabling mod_security which is there on purpose on this server (multi domain hosting). An attacker tries something malicious, and because you disabled this, he might be successfull. Time saved?

Yes, this (Github) is not a helpdesk; if you have questions/issues which are URGENT -> https://chamilo.org/en/providers/ (and compared to other open source solutions, these are really qualified) In general, there is ALWAYS a logfile -> there you have all informations. If not, switch to a better hosting. I promise you, this kind of error is found within max. 15 minutes.

ywarnier commented 4 years ago

Let's not be too harsh. Everyone makes mistakes and I don't see why this contribution wouldn't have been with good intentions, but yes, essentially we need to go deeper on this and find what's making mod-security crazy, but this is something the community will not be able to attend, I'm afraid (too time-consuming for too specific of a use-case).

I'm closing this issue because I think the point is made that Chamilo is not 100% compatible with a default mod-security setup, and that there are ways to circumvent the issue (good ways and fast ways). This will remain as historical information for whom might want to look further into this.

FraGoTe commented 1 year ago

Another modsecurity configuration problem with WHM

[Thu Jan 26 10:42:40.901166 2023] [:error] [pid 14231:tid 47444657301248] [remote 204.199.173.196:52985] [client 204.199.173.196] ModSecurity: Access denied with code 403 (phase 2). Match of "rx ((?:submit(?:\\\\+| )?(request)?(?:\\\\+| )?>+|<<(?:\\\\+| )remove|(?:sign ?in|log ?(?:in|out)|next|modifier|envoyer|add|continue|weiter|account|results|select)?(?:\\\\+| )?>+)$|^< ?\\\\??(?: |\\\\+)?xml|^<samlp|^>> ?$)" against "ARGS:choice[163]" required. [file "/etc/apache2/conf.d/modsec2/10_asl_rules.conf"] [line "1000"] [id "350147"] [rev "140"] [msg "Atomicorp.com UNSUPPORTED DELAYEDRules: Potentially Untrusted Web Content Detected"] [data "325"] [severity "CRITICAL"] [hostname ""] [uri "/main/inc/ajax/exercise.ajax.php"] [unique_id "Y9KfcGxVsq86wewQWFgLWAABlwE"], referer:/main/exercise/exercise_submit.php?cidReq=TEST&id_session=0&gidReq=0&gradebook=0&origin=&exerciseId=19&learnpath_id=&learnpath_item_id=&learnpath_item_view_id=

chamilo / chamilo-lms