Internal Server Error after Installation, when loading manager

[potatoBag]

Summary

I was trying to install the latest version 2.5.7 (also occures with 2.5.6) and run into an 500 Internal Server Error after installation. The window with "go to Manager" shows up, but then after the php_max_execution_timeout the error_log returns "script timed out before header returned". While waiting the load of the CPU is 100%.

Step to reproduce

Download the 2.5.7 version and install it.

When the Message "Thank you for installing MODX Revolution." shows up. Then look into the core/cache/logs/install.config log file.

The error is the following: [2017-05-09 13:47:52] (ERROR @ /var/www/html/modx/modx-2.5.7-pl/setup/includes/tables_create.php : 116) Could not create table modx_access_policies SQL: CREATE TABLE modx_access_policies (id INTEGER unsigned NOT NULL AUTO_INCREMENT, name VARCHAR(255) NOT NULL, description MEDIUMTEXT NULL, parent INT(10) unsigned NOT NULL DEFAULT '0', template INT(10) unsigned NOT NULL DEFAULT '0', class VARCHAR(255) NOT NULL DEFAULT '', data TEXT NULL, lexicon VARCHAR(255) NOT NULL DEFAULT 'permissions', PRIMARY KEY (id), UNIQUE KEY name (name), INDEX parent (parent), INDEX class (class), INDEX template (template)) ENGINE=MyISAM ERROR: Array ( [0] => 42000 [1] => 1071 [2] => Specified key was too long; max key length is 1000 bytes ) ..... same for modx_content_type - modx_context_resource - modx_context_setting - modx_menus - modx_site_plugin_events - modx_documentgroup_names - modx_session - modx_membergroup_names - modx_user_group_roles - modx_workspaces - modx_register_messages - modx_register_queues - modx_transport_packages - modx_transport_providers

install.config.2017-05-09T13.47.52.log.txt

Observed behavior

It is running in an endless loop with high cpu load. The apache error_log returns "script timed out before header sent". An internal server error 500 is thrown after max_execution_time

Expected behavior

It is expected that the Login form of the Manager shows up.

Environment

Server: apache modx: 2.5.6 or 2.5.7 suhosin + cloudlinux enabled (shared hosting) php 7.0.+ cgi (endless loop and 100% cpu load) php 5.5 and 5.6.+ cgi (blank white page, but no cpu load, instant blank) mysql Server: mysql-community-server-5.5.54

[ghost]

The installer does not check to see if the provided db.user account has permission to create the necessary tables, and yet completes the installation with success message and without creating the tables.

Installer needs to check and see if db.user can perform require table creation and to double check that said tables were created.

Not sure if this issue is logged separately, but I found this out the hard way...and realized there are a bunch of (unnecessary, IMO) varying methods to install, all of which involve pain and are error prone (+ permissions hell). Not to mention upgrades.

I would love to see a new installer instead of 3 half-baked ways of installing/upgrading this thing. (i.e., why do we need Standard and Advanced; a special build script just to build an install package?) (yes, I know, it's not how it works and it's probably needing deep cuts to change it)

Especially now that it's unsafe to install the standard way there needs to be flexibility and ease here. I'd be willing to do my part to but would need to know what the plan was overall before committing (time being as valuable as it is).

[OptimusCrime]

This is a known issue. I looked at it previously and there is no easy way to check if a user has permissions to create tables. This is because this information is stored in a special table, and in rare cases the user might lack permissions to even use the SELECT command, which renders this process very complicated and error prune. Not to mention that this functionality needs to be supported for all three database systems that xpdo and modx supports (mssql (?), sqlite and mysql).

Secondly, I see a some issues that states that the columns are too wide. Are you using utf8mb4? Modx is not 100% compatible with this yet (as far as I know), so I suggest using utf8 and utf8_general_ci for now.

Thirdly, I am not sure just what your other statements mean. There are three different packages for Modx. Traditional, advanced and SDK. These needs to be included in the setup process. Additionally, we have different scenarios where users might wish to overwrite the system, upgrade with files in place, or upgrade without the files in place. While this process might be more streamlined, I have never had any problems with them, and I am not sure what your complaint about this is?

Final thoughts: You state it is "unsafe" to install. I disagree. You failed to configure the database correctly with permissions. Modx did not catch the error and presented it gracefully, but like I said, checking and validating this is difficult. If you give your SQL system appropriate permissions, the setup should be swift and easy.

[ghost]

Hi Optimus,

Thanks for your reply..

If you are satisfied with how it all works, we don't have to agree. No problem. I'm just coming back to MODX one more time to reevaluate it as a product and the project overall.

To clarify:

1. I did not mention anything about columns being too wide.

2. Safe = the default basic install should offer installing of MODX "out of web root" as a security measure. Having the controllers and all the manager files in the web root is not cool either.

3. I'm not a noob looking for support-I was testing the product and trying variations in installation in search of the most secure way and to find out how hard it is to achieve. My concerns are the usability of the product and its long term success.

~

OK, what is at issue for me is usability of the install/upgrade process, not a fix.

To me the installer could be vastly improved (single web installer w/advanced features tab, one CL for total customizations), but that is a long conversation. I will not take it up further here.

However, to the detail of: Install fail due to lack of db.user permissions.

Your response is saying that this is a known issue and...? (a) it's not easily fixable (b) or really doesn't need fixing. Maybe a bit of both?

My thoughts, FWIW to your team: The installer should not complete without an error message if the tables are not created. the installer did not hint at anything being wrong, but the SQL errors were in the log file. Tests could be performed to see if the MODX tables can be created, or after, to see if the tables exist.

There are more possibilities I'm sure. The error log shows specific SQL errors in this case, so I think it's a solvable problem.

Best, G

[OptimusCrime]

First of all I have to state that I am not in any way a part of the MODX team. I am a contributer like everyone else that has submitted pull requests to this repository, but I have nothing to do with MODX LLC, so there is no team. The opinions I share here are my own.

Of course I am not satisfied with everything. Everything can be improved and there are many places where MODX can improve a lot. I was just trying to explain that the original issue you posted has already been reported. The most detailed issue on this is actually my own (#13286) posted in February this year. Like I said, I just meant to point this out.

About your three points:

1. In the original post you have copied the following error message [0]. As you can see, this issue concerns the length of the keys and is something that is also reported before in issue #13121 (dup), #13373. This is also something the LLC has put effort into solving in the next version, as seen in PR #13462 although this still needs work, and as far as I've understood, as second PR to fix. The issue with too long key is often seen in context with incompatible collation, which was why I asked about that, and your original post did not state this configuration. It might not be the case here, but I thought it was worth asking.
2. Okey, I misunderstood what you meant here then. I guess that is a fair point, and people have started looking at ways to have a single installer with all the same possibilities. I guess they were split before to keep things easier. Additionally, I think both the controllers and the manager files needs to be accessible from the webroot, as there are requests done directly to these files. If it was rewritten, then there could be a single entry file, but this requires a lot of work, and will most likely not happen right away.
3. Of course, and as a developer I agree. MODX is a great product, but needs to be modernized in several ways in the near future. People are aware of this, and a lot of work and hours is put into this. However, as MODX Revolution is a CMS that is worth is weight mostly because of extras and modules developed externally, we naturally want to keep these compatible, which is why this task becomes very complex.

~

As I stated above, I have already posted a detailed issue (#13286) where I state that this should be fixed (and that it is important). However, as I said in my previous post: it is not easy to implement. This feature would need to be shipped simultaneously for all three database drivers and would need a common interface. I have never used MSSQL and I have no access to any platforms which runs this software. I am therefore unable to do anything concerning this driver. I DO believe that this issue should be fixed, but it should not be half done.

Finally I'd like to chim in that MODX is open sourced and while the LLC and the MODX MAB decides the major stuff, contributors are necessary to get things moving. The entire codebase for MODX Revolution is gigantic. It is simply too much work for a handful of people to handle such amounts of work. That is why I, and a lot of other people, are using our free time to help the project. This is also the reason why some issue are reported and either resolved much later, or never resolved at all. Someone needs to spend hours upon hours to fix these things.

What I am trying to say is that if you have a clear vision of how the setup should work, and you have some free time, join the project. I'll admit that I have not contributed much lately, as I've been busy finishing my masters degree in artificial intelligence, but I'll be sure to spend some hours when I find the time :)

[0]:

(
[0] => 42000
[1] => 1071
[2] => Specified key was too long; max key length is 1000 bytes
)

[Mark-H]

Regarding the columns too wide - that's the error mentioned in the issue. Perhaps you meant to open a new issue @gharford, but commenting here suggested a relation, which does not appear to exist apparently.

There's an issue #13286 related to the setup not clearly showing error messages when the user lacks permissions. Opened by @OptimusCrime, funny enough. Perhaps you have additional thoughts worth sharing there, otherwise I'd suggest we keep this issue related to the columns being too wide message from the opening comment :)

12629 relates to consolidating traditional/advanced into a single setup too.

EDIT: Nice timing, @OptimusCrime posted a couple seconds earlier :P

[OptimusCrime]

@Mark-H Beat you by less than a minute :)

[ghost]

@Mark-H : My bad! ;) I was just tossing it in the likeliest visible bucket while I had a second FWIW. Thanks for the refs but I'd be pressed where to post. A lot of these items are interrelated but it's not obvious. i.e., #13149 is the break, whereas #13286 is related, but as a suggestion to fix the error message. Can we not check for SQL errors? Test the db.account by trying to create the first table and abort if unsuccessful? Check after install that all the tables were created? Oh, my bad again! LOL Cheers. (and thanks for all you bring)

@OptimusCrime (just following on but its fair to close the thread) Yes, I am aware of this being an open project, and please don't assume I am expecting someone else to fix my problem. Oh no. ;) If you have worked late into the evening on some obscure problems then we are in the same boat--even end users of MODX need good technical troubleshooting skills at times.

I think everyone's time is valuable but as time moves on, it's important to decide where we spend it and on what. I don't think it's valuable for me to design a new solution...(to explain)...

I have some serious history with the project and the product, having spent many (did I say many?) hours under the hood as a user of this product to make it work for clients (since before Revo). My issue today was part of an attempt to get the installer to make the safest, securest install possible as part of my own vision of what I want to bring to new clients. I'm revisiting MODX with more stringent requirements for clients, you might say.

I know it's taken a ton of people just to get the product to where it is. Same time, however, when it comes to important features, such as the installer, or the manager, beyond break-fixes there is a limit to the ideas that will be implemented. I'm not the only long timer who says this, please trust me here.

It's rare for any deep/core features to be done outside the team because a lot of things touch the design/architecture, and that architecture has one owner. And maybe that is what makes MODX as great as it is.

Over the long term, I find feedback to suggestions is always the same, and in the same progressive order (you gave me one and two):

take 1: Hey it works, sorta. But we're working on it. Here's what to do in the meantime. take 2: This is an open project. If you have good ideas, get involved. take 3: Great idea. (Oh, wait, this affects the core). You'll have to talk to Jason.

Adding to the core, or fixing what's broke is open. But there is a vision for this product, and it's not a shared one, otherwise 3.x would be up and happening.

I also think that there is plenty of fixes but possibly no new innovation going into v 2.x since 3.x has been waiting for a while now.

My $0.02. I appreciate your input and thoughts, thanks.

[OptimusCrime]

I think this problem is solved with PR #13559?

[OptimusCrime]

@modxbot close