maths / moodle-qtype_stack

Stack question type for Moodle
GNU General Public License v3.0
140 stars 148 forks source link

Maxima for Windows builds do not work with Stack out of the box. #332

Closed sgparry closed 6 months ago

sgparry commented 6 years ago

Currently, it is difficult to get a working Windows build and no explicit option for optimised Windows exists, even though you can optimise recent Windows builds and some do work with Stack if it is forced to.

sangwinc commented 6 years ago

Did you see this very recent branch? https://github.com/maths/moodle-qtype_stack/tree/maxima5-40-0 I checked this and added it to master today.

We still have the Windows issue to fix.

sgparry commented 6 years ago

Sorry Chris - It's hard to keep up!

sangwinc commented 6 years ago

Thanks to the very helpful folks in the Edinburgh University maths computer support team, I've got my Windows 7 laptop back from the coffee incident of a couple of weeks ago without too much damage!

Firstly, for the record, I've stopped using the Windows version of STACK for dev/demo. I now use VirtualBox, so that my dev set up mirrors our live server. At this point I highly recommend VirtualBox for anyone who actually uses linux. Some people won't want to do this, and for them I will try to keep the "win" version running for now.

As I suspected, a lot has changed on the Maxima/Gnuplot side in recent releases. This includes (1) where Maxima ends on on the hard drive and (2) how to call it. It wasn't difficult for me to get this working again out of the box with the latest Windows release of Maxima 5.41.0.

Over the years, almost every Maxima release has required some fiddling around with, so please expect some ongoing maintenance need. While people actually use this seriously I'll do my best to support it.

Please let me know how you get on with this branch. I have tested this on linux, (without running all the unit tests). I'll merge this all back into master in due course.

mocsap commented 6 years ago

Is there any stable working STACK and Maxima combination, which will work in Moodle/Windows environment?

I'm struggling to configure the latest STACK release 4.1 with Maxima versions 5.36.1, 5.38.1, 5.40.1.

I'm constantly getting an empty string during the healthcheck in Moodle with a CAS result: unpack_raw_result: no timestamp returned. Data returned was: ]]]]

Another question will be, whether is it reasonable to use STACK&Maxima in a production environment under Windows?

Maybe it would be possible to run Maxima on a Linux-based server and send the http-requests from a Windows based Moodle-server with STACK-Plugin installed? If so, I will appreciate for any suggestions how to configure it.

sangwinc commented 6 years ago

I'm very sorry about the recent lack of support for Windows. As I mentioned above, this is due to hardware problems at my end. I've checked these changes on my linux dev machine, and they don't have any ill effects so they have been merged back into the master branch. If you have the code in master, I'd expect the latest STACK release 4.1 with Maxima versions 5.36.1, 5.38.1, 5.40.1 to work. It does depend where on your machine these are installed, and that keeps changing.

I have no expertise about server support for production servers on Windows. The optimization, and the use of the "timeout" command on linux are important. We don't currently have support for optimization on Windows, and I don't know the equivalent of "timeout", so I can't comment on this. I suspect this is not the best solution.

Your suggestion to run Maxima on a Linux-based server and send the http-requests from a Windows based Moodle-server with STACK-Plugin installed is already supported. The "server" you need is here: https://github.com/maths/stack_util_maximapool

I've done this on my home network and it works well on my windows machine. At one stage I used a raspberry pi 3 for the Maxima end, obviously not suitable for production, but it worked well for testing!

As I hope you understand, as an open source project I can't support all configurations, but we will do our best to help.

sgparry commented 6 years ago

@sangwinc I have made a number of fixes relating to the windows build which I will upload soon.

  1. It now works without manually setting a path.
  2. Graphs are working.
  3. You can select which lisp to use.
  4. It allows you to build and use an optimized executable for SBCL (working) and CLISP (some DLL issues).

The fixes are still a little experimental because, although I have tested with paths with spaces in and have had no problems, I am not sure exactly where you had issues previously to check. Fortunately recent builds of Maxima all default to 'off-root' installations.

sangwinc commented 6 years ago

Thank you, much appreciated. Please send me a pull request and I'll happily take a look! Happy New Year all! Chris

sgparry commented 6 years ago

Apologies for the delay. I did not like the state of my code and a small fix has led to some more extensive refactoring. I should be done in a couple more days!

mocsap commented 6 years ago

@sgparry Have you been able to make the necessary changes? It will be great to have a Windows compatible and functionable version. Many thanks for your efforts!

sgparry commented 6 years ago

I am very close and working on it this weekend. The code works but I am still refactoring / commenting / shaking out some issues. I need to retest under linux too in case I have broken anything there. I should be done by Monday.

On 19 January 2018 15:16:54 GMT+00:00, mocsap notifications@github.com wrote:

@sgparry Have you been able to make the necessary changes? It will be great to have a Windows compatible and functionable version. Many thanks for your efforts!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-358994439

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

sangwinc commented 6 years ago

Thanks folks, your help is much appreciated.

sgparry commented 6 years ago

OK, I have finally posted what I have. I have not had a chance to test it on Linux yet - if someone could save me the aggro, I would be deeply grateful. Caveat: you will need to review your command settings carefully, as this update relies on you not setting a value for maxima / plot command unless absolutely necessary.

sgparry commented 6 years ago

Pushed some additional edits to fix the install process...

sangwinc commented 6 years ago

Thanks for pushing this, I'll certainly have a look both on Windows and Linux.

mocsap commented 6 years ago

Thank you very much for your time and your efforts! I will also take a look whether it will work on our Windows server. Great job! Thanks!

sgparry commented 6 years ago

Beware, there is a problem still in there with stack_utils::refresh_config(). I have fixed locally but I cannot get it pushed. Should be there in a couple of hours.

On 1 February 2018 13:22:39 GMT+00:00, mocsap notifications@github.com wrote:

Thank you very much for your time and your efforts! I will also take a look whether it will work on our Windows server. Great job! Thanks!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-362264171

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

sangwinc commented 6 years ago

Thanks,

I've been trying out your code on my linux machine. I get the error

"Debug info: Argument 1 passed to stack_platform_base::__construct() must be an instance of string, string given, called in [dirroot]/question/type/stack/stack/cas/platform.unix.class.php on line 68 and defined Error code: codingerror"

Is this related to your stack_utils::refresh_config() problem?

Chris

On Thu, Feb 1, 2018 at 3:36 PM, sgparry notifications@github.com wrote:

Beware, there is a problem still in there with stack_utils::refresh_config(). I have fixed locally but I cannot get it pushed. Should be there in a couple of hours.

On 1 February 2018 13:22:39 GMT+00:00, mocsap notifications@github.com wrote:

Thank you very much for your time and your efforts! I will also take a look whether it will work on our Windows server. Great job! Thanks!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332# issuecomment-362264171

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-362303385, or mute the thread https://github.com/notifications/unsubscribe-auth/AAvtL17E1YMCqNhcJKVGWG7aJezs0qlmks5tQdnxgaJpZM4Q_lR7 .

sgparry commented 6 years ago

The stack_utils::refresh_config() is fixed now. @sangwinc No, that's new, and very bizarre. Your server is basically complaining that a string was passed when a string was expected ???? Can you get a deeper stack trace out of it please?

sangwinc commented 6 years ago

Ok, here is the trace!

Debug info: Argument 1 passed to stack_platform_base::__construct() must be an instance of string, string given, called in [dirroot]/question/type/stack/stack/cas/platform.unix.class.php on line 68 and defined Error code: codingerror Stack trace:

On Thu, Feb 1, 2018 at 5:15 PM, sgparry notifications@github.com wrote:

The stack_utils::refresh_config() is fixed now. @sangwinc https://github.com/sangwinc No, that's new, and very bizarre. Your server is basically complaining that a string was passed when a string was expected ???? Can you get a deeper stack trace out of it please?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-362337039, or mute the thread https://github.com/notifications/unsubscribe-auth/AAvtL6RXzeLAuJcxFQTIFDgAOoZmsS-5ks5tQfE4gaJpZM4Q_lR7 .

sgparry commented 6 years ago

Which version of PHP are you using @sangwinc ? I am currently setting up my local Linux box to host, but this has me baffled.

sgparry commented 6 years ago

OK I have fixed that issue. It was a PHP5 v PHP7 issue - apparently you are not allowed to type hint with scalars in PHP 5, even if you want to - but PHP 7 lets you. I hate weakly typed and weakly declared languages :-(

sangwinc commented 6 years ago

Yes, this was PHP 5. I should have thought of that...

Did you push your changes? I can't see them on https://github.com/sgparry/moodle-qtype_stack

Chris

On Fri, Feb 2, 2018 at 1:17 AM, sgparry notifications@github.com wrote:

OK I have fixed that issue. It was a PHP5 v PHP7 issue - apparently you are not allowed to type hint with scalars in PHP 5, even if you want to - but PHP 7 lets you. I hate weakly typed and weakly declared languages :-(

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-362455475, or mute the thread https://github.com/notifications/unsubscribe-auth/AAvtL8EyICFSOrTIj-oraUzq6VqgOGTvks5tQmIcgaJpZM4Q_lR7 .

sgparry commented 6 years ago

I have, I just overwrote the original commit - I tend to do that otherwise you'll get dozens from me. I have the code starting to work on my main Linux server but it is struggling to find Maxima currently - the new code checks for Maxima being present.

aharjula commented 6 years ago

I'll try to look at this next week but quick read through did not spot anything major. Will test couple of Linux distros and FreeBSD as well as the Pool. Some of the refactoring may provide better anchor points for the locale logic tested in the encoding branch.

However, that last comment scares me. Please, never overwrite commits that you have already pushed as we may actually have built things over them and they may be living in massive distributed and cached systems like github that assume that history does not change...

sgparry commented 6 years ago

Pushed fixes for the "cannot find Maxima on system path" problem. All that seems to be outstanding is getting the travis tests working with the new configuration settings. @sangwinc is that something you might be able to help with please?

sangwinc commented 6 years ago

I'm really sorry to be slow, but where did you push the fix to?

I can't see it here: https://github.com/sgparry/moodle-qtype_stack/branches

The pull request https://github.com/maths/moodle-qtype_stack/pulls is unchanged for 2 days.

Chris

On Fri, Feb 2, 2018 at 11:15 AM, sgparry notifications@github.com wrote:

Pushed fixes for the "cannot find Maxima on system path" problem. All that seems to be outstanding is getting the travis tests working with the new configuration settings. @sangwinc https://github.com/sangwinc is that something you might be able to help with please?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-362559383, or mute the thread https://github.com/notifications/unsubscribe-auth/AAvtL-8IpPmiVYTtrHXzOuDahrXJWWHVks5tQu5bgaJpZM4Q_lR7 .

aharjula commented 6 years ago

Could be that if he overwrites the commits the old commits with the same id are the ones that are being shown as they are cached... and even if he renames the commits the pull request will still point to the old commit id... It is a very bad idea to modify a commit that has already been pushed to other repositories. The only reasonable way to clear that mess will probably be to build a new branch and manually reconstruct the commits as new commits without any references to the old and already corrupted ones.

In any case Chris expect a JSXGraph branch in a couple of hours once I can finish my example question.

sgparry commented 6 years ago

OK, Sorry guys. I've not had this problem before. The changes are in there - I checked a couple of lines of the actual source and the source browser shows changes dating from a couple of hours ago. I will create a new branch iss332a and copy in the updated sources into that and create a fresh pull request, if that is acceptable.

sgparry commented 6 years ago

Have created fresh branch and pull request. It contains all fixes to date.

sangwinc commented 6 years ago

Thank you very much for doing this, I thought I was the only one having problems with the complexities of GIT!

I very much like the new "Maxima Pre-Optimisation Command" and etc. I've been meaning to do this for a while, and it is a big improvement. Thanks.

I've added in a "default" version to the settings. This is, I think, an omission? Easy to fix.

I'm getting a regular warning:
Warning: preg_match(): No ending delimiter '^' found in /var/www/moodle33/question/type/stack/stack/cas/platform.class.php on line 373

This creates lots of warnings etc. I've modified this, so could you check you agree this is now warning free & correct?

More seriously, when I use the cache I'm getting the following fatal error:

Fatal error: Call to undefined method stack_cas_connection_db_cache::get_maxima_available() in /var/www/moodle33/question/type/stack/stack/cas/platform.class.php on line 360

To remove this error I've added a trivial method to stack_cas_connection_db_cache, which can't possibly be the correct thing to do! I'm not quite sure what you've changed here (and it is late on a Friday). Can you see what needs to be done? Otherwise, I'll try to pick this up later.

I'll certainly run all the unit tests, and I'll also like to see how we get on upgrading existing servers. I think some problems were related to the undefined method stack_cas_connection_db_cache::get_maxima_available(), but I'd really like to check this out on some other test servers in various states before merging back into master to make sure we get a smooth upgrade process.

Also, I'd be very grateful if Matti could checkout he is happy with the "server" modes, which I don't have easy access to on a test server.

According to the Moodle style guide "extract" is forbidden. Is there an alternative?

I've sepateted the above changes from a large "code tidy" commit. Sorry if this looks petty snd picky, but Moodle has a code style and I've used the auto code checker tool to implement this style. I didn't want to bother you with these minor things, but I want to get the code tidy done early so we can ensure I haven't introduced any mistakes in the process.

I'm very grateful for your work. Thanks. Chris

sangwinc commented 6 years ago

Can I also ask if the "=" on this line should be "=" or "=="? https://github.com/maths/moodle-qtype_stack/blob/iss332a/stack/cas/platform.unix.class.php#L309

sgparry commented 6 years ago

code tidy is good - I wanted to run it on my code too but couldn't work out how. default lisp setting will need some careful consideration - the lisp setting affects for example the default command line for the auto-optimised image, It is not always possible for the code to check Maxima to find out (catch 22). regexp looks right. Not sure where my brain was with that. Re: get_maxima_available(), no there was an intermediary call to get_raw() missing; that function allows the raw underlying stack_connection_base object to be fetched from the cache objects. Extract was supposed to be history, but I missed one.

I have pushed the fixes, hopefully you can see them ok.

sangwinc commented 6 years ago

Thanks very much. As you can see from here https://github.com/maths/moodle-qtype_stack/tree/iss332a I've been going a fair bit of testing etc. I've also done some more code tidying. Sorry, the Moodle style is quite strict and (whatever my personal views on some of their choices) I'd like to try to keep the whole codebase within this style if possible. I'm not asking you to make all these changes, but I hope you don't mind!

On Linux things have gone very well indeed. I've even got to the stage of deliberately trying to break things by choosing "Windows" to see what would happen. It broke, of course, but in a sensible sort of way I think. No blank screens of death, or fatal PHP errors. Things were caught within Moodle.

On Windows I've used an old install, and installed from scratch.

  1. I had to create the [dataroot]\stack directory by hand. It is unlikely a novice user will know what to do here!

Otherwise, once done windows normal seems fine.

I can't get the optimised image to work on Windows. I'm using the default, out of the box, Moodle install.

  1. Create Optimised Maxima Image FAILED : Evaluation test failed, errors:CAS failed to return any evaluated expressions. Please check your connection with the CAS.

I got a "missing DLL" error, as shown in the attached image I'll email you off list. Any suggestions?

All very encouraging really. Thanks!

Chris

sgparry commented 6 years ago

Which did you pick? SBCL should not need any DLLs; For CLisp the DLLs should be copied into place by Stack for you. I am fixing the lisp checking just now - there's an issue with it (I missed that fix last night).

sgparry commented 6 years ago

OK, done a lot of reworking and retesting on Linux. It now seems to operate much more smoothly. There is longstanding gremlin hiding in there somewhere regarding switching to/from optimised set-ups - sometimes the settings screen lies as to which is the currently selected platform. I can't reliably reproduce it though to debug it. I have pushed those changes and I am now going to head back to Windows to retest try and reproduce the issues you described.

sgparry commented 6 years ago

@sangwinc Chris, I can reproduce the directory bug, but I cannot pinpoint where in the code the directory gets created on Linux. Any ideas please?

sgparry commented 6 years ago

The code I have just pushed is about as far along as I can get it with just my testing, so everyone please feel free to give it another good kick.

sgparry commented 6 years ago

Things seem to have gone very quiet. Any feedback guys?

aharjula commented 6 years ago

Sorry, my priority queue had some tasks inserted in front of testing this. Hopefully, next week. This has quite high priority in that queue so it will happen it just could not get done this week like I imagined.

sgparry commented 6 years ago

No problem, I really appreciate it.

On 16 February 2018 07:09:24 GMT+00:00, Matti Harjula notifications@github.com wrote:

Sorry, my priority queue had some tasks inserted in front of testing this. Hopefully, next week. This has quite high priority in that queue so it will happen it just could not get done this week like I imagined.

-- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-366160082

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

sangwinc commented 6 years ago

Sorry to have gone a little quiet, this is a priority to look at again this weekend! I'd really like to get this code into the main base.

sangwinc commented 6 years ago

First let me thank you for all your work on this code. It is more complicated, but is much better engineered. I really like the new warnings/error system and the navigation is much better.

I've tested this on Linux and Windows, and I have no further comments. I've pushed a code tidy today, and pending any further comments from Matti and Tim I think we should merge this back into master and close this issue.

I'm confident we will need to maintain this, as over the years the install has been an ongoing maintenance issue, as Maxima changes subtly etc.

sgparry commented 6 years ago

Thanks Chris. I will be really pleased when this gets merged in as its been holding me back from base-n and so on for a while. It was a two day 'might-as-well-fix-this-I'm-at-it' that turned into two and half month monster! Here are some of my on hold ideas - let me know what you think of them.

  1. Base-N (obviously), including multi-line and matrix support.
  2. Some useful support functions for base-N, e.g. explode digits (expands a base-n number into a matrix of digits), add with carry (takes two exploded numbers and adds or subtracts them, producing a sum matrux and a carry matrix) - these are useful for worked solutions.
  3. Clone question type. Basically links to an existing STACK question, using a configured subset of the deployed question/answers of the 'master'. Should allow banks of well tested quality drill questions to be constructed quickly and used in quizzes.
  4. Reusable user libraries. These would be enter using stack, allowing the usual stack sanity checking, but loadable into multiple questions and optionally compiled into the main image.
  5. An urgent requirement is getting the russian contrib logic module to work well with Stack, so I can set boolean algebra problems.
  6. Karnaugh maps - related to 5 and equally urgent.

On 17 February 2018 16:17:10 GMT+00:00, Chris Sangwin notifications@github.com wrote:

First let me thank you for all your work on this code. It is more complicated, but is much better engineered. I really like the new warnings/error system and the navigation is much better.

I've tested this on Linux and Windows, and I have no further comments. I've pushed a code tidy today, and pending any further comments from Matti and Tim I think we should merge this back into master and close this issue.

I'm confident we will need to maintain this, as over the years the install has been an ongoing maintenance issue, as Maxima changes subtly etc.

-- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-366451825

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

sangwinc commented 6 years ago

Thanks very much indeed for your continued interest.

  1. & 2. would be great. We can add in details of the design etc. in this issue: https://github.com/maths/moodle-qtype_stack/issues/317

  2. I'm working on support for a YAML import/export etc. format in the api_v4 branch. I think this could be a useful way to go here, with a "cascading" system of question overlays. E.g. we have a reliable question for X, but we adjust the question variables to constrain the randomization. More thought needed, but could be very effective.

  3. Lots of people have asked for this. At what level would this be? It is related to this issue: https://github.com/maths/moodle-qtype_stack/issues/306 To be honest, I'd like people to contribute their packages back to the core, as you are generously doing with the base N contribution. If people write their own libraries we might end up in a real mess. Let's continue to discuss this as I'm really keen to get people on board, and make it easy to contribute. I'm very open to suggestions for a mechanism.

  4. & 6. I think this is a new one on me?! My colleague George opened this issue recently: https://github.com/maths/moodle-qtype_stack/issues/348 as a request. I'm not familiar with this package (sorry!) but I do need better support for logic myself in the "equivalence reasoning" stuff I'm working away at (when I don't waste hours over a == vs === as in issue #350 !).

Thanks for your continued help.

sangwinc commented 6 years ago

More testing on a variety of Moodle platforms. On Moodle 3.0.6+ (Build: 20160923) I'm getting the following error:

Fatal error: Undefined class constant 'NOTIFY_ERROR' in /var/www/html/demo/question/type/stack/healthcheck.php on line 86

Any idea why this is being thrown? Is it related to this?

https://tracker.moodle.org/browse/MDL-53786

sgparry commented 6 years ago

News to me - I am testing on 3.1 and do not see this. Do we need this 3.0.x support?

On 19 February 2018 09:07:27 GMT+00:00, Chris Sangwin notifications@github.com wrote:

More testing on a variety of Moodle platforms. On Moodle 3.0.6+ (Build: 20160923) I'm getting the following error:

Fatal error: Undefined class constant 'NOTIFY_ERROR' in /var/www/html/demo/question/type/stack/healthcheck.php on line 86

Any idea why this is being thrown? Is it related to this?

https://tracker.moodle.org/browse/MDL-53786

-- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-366627412

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

sangwinc commented 6 years ago

Good question, do we need 3.0.x support? Well, I'm glad I tested my old machine so we know we have to make a choice...

We're using Moodle 3.2.2 (Build: 20170313) in anger. Personally, I could live without it, but I suspect many other people will need us to continue to support this for a little while longer.

sangwinc commented 6 years ago

That all said, I do note this: https://docs.moodle.org/dev/Releases

Comments Tim & Matti?

aharjula commented 6 years ago

Well, as that says support ended we might as well drop it too if it causes trouble, maybe note somewhere that STACK version 20180XYY is the last to support Moodle 3.0, but otherwise as long as we support those that are supported there, we are meeting the needs of our user base as our user base obviously follows that support schedule, at least those with commercial Moodles...

Just remember to upgrade the version.php to match our new requirements.

sgparry commented 6 years ago

Frustrates me a lot that the moodle api is not stable enough from release to release to allow plugins to stay functional by default but the release schedule means that as a developer you are burdened a lot in keeping up. About a quarter of the plugins I use are not supported on 3.3, and 3.4 is iminent. PHP does not help, because you cannot even get static analysis / compile out of the language, which makes checking an upgrade 10 times as difficult. What is the fetish nowadays with quick-and-dirty dynamic weakly everything languages? I'd rather have my code thoroughly compiled, type-checked etc. before it gets anywhere near a user.

On 19 February 2018 09:36:19 GMT+00:00, Chris Sangwin notifications@github.com wrote:

That all said, I do note this: https://docs.moodle.org/dev/Releases

Comments Tim & Matti?

-- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: https://github.com/maths/moodle-qtype_stack/issues/332#issuecomment-366634483

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.