Modular form error - Githubissues

jwj61 commented 8 years ago

From Brian via the feedback page:

"From the page ""Newforms of weight 9 for Gamma_1(23)"" http://www.lmfdb.org/ModularForm/GL2/Q/holomorphic/23/9/?group=1

if I click on S9(\chi{23}(22,\cdot))

I get the error ""The page you are looking for has produced a server error."""

When I tried this, I didn't see that I could click on S_9(...), but clicked on the first form 23.9.22.a and got an error.

sanni85 commented 8 years ago

On the same page I tried to click on a grey link (/23/9/2/) and I got a page which says: "Unfortunately, we do not have this space in the database, yet." but below the table is written: "Spaces in grey are trivial because the character and the weight have opposite parity."

AndrewVSutherland commented 8 years ago

The problem is http://www.lmfdb.org/ModularForm/GL2/Q/holomorphic/23/9/22/a/. It is crashing while trying to display an error message, looking at it now.

JohnCremona commented 8 years ago

At ModularForm/GL2/Q/holomorphic/23/9/22/ form a is displayed but in the Dimension column it has 0. Should be 1.

AndrewVSutherland commented 8 years ago

It's crashing because max_cn is not defined in line 12 of emf_web_newform_post_load_scripts.html where it tries to compute max_cn-1 while formatting an error message. If you fix this it then says that the space is empty, which it obviously is not (like JC said, the dimension should be 1 not 0)

JohnCremona commented 8 years ago

Where is it getting that dimension from? @fredstro or @sehlen ?

AndrewVSutherland commented 8 years ago

@JohnCremona Not from the database, there is only one record in modularforms2.webnewforms with parent 23.9.22, and it has hecke_orbit_label 23.9.22.'b', there is no record with hecke_orbit_label label '23.9.22.a'.

It also seems strange that the 15-dimensional Galois orbit S_9^new(Gamma_0(23),chi_23(22,.)) is decomposing as 0+12. Even if it were 1+12 there would still be something wrong here.

sehlen commented 8 years ago

On May 7, 2016, at 15:32, AndrewVSutherland notifications@github.com wrote:

@JohnCremona https://github.com/JohnCremona Not from the database, there is only one record in modularform2.webnewforms with parent 23.9.22, and it has hecke_orbit_label 23.9.22.'b', there is no record with hecke_orbit_label label '23.9.22.a'.

That’s true and then it’s not. The dimension is shown as 0 because there is no db record and it uses the default value. However, there is a gridfs file and that’s where it gets the information from that it’s showing… However, there are a lot of things wrong with this record apparently… I can try to fix the record and if I fail @fredstro could probably do it on Monday.

What I can do now and probably should be done is make the page display an error message because the data is obviously not valid. I think it’s always better to show an error message (we don’t have this space in the db or so) than showing wrong data.

It also seems strange that the 15-dimensional Galois orbit S_9^new(Gamma_0(23),chi_23(22,.)) is decomposing as 0+12. Even if it were 1+12 there would still be something wrong here.

Yes, the number field shown is also wrong. The form does NOT have rational coefficients in fact (it just seems to happen that the first non-rational one is at q^13, which seems funny, I’ll try to check it).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217635071

AndrewVSutherland commented 8 years ago

I agree that displaying an error message is better than displaying wrong data. But ideally that error message should be meaningful, not "The page you are looking for has produced a server error."

I am very nervous about making changes (to either the code or the data) on Monday, the day before the launch. We really need things to be stable before then (even if that means hiding things that don't work).

sehlen commented 8 years ago

On May 7, 2016, at 16:31, AndrewVSutherland notifications@github.com wrote:

I agree that displaying an error message is better than displaying wrong data. But ideally that error message should be meaningful, not "The page you are lookingfor has produced a server error."

Working on it. I am very nervous about making changes (to either the code or the data) on Monday, the day before the launch. We really need things to be stable before then (even if that means hiding things that don't work).

I agree. However, I personally won’t be able to do much before Monday (or Sunday evening after I arrived in California) and if I understood Fredrik correctly, the same is true for him. So, we can either leave things as they are or someone else comes up with a fix or take the risk on Monday. Sorry, but that’s just how it is.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217641299

AndrewVSutherland commented 8 years ago

@sehlen If we can get a reasonable error message up, I'd be happy to downgrade this from critical and not try to fix the underlying problem before May 10.

sehlen commented 8 years ago

What about this: If the db record does not exist, then the page would show this (note that the label is not clickable in this case) (I don't really get why github is not showing the image but you can click the link...)

On May 7, 2016, at 16:34, Stephan Ehlen stephan.j.ehlen@gmail.com wrote:

On May 7, 2016, at 16:31, AndrewVSutherland <notifications@github.com mailto:notifications@github.com> wrote:

I agree that displaying an error message is better than displaying wrong data. But ideally that error message should be meaningful, not "The page you are lookingfor has produced a server error."

Working on it. I am very nervous about making changes (to either the code or the data) on Monday, the day before the launch. We really need things to be stable before then (even if that means hiding things that don't work).

I agree. However, I personally won’t be able to do much before Monday (or Sunday evening after I arrived in California) and if I understood Fredrik correctly, the same is true for him. So, we can either leave things as they are or someone else comes up with a fix or take the risk on Monday. Sorry, but that’s just how it is.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217641299

AndrewVSutherland commented 8 years ago

Sounds good to me.

AndrewVSutherland commented 8 years ago

FYI, I have a test script running now that is checking the web page for every hecke_orbit_label it can find in modularforms2.webnewforms.

sehlen commented 8 years ago

What does it exactly check? Because the issue here is that the db record does NOT exist so would it be checked by your script?

AndrewVSutherland commented 8 years ago

It checks that each page displays without an error and looks for obviously missing labels (e.g. x.y.z.b is present but x.y.z.a is not). It won't catch cases where the last label is missing.

AndrewVSutherland commented 8 years ago

Below is a list of hecke orbit labels that appear to be missing. Not all of these indicate a critical issue, because in some cases the space in question is not reachable (e.g. form www.lmfdb.org/ModularForm/GL2/Q/holomorphic/90/3/ there is no way to get to www.lmfdb.org/ModularForm/GL2/Q/holomorphic/90/3/37, let alone www.lmfdb.org/ModularForm/GL2/Q/holomorphic/90/3/37/b).

But in many cases one can navigate to an internal server error, e.g www.lmfdb.org/ModularForm/GL2/Q/holomorphic/11/11 --> www.lmfdb.org/ModularForm/GL2/Q/holomorphic/11/11/10 --> www.lmfdb.org/ModularForm/GL2/Q/holomorphic/11/11/10/a.

But all of these missing labels likely indicate some sort of problem on the backend, so I will give the complete list and then figure out which are reachable.

11.11.10.a 11.13.10.a 11.15.10.a 11.17.10.a 11.19.10.a 11.7.10.a 11.9.10.a 12.11.5.a 19.3.18.a 20.11.19.c 20.15.19.c 20.19.19.c 20.7.19.c 23.11.22.a 23.7.22.b 23.9.22.a 24.3.5.a 25.10.24.a 25.12.24.a 27.7.26.a 3.25.2.a 3.31.2.a 3.37.2.a 3.43.2.a 3.49.2.a 3.55.2.a 3.61.2.a 3.67.2.a 3.73.2.a 3.79.2.a 3.85.2.a 3.91.2.a 3.97.2.a 3.98.1.a 30.10.19.a 30.12.19.a 30.6.19.a 30.8.19.a 31.3.30.a 31.5.30.a 45.4.19.a 48.10.47.a 48.11.17.b 48.11.31.a 48.12.47.a 48.4.47.a 48.6.47.c 48.8.47.a 48.9.31.b 49.12.1.c 49.8.1.d 50.4.49.a 7.13.6.a 7.15.6.a 7.17.6.a 7.19.6.a 90.3.37.b

sehlen commented 8 years ago

On May 7, 2016, at 17:31, AndrewVSutherland notifications@github.com wrote:

Below is a list of hecke orbit labels that appear to be missing. Not all of these indicate a critical issue, because in some cases the space in question is not reachable (e.g. form www.lmfdb/org/ModularForm/GL2/Q/holomorphic/90/3/ http://www.lmfdb/org/ModularForm/GL2/Q/holomorphic/90/3/ there is no way to get to ModularForm/GL2/Q/holomorphic/90/3/37, let alone ModularForm/GL2/Q/holomorphic/90/3/37/b).

But in many cases one can navigate to an internal server error, e.g ModularForm/GL2/Q/holomorphic/11/11 --> ModularForm/GL2/Q/holomorphic/11/11/10 --> ModularForm/GL2/Q/holomorphic/11/11/10/a.

Not with my latest fix, which exactly avoids this by hiding the link and showing an error (at least for me, you seem to have obtained different results which I don’t really understand).

But all of these missing labels likely indicate some sort of problem on the backend, so I will give the complete list and then figure out which are reachable.

It is a data problem. The db records are simply missing for all cases that I checked so far.

11.11.10.a 11.13.10.a 11.15.10.a 11.17.10.a 11.19.10.a 11.7.10.a 11.9.10.a 12.11.5.a 19.3.18.a 20.11.19.c 20.15.19.c 20.19.19.c 20.7.19.c 23.11.22.a 23.7.22.b 23.9.22.a 24.3.5.a 25.10.24.a 25.12.24.a 27.7.26.a 3.25.2.a 3.31.2.a 3.37.2.a 3.43.2.a 3.49.2.a 3.55.2.a 3.61.2.a 3.67.2.a 3.73.2.a 3.79.2.a 3.85.2.a 3.91.2.a 3.97.2.a 3.98.1.a 30.10.19.a 30.12.19.a 30.6.19.a 30.8.19.a 31.3.30.a 31.5.30.a 45.4.19.a 48.10.47.a 48.11.17.b 48.11.31.a 48.12.47.a 48.4.47.a 48.6.47.c 48.8.47.a 48.9.31.b 49.12.1.c 49.8.1.d 50.4.49.a 7.13.6.a 7.15.6.a 7.17.6.a 7.19.6.a 90.3.37.b

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217644655

AndrewVSutherland commented 8 years ago

Which link does your version hide, the link to 11/11/10 or the link to 11/11/10/a? When I try to navigate to http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/ the debug logger shows the error:

CRITICAL:emf@2016-05-07 11:40:26,496: record with key:{'version': 1.3, 'modulus': 11, 'number': 10} was not found! [web_object.py:632]

Are you sure you are not running on your master branch (which I notice is 13 commits ahead)? I'm running your branch missing_db_record, which only changes the file display-list-newforms.html.

sehlen commented 8 years ago

On May 7, 2016, at 17:43, AndrewVSutherland notifications@github.com wrote:

Which link does your version hide, the link to 11/11/10 or the link to 11/11/10/a?

Of course to 11/11/10/a on the page for 11/11/10.

Then I try to navigate to http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/ http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/ the debug logger shows the error:

CRITICAL:emf@2016-05-07 11:40:26,496: record with key:{'version': 1.3, 'modulus': 11, 'number': 10} was not found! [web_object.py:632]

Yes, it shows that because the record does not exist. But the page loads fine for me and does what it should for me. Are you sure you are not running on your master branch (which I notice is 13 commits ahead?) I'm running your branch missing_db_record, which only changes the file display-list-newforms.html.

Yes, I’m on the same branch and I indeed only changed this one file. I know that my master branch is ahead and that seems to stem from the December workshop on classical mf’s and I’m trying to sort out in parallel what and why things haven’t made it to the lmfdb master. But I started the fix on top of lmfdb/master on purpose to not pull in unrelated changes!

Note that I just updated the pull request which improves the fix quite a bit.

However, I don’t understand how you could get anything different (with the old version of this branch as well as the new one).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217645951

AndrewVSutherland commented 8 years ago

It's possible there is something inconsistent in what I pulled. Let me try to reset my git configuration. In the meantime I will ask someone else to review pull request https://github.com/LMFDB/lmfdb/pull/1219 as well.

On 2016-05-07 11:55, Stephan Ehlen wrote:

On May 7, 2016, at 17:43, AndrewVSutherland notifications@github.com wrote:

Which link does your version hide, the link to 11/11/10 or the link to 11/11/10/a?

Of course to 11/11/10/a on the page for 11/11/10.

Then I try to navigate to http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/ http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/ the debug logger shows the error:

CRITICAL:emf@2016-05-07 11:40:26,496: record with key:{'version': 1.3, 'modulus': 11, 'number': 10} was not found! [web_object.py:632]

Yes, it shows that because the record does not exist. But the page loads fine for me and does what it should for me. Are you sure you are not running on your master branch (which I notice is 13 commits ahead?) I'm running your branch missing_db_record, which only changes the file display-list-newforms.html.

Yes, I’m on the same branch and I indeed only changed this one file. I know that my master branch is ahead and that seems to stem from the December workshop on classical mf’s and I’m trying to sort out in parallel what and why things haven’t made it to the lmfdb master. But I started the fix on top of lmfdb/master on purpose to not pull in unrelated changes!

Note that I just updated the pull request which improves the fix quite a bit.

However, I don’t understand how you could get anything different (with the old version of this branch as well as the new one).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217645951

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217646678

sehlen commented 8 years ago

Also note that this fixes the server error that appears when navigating to

/ModularForm/GL2/Q/holomorphic/90/3/37/

(even though there shouldn't be a link to this page).

fredstro commented 8 years ago

I'm currently still in the Cotswold so can't check but it seems that this list of problems agrees with the list found by Bober and which I tried to recompute when all the mongo problems occurred and I started to get the corrupted records bin gridfs. The last time I was in front of my computer I started a computation which shoukd have fixed these but apparently something went wrong.... Any individual broken record can be fixed (probably ) by me or Stephan calling WebNewForm_computing (which is defined in a separate repo ) with the correct label and parameter recompute =true. If you add in some kind of good error message then one of us should be able to fix the data on Monday (or in my case possibly late Sunday night) A good check for completeness is to extract the dimensions of all the orbits and add them up and compare with what sage says.

Fredrik

On Sat, 7 May 2016 16:55 Stephan Ehlen, notifications@github.com wrote:

On May 7, 2016, at 17:43, AndrewVSutherland notifications@github.com wrote:

Which link does your version hide, the link to 11/11/10 or the link to 11/11/10/a?

Of course to 11/11/10/a on the page for 11/11/10.

Then I try to navigate to http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/ < http://127.0.0.1:37777/ModularForm/GL2/Q/holomorphic/11/11/10/> the debug logger shows the error:

CRITICAL:emf@2016-05-07 11:40:26,496: record with key:{'version': 1.3, 'modulus': 11, 'number': 10} was not found! [web_object.py:632]

Yes, it shows that because the record does not exist. But the page loads fine for me and does what it should for me. Are you sure you are not running on your master branch (which I notice is 13 commits ahead?) I'm running your branch missing_db_record, which only changes the file display-list-newforms.html.

Yes, I’m on the same branch and I indeed only changed this one file. I know that my master branch is ahead and that seems to stem from the December workshop on classical mf’s and I’m trying to sort out in parallel what and why things haven’t made it to the lmfdb master. But I started the fix on top of lmfdb/master on purpose to not pull in unrelated changes!

Note that I just updated the pull request which improves the fix quite a bit.

However, I don’t understand how you could get anything different (with the old version of this branch as well as the new one).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub < https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217645951>

— You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217646678

sehlen commented 8 years ago

@fredstro the error message part has been done now because I think it is useful anyway. I will try to look at the data problems but I'm not sure I can really do it ;-) (see https://github.com/LMFDB/lmfdb/pull/1219)

fredstro commented 8 years ago

@sehlen we can try to deal with the Saya problems later together bit gor now, if you have access to a computer you could just fix these records by calling the compute function. But you shouldn't do it in parallel since then all problems (e.g. from mongo or otherwise ) gets hidden. Simply open 10 or 20 screen tabs on lehner and do one computation on each. Most of them should finish successfully (but it might take some time).

Fredrik

On Sat, 7 May 2016 17:13 Stephan Ehlen, notifications@github.com wrote:

@fredstro https://github.com/fredstro the error message part has been done because I think it is useful anyway. I will try to look at the data problems but I'm not sure I can really do it ;-)

— You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub https://github.com/LMFDB/lmfdb/issues/1217#issuecomment-217648041

JohnCremona commented 8 years ago

Now that PR #1219 is merged and pushed to beta and prod this at least fails politely. Tis issue should not be closed until the missing data is in. Here is Drew's checklist:

(wait until the cloud server has picked up the latest push to prod before testing that those no longer crash!)

AndrewVSutherland commented 8 years ago

I note that http://www.lmfdb.org/ModularForm/GL2/Q/holomorphic/49/8/1/ is different from the others. The Hecke orbit 49.8.1.d is missing (and presumably has dimension 1=21-(1+1+2+4+4+8)), but it is not shown as unavailable, it is simply omitted entirely.

sehlen commented 8 years ago

Good catch, @AndrewVSutherland. I would guess that the orbit failed to compute and was not added to the list of orbits of that space. But I don't really know and I have no idea why such data was ever inserted into the db and we have to wait for @fredstro to look into it (or for me to take some time to review the computing code which I can't do now) and do many many checks on the data and on the way it is being computed and inserted. Anyway, I think this issue here can be closed and a new one should be opened regarding your latest find and also regarding the Hecke orbits which are now officially hidden.

@fredstro I have a lot of trouble getting the computing classes to work which might be my fault but I really have to stop now. Sorry. If I just call them on lehner with recompute=True, all examples that I tried don't seem to actually recompute anything and don't give better results than what's on the webpage.

AndrewVSutherland commented 8 years ago

@sehlen http://www.lmfdb.org/ModularForm/GL2/Q/holomorphic/49/12/1/ has a similar issue, 49.12.1.c is missing.

AndrewVSutherland commented 8 years ago

I'm going to close this issue since the original problem that started this issue has been addressed by https://github.com/LMFDB/lmfdb/pull/1219 and https://github.com/LMFDB/lmfdb/pull/1222, and the bad/missing data problems exposed while looking into this are covered by the critical issue https://github.com/LMFDB/lmfdb/issues/1223. It will be simpler to move all the discussion to that thread.

LMFDB / lmfdb

Modular form error #1217