Cache issues

[edgarcosta]

Hello,

It looks like our generation of a random object is somewhat cached on the browser. Is there a way for us to disable this for our users?

For example, if you open multiple new tabs with the link http://www.lmfdb.org/Genus2Curve/Q/random then you always get the same curve.

I was able to recreate this throughout multiple browsers and in the respective private/incognito modes. I was only able to avoid this issue by disabling the cache in Chrome/Firefox DevTools.

I also observed the same behavior in other classes: Modular Forms/Siegel, Elliptic curves/Numberfields, NF/Local. Artin representations, Galois Groups and Sato-Tate groups.

Cheers, Edgar

[AndrewVSutherland]

This issue is specific to the cloud, it does not happen on http://beta.lmfdb.org/Genus2Curve/Q/Random.

[edgarcosta]

I agree.

But there is something underlying in all the classes that I mentioned.

For example: http://www.lmfdb.org/EllipticCurve/Q/random works fine.

[AndrewVSutherland]

The only difference I can see is that in the elliptic curves case the function random_object_from_collection is used, but in the other cases the function random_value_from_collection is used (both are implemented in https://github.com/LMFDB/lmfdb/blob/master/lmfdb/utils.py). But looking at the implementations of these functions I see no obvious reason why they would produce different caching behavior, they are basically the same except that the latter should in theory be slightly more efficient because it is reducing the amount of data being returned by mongo db. Compare lines 41 and 56 to see the difference.

[edgarcosta]

It is indeed a cloud issue, but nothing to do with the load balancer, as accessing directly the servers doesn't change anything: http://www-central0.lmfdb.xyz/Genus2Curve/Q/random

I will look into it.

[edgarcosta]

I think I figured it out. All the problematic redirects use the code 301 to redirect to the object, instead of 302.

See http://flask.pocoo.org/docs/0.12/api/#flask.redirect and https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection

$ for f in */*.py; do  egrep 'route\(".*/random"' -A10 -B1 $f -H | egrep 'redirect'  -B10 -A1  --color ; done
artin_representations/main.py-
artin_representations/main.py:@artin_representations_page.route("/random")
artin_representations/main.py-def random_representation():
artin_representations/main.py-    rep = random_object_from_collection(ArtinRepresentation.collection())
artin_representations/main.py-    num = random.randrange(0, len(rep['GaloisConjugates']))
artin_representations/main.py-    label = rep['Baselabel']+"c"+str(num+1)
artin_representations/main.py-    return redirect(url_for(".render_artin_representation_webpage", label=label), 301)
artin_representations/main.py-
ecnf/main.py-
ecnf/main.py:@ecnf_page.route("/random")
ecnf/main.py-def random_curve():
ecnf/main.py-    E = random_object_from_collection(db_ecnf())
ecnf/main.py-    return redirect(url_for(".show_ecnf", nf=E['field_label'], conductor_label=E['conductor_label'], class_label=E['iso_label'], number=E['number']), 301)
ecnf/main.py-
elliptic_curves/elliptic_curve.py-
elliptic_curves/elliptic_curve.py:@ec_page.route("/random")
elliptic_curves/elliptic_curve.py-def random_curve():
elliptic_curves/elliptic_curve.py-    label = random_object_from_collection( db_ec() )['lmfdb_label']
elliptic_curves/elliptic_curve.py-    cond, iso, num = split_lmfdb_label(label)
elliptic_curves/elliptic_curve.py-    return redirect(url_for(".by_triple_label", conductor=cond, iso_label=iso, number=num))
elliptic_curves/elliptic_curve.py-
galois_groups/main.py-
galois_groups/main.py:@galois_groups_page.route("/random")
galois_groups/main.py-def random_group():
galois_groups/main.py-    label = random_object_from_collection(base.getDBConnection().transitivegroups.groups)['label']
galois_groups/main.py-    return redirect(url_for(".by_label", label=label), 301)
galois_groups/main.py-
genus2_curves/main.py-
genus2_curves/main.py:@g2c_page.route("/Q/random")
genus2_curves/main.py-def random_curve():
genus2_curves/main.py-    label = random_value_from_collection(g2c_db_curves(), 'label')
genus2_curves/main.py-    return redirect(url_for_curve_label(label), 301)
genus2_curves/main.py-
higher_genus_w_automorphisms/main.py-
higher_genus_w_automorphisms/main.py:@higher_genus_w_automorphisms_page.route("/random")
higher_genus_w_automorphisms/main.py-def random_passport():
higher_genus_w_automorphisms/main.py-    C = base.getDBConnection()
higher_genus_w_automorphisms/main.py-    label = random_value_from_collection(C.curve_automorphisms.passports,'passport_label')
higher_genus_w_automorphisms/main.py-    return redirect(url_for(".by_passport_label", passport_label=label))
higher_genus_w_automorphisms/main.py-
lattice/main.py-# Random Lattice
lattice/main.py:@lattice_page.route("/random")
lattice/main.py-def random_lattice():
lattice/main.py-    res = random_object_from_collection( getDBConnection().Lattices.lat )
lattice/main.py-    return redirect(url_for(".render_lattice_webpage", label=res['label']))
lattice/main.py-
local_fields/main.py-
local_fields/main.py:@local_fields_page.route("/random")
local_fields/main.py-def random_field():
local_fields/main.py-    label = random_object_from_collection(base.getDBConnection().localfields.fields)['label']
local_fields/main.py-    return redirect(url_for(".by_label", label=label), 301)
local_fields/main.py-
modlmf/main.py-# Random modlmf
modlmf/main.py:@modlmf_page.route("/random")
modlmf/main.py-def random_modlmf():
modlmf/main.py-    res = random_object_from_collection( getDBConnection().mod_l_eigenvalues.modlmf )
modlmf/main.py-    return redirect(url_for(".render_modlmf_webpage", label=res['label']))
modlmf/main.py-
number_fields/number_field.py-
number_fields/number_field.py:@nf_page.route("/random")
number_fields/number_field.py-def random_nfglobal():
number_fields/number_field.py-    label = random_object_from_collection( getDBConnection().numberfields.fields )['label']
number_fields/number_field.py-    #This version leaves the word 'random' in the URL:
number_fields/number_field.py-    #return render_field_webpage({'label': label})
number_fields/number_field.py-    #This version uses the number field's own URL:
number_fields/number_field.py-    #url =
number_fields/number_field.py-    return redirect(url_for(".by_label", label= label))
number_fields/number_field.py-
rep_galois_modl/main.py-# Random rep_galois_modl
rep_galois_modl/main.py:@rep_galois_modl_page.route("/random")
rep_galois_modl/main.py-def random_rep_galois_modl():
rep_galois_modl/main.py-    res = random_object_from_collection( getDBConnection().mod_l_galois.reps )
rep_galois_modl/main.py-    return redirect(url_for(".render_rep_galois_modl_webpage", label=res['label']))
rep_galois_modl/main.py-
siegel_modular_forms/siegel_modular_form.py-
siegel_modular_forms/siegel_modular_form.py:@smf_page.route("/random")
siegel_modular_forms/siegel_modular_form.py-def random_sample():
siegel_modular_forms/siegel_modular_form.py-    return redirect(url_for('.by_label', label='.'.join(sample.random_sample_name())), 301)
siegel_modular_forms/siegel_modular_form.py-

[JohnCremona]

So it sounds as if we should use 302 not 301 throughout?

John

On 16 February 2017 at 15:58, edgarcosta notifications@github.com wrote:

I think I figured it out. All the problematic redirects use the code 301 to redirect to the object, instead of 302.

See http://flask.pocoo.org/docs/0.12/api/#flask.redirect and https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection

$ for f in /.py; do egrep 'route(".*/random"' -A10 -B1 $f -H | egrep 'redirect' -B10 -A1 --color ; done artin_representations/main.py- artin_representations/main.py:@artin_representations_page.route("/random") artin_representations/main.py-def random_representation(): artin_representations/main.py- rep = random_object_from_collection(ArtinRepresentation.collection()) artin_representations/main.py- num = random.randrange(0, len(rep['GaloisConjugates'])) artin_representations/main.py- label = rep['Baselabel']+"c"+str(num+1) artin_representations/main.py- return redirect(url_for(".render_artin_representation_webpage", label=label), 301) artin_representations/main.py- ecnf/main.py- ecnf/main.py:@ecnf_page.route("/random") ecnf/main.py-def random_curve(): ecnf/main.py- E = random_object_from_collection(db_ecnf()) ecnf/main.py- return redirect(url_for(".show_ecnf", nf=E['field_label'], conductor_label=E['conductor_label'], class_label=E['iso_label'], number=E['number']), 301) ecnf/main.py- elliptic_curves/elliptic_curve.py- elliptic_curves/elliptic_curve.py:@ec_page.route("/random") elliptic_curves/elliptic_curve.py-def random_curve(): elliptic_curves/elliptic_curve.py- label = random_object_from_collection( db_ec() )['lmfdb_label'] elliptic_curves/elliptic_curve.py- cond, iso, num = split_lmfdb_label(label) elliptic_curves/elliptic_curve.py- return redirect(url_for(".by_triple_label", conductor=cond, iso_label=iso, number=num)) elliptic_curves/elliptic_curve.py- galois_groups/main.py- galois_groups/main.py:@galois_groups_page.route("/random") galois_groups/main.py-def random_group(): galois_groups/main.py- label = random_object_from_collection(base.getDBConnection().transitivegroups.groups)['label'] galois_groups/main.py- return redirect(url_for(".by_label", label=label), 301) galois_groups/main.py- genus2_curves/main.py- genus2_curves/main.py:@g2c_page.route("/Q/random") genus2_curves/main.py-def random_curve(): genus2_curves/main.py- label = random_value_from_collection(g2c_db_curves(), 'label') genus2_curves/main.py- return redirect(url_for_curve_label(label), 301) genus2_curves/main.py- higher_genus_w_automorphisms/main.py- higher_genus_w_automorphisms/main.py:@higher_genus_w_automorphisms_page.route("/random") higher_genus_w_automorphisms/main.py-def random_passport(): higher_genus_w_automorphisms/main.py- C = base.getDBConnection() higher_genus_w_automorphisms/main.py- label = random_value_from_collection(C.curve_automorphisms.passports,'passport_label') higher_genus_w_automorphisms/main.py- return redirect(url_for(".by_passport_label", passport_label=label)) higher_genus_w_automorphisms/main.py- lattice/main.py-# Random Lattice lattice/main.py:@lattice_page.route("/random") lattice/main.py-def random_lattice(): lattice/main.py- res = random_object_from_collection( getDBConnection().Lattices.lat ) lattice/main.py- return redirect(url_for(".render_lattice_webpage", label=res['label'])) lattice/main.py- local_fields/main.py- local_fields/main.py:@local_fields_page.route("/random") local_fields/main.py-def random_field(): local_fields/main.py- label = random_object_from_collection(base.getDBConnection().localfields.fields)['label'] local_fields/main.py- return redirect(url_for(".by_label", label=label), 301) local_fields/main.py- modlmf/main.py-# Random modlmf modlmf/main.py:@modlmf_page.route("/random") modlmf/main.py-def random_modlmf(): modlmf/main.py- res = random_object_from_collection( getDBConnection().mod_l_eigenvalues.modlmf ) modlmf/main.py- return redirect(url_for(".render_modlmf_webpage", label=res['label'])) modlmf/main.py- number_fields/number_field.py- number_fields/number_field.py:@nf_page.route("/random") number_fields/number_field.py-def random_nfglobal(): number_fields/number_field.py- label = random_object_from_collection( getDBConnection().numberfields.fields )['label'] number_fields/number_field.py- #This version leaves the word 'random' in the URL: number_fields/number_field.py- #return render_field_webpage({'label': label}) number_fields/number_field.py- #This version uses the number field's own URL: number_fields/number_field.py- #url = number_fields/number_field.py- return redirect(url_for(".by_label", label= label)) number_fields/number_field.py- rep_galois_modl/main.py-# Random rep_galois_modl rep_galois_modl/main.py:@rep_galois_modl_page.route("/random") rep_galois_modl/main.py-def random_rep_galois_modl(): rep_galois_modl/main.py- res = random_object_from_collection( getDBConnection().mod_l_galois.reps ) rep_galois_modl/main.py- return redirect(url_for(".render_rep_galois_modl_webpage", label=res['label'])) rep_galois_modl/main.py- siegel_modular_forms/siegel_modular_form.py- siegel_modular_forms/siegel_modular_form.py:@smf_page.route("/random") siegel_modular_forms/siegel_modular_form.py-def random_sample(): siegel_modular_forms/siegel_modular_form.py- return redirect(url_for('.by_label', label='.'.join(sample.random_sample_name())), 301) siegel_modular_forms/siegel_modular_form.py-

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/LMFDB/lmfdb/issues/1970#issuecomment-280371514, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9N6bAO4yXErikLb7-oTg7ep1xFFLFbks5rdHIQgaJpZM4MCXMV .

[edgarcosta]

Yes, that would fix it. However, I'm not sure if one should use 302 or 307, the wikipedia page confuses me. I'm currently stuck building sage on my desktop before I can try it out.

[AndrewVSutherland]

PR 1973 now appears to be live on the cloud, but it doesn't seem to have fixed the problem.

[edgarcosta]

I have just tried: http://www.lmfdb.org/Genus2Curve/Q/random and I'm getting different curves.

[AndrewVSutherland]

After clearing the cache in my browser it now works for me too. Closing.