search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.32k stars 451 forks source link

Many unpickled objects from the pickle jar are totally broken #16311

Closed pjbruin closed 6 years ago

pjbruin commented 10 years ago

There is a doctest in sage/structure/sage_object.pyx to verify that all objects in the pickle jar can be unpickled to some Python object without raising an error, but that is about it as far as checking of the unpickling functionality for old pickles goes.

Dozens of unpickled objects are badly corrupted. For example, even asking for their string representation often fails and in at least one case causes Sage to crash.

After unpacking src/ext/pickle_jar/pickle_jar.tar.bz2 and changing to the resulting directory, run the following:

sage: files = [f for f in os.listdir('.') if f.endswith('.sobj')]
sage: for f in files:                                          
....:     try:
....:         s = str(load(f))
....:     except Exception, e:
....:         print('%s raised %s' % (f, e))
....: 
_class__sage_quadratic_forms_binary_qf_BinaryQF__.sobj raised 'BinaryQF' object has no attribute '_a'
/usr/local/sage/sage-current/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py:2834: DeprecationWarning: This class is replaced by Matrix_modn_dense_float/Matrix_modn_dense_double.
See http://trac.sagemath.org/4260 for details.
  exec code_obj in self.user_global_ns, self.user_ns
_class__sage_combinat_set_partition_ordered_OrderedSetPartitions_scomp__.sobj raised 'OrderedSetPartitions_scomp' object has no attribute '_set'
_class__sage_combinat_set_partition_SetPartitions_set__.sobj raised 'SetPartitions_set' object has no attribute '_set'
_type__sage_rings_morphism_RingHomomorphism_cover__.sobj raised 'QuotientRing_generic' object has no attribute '_QuotientRing_nc__R'
_class__sage_combinat_root_system_weight_space_WeightSpace__.sobj raised 'WeightSpace' object has no attribute '_name_string_helper'
_class__sage_combinat_set_partition_ordered_OrderedSetPartitions_sn__.sobj raised 'OrderedSetPartitions_sn' object has no attribute '_set'
_class__sage_combinat_free_module_CombinatorialFreeModuleElement__.sobj raised 'CombinatorialFreeModule' object has no attribute '_print_options'
/usr/local/sage/sage-current/src/bin/sage-ipython:3: DeprecationWarning: OrderedAlphabet is deprecated; use Alphabet instead.
See http://trac.sagemath.org/8920 for details.
  """
_class__sage_combinat_integer_vector_weighted_WeightedIntegerVectors_nweight__.sobj raised 'WeightedIntegerVectors_nweight' object has no attribute '_n'
_class__sage_combinat_root_system_type_G_ambient_space__.sobj raised 'AmbientSpace' object has no attribute '_name_string_helper'
_class__sage_rings_quotient_ring_QuotientRing_generic__.sobj raised 'QuotientRing_generic' object has no attribute '_QuotientRing_nc__R'
_class__sage_groups_abelian_gps_abelian_group_AbelianGroup_class__.sobj raised 'AbelianGroup_class' object has no attribute '_gens_orders'
_class__sage_rings_number_field_galois_group_GaloisGroup__.sobj raised 'GaloisGroup_v1' object has no attribute '_GaloisGroup_v1__group'
_class__sage_modular_modform_find_generators_ModularFormsRing__.sobj raised 'ModularFormsRing' object has no attribute '_ModularFormsRing__base_ring'
_class__sage_combinat_root_system_type_dual_CartanType__.sobj raised <abstract method is_affine at 0x3cd5668>
_class__sage_combinat_permutation_StandardPermutations_avoiding_321__.sobj raised 'StandardPermutations_avoiding_321' object has no attribute 'a'
_class__sage_combinat_permutation_StandardPermutations_avoiding_123__.sobj raised 'StandardPermutations_avoiding_123' object has no attribute 'a'
_class__sage_combinat_permutation_StandardPermutations_avoiding_312__.sobj raised 'StandardPermutations_avoiding_312' object has no attribute 'a'
_class__sage_homology_examples_SimplicialSurface__.sobj raised 'SimplicialComplex' object has no attribute '_facets'
_class__sage_combinat_set_partition_ordered_OrderedSetPartitions_s__.sobj raised 'OrderedSetPartitions_s' object has no attribute '_set'
_class__sage_combinat_tableau_SemistandardTableaux_p__.sobj raised 'SemistandardTableaux_shape' object has no attribute 'shape'
_class__sage_combinat_permutation_StandardPermutations_avoiding_213__.sobj raised 'StandardPermutations_avoiding_213' object has no attribute 'a'
_class__sage_combinat_permutation_StandardPermutations_avoiding_231__.sobj raised 'StandardPermutations_avoiding_231' object has no attribute 'a'
_class__sage_combinat_root_system_root_space_RootSpace__.sobj raised 'RootSpace' object has no attribute '_name_string_helper'
_class__sage_combinat_permutation_StandardPermutations_avoiding_132__.sobj raised 'StandardPermutations_avoiding_132' object has no attribute 'a'
_class__sage_combinat_set_partition_SetPartitions_setparts__.sobj raised 'SetPartitions_setparts' object has no attribute '_set'
_class__sage_combinat_sf_schur_SymmetricFunctionAlgebraElement_schur__.sobj raised 'SymmetricFunctionAlgebra_schur' object has no attribute '_print_options'
_class__sage_combinat_tableau_StandardTableaux_partition__.sobj raised 'StandardTableaux_shape' object has no attribute 'shape'
_class__sage_combinat_tableau_Tableaux_n__.sobj raised 'Tableaux_size' object has no attribute 'size'
_class__sage_combinat_skew_tableau_SemistandardSkewTableaux_n__.sobj raised 'SemistandardSkewTableaux_size' object has no attribute 'max_entry'
_class__sage_combinat_set_partition_SetPartitions_setn__.sobj raised 'SetPartitions_setn' object has no attribute '_set'
_class__sage_combinat_root_system_weyl_group_WeylGroupElement__.sobj raised 'WeylGroup_gens' object has no attribute '_prefix'
_class__sage_homology_simplicial_complex_SimplicialComplex__.sobj raised 'SimplicialComplex' object has no attribute '_facets'
_class__sage_combinat_skew_tableau_SemistandardSkewTableaux_p__.sobj raised 'SemistandardSkewTableaux_shape' object has no attribute 'max_entry'
_class__sage_combinat_permutation_StandardPermutations_avoiding_21__.sobj raised 'StandardPermutations_avoiding_21' object has no attribute 'a'
_class__sage_algebras_free_algebra_FreeAlgebra_generic__.sobj raised 'FreeAlgebra_generic' object has no attribute '_basis_keys'
_type__sage_rings_morphism_RingHomomorphism_im_gens__.sobj raised 'QuotientRing_generic' object has no attribute '_QuotientRing_nc__R'
_class__sage_algebras_free_algebra_quotient_element_FreeAlgebraQuotientElement__.sobj raised 'FreeAlgebra_generic' object has no attribute '_basis_keys'
_class__sage_algebras_free_algebra_element_FreeAlgebraElement__.sobj raised 'FreeAlgebraElement' object has no attribute '_monomial_coefficients'
_class__sage_combinat_root_system_weyl_group_WeylGroup_gens__.sobj raised 'WeylGroup_gens' object has no attribute '_domain'
_class__sage_combinat_permutation_StandardPermutations_avoiding_12__.sobj raised 'StandardPermutations_avoiding_12' object has no attribute 'a'
_class__sage_rings_homset_RingHomset_quo_ring__.sobj raised 'QuotientRing_generic' object has no attribute '_QuotientRing_nc__R'
_class__sage_combinat_alternating_sign_matrix_AlternatingSignMatrices_n__.sobj raised 'AlternatingSignMatrices' object has no attribute '_n'
_class__sage_schemes_generic_hypersurface_AffineHypersurface__.sobj raised 'AffineSpace_generic' object has no attribute '_dimension_relative'
sig_error() without sig_on()
...
Unhandled SIGABRT: An abort() occurred in Sage.
This probably occurred because a *compiled* component of Sage has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Sage will now terminate.

The following doctest marked "# not tested" from sage.structure.sage_object.unpickle_all also uncovers many errors, although many of these are due to the TestSuite failing, as indicated in the docstring:

sage: sage.structure.sage_object.unpickle_all(run_test_suite=True)
 * unpickle failure: TestSuite(load('/home/pbruin/.sage/temp/selmer/28748/dir_5OzW9h//pickle_jar/_class__sage_algebras_free_algebra_FreeAlgebra_generic__.sobj')).run()
 * unpickle failure: TestSuite(load('/home/pbruin/.sage/temp/selmer/28748/dir_5OzW9h//pickle_jar/_class__sage_algebras_free_algebra_element_FreeAlgebraElement__.sobj')).run()
 * unpickle failure: TestSuite(load('/home/pbruin/.sage/temp/selmer/28748/dir_5OzW9h//pickle_jar/_class__sage_algebras_free_algebra_quotient_FreeAlgebraQuotient__.sobj')).run()
...
[dozens of similar errors and various other ones]
...
 * unpickle failure: TestSuite(load('/home/pbruin/.sage/temp/selmer/28748/dir_5OzW9h//pickle_jar/_class__sage_combinat_sf_jack_JackPolynomials_q__.sobj')).run()
 * unpickle failure: TestSuite(load('/home/pbruin/.sage/temp/selmer/28748/dir_5OzW9h//pickle_jar/_class__sage_combinat_sf_llt_LLT_cospin__.sobj')).run()
 * unpickle failure: TestSuite(load('/home/pbruin/.sage/temp/selmer/28748/dir_5OzW9h//pickle_jar/_class__sage_combinat_sf_llt_LLT_spin__.sobj')).run()
sig_error() without sig_on()
...
Unhandled SIGABRT: An abort() occurred in Sage.
This probably occurred because a *compiled* component of Sage has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Sage will now terminate.

It looks like we have lots of unpickling functions to repair.

Component: pickling

Keywords: corrupted pickle crash

Issue created by migration from https://trac.sagemath.org/ticket/16311

pjbruin commented 10 years ago

Description changed:

--- 
+++ 
@@ -4,7 +4,7 @@

 After unpacking `src/ext/pickle_jar/pickle_jar.tar.bz2` and changing to the resulting directory, run the following:

-```python
+```
 sage: files = [f for f in os.listdir('.') if f.endswith('.sobj')]
 sage: for f in files:                                          
 ....:     try:
jdemeyer commented 6 years ago
comment:3

Fixed by #24337.