abelardopardo / ontask_b

A platform offering teachers and educational designers the capacity to use data to personalise the learner experience.
https://ontasklearning.org
MIT License
42 stars 36 forks source link

dataops Upload .csv issue #19

Closed scmc closed 6 years ago

scmc commented 6 years ago

Hi Abelardo,

We are running the most recent release (pulled from master).

After upgrading, migrating now on the dataops CSV file upload we are experiencing a system error on file upload I am wondering if I missed something:

on the dataops/upload_s2/ page after upload:

System error Oops! The platform had an internal error. We are sorry. We hope to address this anomaly soon.

the logs are showing:

[10/May/2018 16:00:37] ERROR [/usr/lib/python2.7/site-packages/django/core/handlers/exception.py:135] Internal Server Error: /dataops/upload_s2/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/usr/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view return view_func(request, *args, *kwargs) File "/www/ontask_b/src/dataops/upload.py", line 156, in upload_s2 data_frame, File "/www/ontask_b/src/dataops/ops.py", line 226, in perform_dataframe_upload_merge which there is a corresponding key in src_df. This means that the data in File "/www/ontask_b/src/dataops/ops.py", line 134, in store_dataframe_in_db return None File "/www/ontask_b/src/dataops/ops.py", line 103, in store_table_in_db name=cname, File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(args, kwargs) File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 394, in create obj.save(force_insert=True, using=self.db) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 807, in save force_update=force_update, update_fields=update_fields) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 837, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 923, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 962, in _do_insert using=using, raw=raw) File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 1076, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1099, in execute_sql cursor.execute(sql, params) File "/usr/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) File "/usr/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) IntegrityError: null value in column "in_viz" violates not-null constraint DETAIL: Failing row contains (41, Firstname, string, t, [], 6, , null, null, null).

-Scott

abelardopardo commented 6 years ago

HI Scott.

Can you please tell me the git commit you have currently installed? I'm half way through an upgrade to a new version and there are some things that are not perfectly lined up. You should be running the release with tag Version_2_5_1

Regards.

On Fri, May 11, 2018 at 8:42 AM Scott McMillan notifications@github.com wrote:

Hi Abelardo,

We are running the most recent release (pulled from master).

After upgrading, migrating now on the dataops CSV file upload we are experiencing a system error on file upload I am wondering if I missed something:

on the dataops/upload_s2/ page after upload:

System error Oops! The platform had an internal error. We are sorry. We hope to address this anomaly soon.

the logs are showing:

[10/May/2018 16:00:37] ERROR [/usr/lib/python2.7/site-packages/django/core/handlers/exception.py:135] Internal Server Error: /dataops/upload_s2/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/usr/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view return view_func(request, *args, *kwargs) File "/www/ontask_b/src/dataops/upload.py", line 156, in upload_s2 data_frame, File "/www/ontask_b/src/dataops/ops.py", line 226, in perform_dataframe_upload_merge which there is a corresponding key in src_df. This means that the data in File "/www/ontask_b/src/dataops/ops.py", line 134, in store_dataframe_in_db return None File "/www/ontask_b/src/dataops/ops.py", line 103, in store_table_in_db name=cname, File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(args, kwargs) File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 394, in create obj.save(force_insert=True, using=self.db) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 807, in save force_update=force_update, update_fields=update_fields) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 837, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 923, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 962, in _do_insert using=using, raw=raw) File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, *kwargs) File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 1076, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1099, in execute_sql cursor.execute(sql, params) File "/usr/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) File "/usr/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit* six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) IntegrityError: null value in column "in_viz" violates not-null constraint DETAIL: Failing row contains (41, Firstname, string, t, [], 6, , null, null, null).

-Scott

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/abelardopardo/ontask_b/issues/19, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnIiPjsvF0WBe5QpueueGV_8nv4IsM9ks5txMlygaJpZM4T6rz_ .

-- ABELARDO PARDO | Professor and Dean Academic Division of Information Technology, Engineering and the Environment Honorary Associate, School of Electrical and Information Engineering, The University of Sydney Research Fellow, University of Texas at Arlington UNIVERSITY OF SOUTH AUSTRALIA Mawson Lakes Campus (IPC MLK-08) GPO Box 2471 | Adelaide | SA | 5001 T +61 8 8302 3200 | Twitter @abelardopardo E abelardo.pardo@unisa.edu.au abelardo.pardo@sydney.edu.au | W people.unisa.edu.au/Abelardo.Pardo http://people.unisa.edu.au/Abelardo.Pardo Project Lead of OnTaskLearning.org https://ontasklearning.org/

abelardopardo commented 6 years ago

Upon a second inspection of the log files, you seem to have a misalignment between the message written by the exception and the current python code. The stack trace points to lines in the code that are comments suggesting that the restart of the platform has not been properly done. I strongly suggest to download from the production branch, and those commits that are labeled with version numbers. Any other commits are likely unstable and reflect intermediate stages of the development.

Regards.

On Fri, May 11, 2018 at 4:49 PM Abelardo Pardo abelardo.pardo@unisa.edu.au wrote:

HI Scott.

Can you please tell me the git commit you have currently installed? I'm half way through an upgrade to a new version and there are some things that are not perfectly lined up. You should be running the release with tag Version_2_5_1

Regards.

On Fri, May 11, 2018 at 8:42 AM Scott McMillan notifications@github.com wrote:

Hi Abelardo,

We are running the most recent release (pulled from master).

After upgrading, migrating now on the dataops CSV file upload we are experiencing a system error on file upload I am wondering if I missed something:

on the dataops/upload_s2/ page after upload:

System error Oops! The platform had an internal error. We are sorry. We hope to address this anomaly soon.

the logs are showing:

[10/May/2018 16:00:37] ERROR [/usr/lib/python2.7/site-packages/django/core/handlers/exception.py:135] Internal Server Error: /dataops/upload_s2/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/usr/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view return view_func(request, *args, *kwargs) File "/www/ontask_b/src/dataops/upload.py", line 156, in upload_s2 data_frame, File "/www/ontask_b/src/dataops/ops.py", line 226, in perform_dataframe_upload_merge which there is a corresponding key in src_df. This means that the data in File "/www/ontask_b/src/dataops/ops.py", line 134, in store_dataframe_in_db return None File "/www/ontask_b/src/dataops/ops.py", line 103, in store_table_in_db name=cname, File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(args, kwargs) File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 394, in create obj.save(force_insert=True, using=self.db) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 807, in save force_update=force_update, update_fields=update_fields) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 837, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 923, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python2.7/site-packages/django/db/models/base.py", line 962, in _do_insert using=using, raw=raw) File "/usr/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, *kwargs) File "/usr/lib/python2.7/site-packages/django/db/models/query.py", line 1076, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1099, in execute_sql cursor.execute(sql, params) File "/usr/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) File "/usr/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit* six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) IntegrityError: null value in column "in_viz" violates not-null constraint DETAIL: Failing row contains (41, Firstname, string, t, [], 6, , null, null, null).

-Scott

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/abelardopardo/ontask_b/issues/19, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnIiPjsvF0WBe5QpueueGV_8nv4IsM9ks5txMlygaJpZM4T6rz_ .

-- ABELARDO PARDO | Professor and Dean Academic Division of Information Technology, Engineering and the Environment Honorary Associate, School of Electrical and Information Engineering, The University of Sydney Research Fellow, University of Texas at Arlington UNIVERSITY OF SOUTH AUSTRALIA Mawson Lakes Campus (IPC MLK-08) GPO Box 2471 | Adelaide | SA | 5001 T +61 8 8302 3200 | Twitter @abelardopardo E abelardo.pardo@unisa.edu.au abelardo.pardo@sydney.edu.au | W people.unisa.edu.au/Abelardo.Pardo http://people.unisa.edu.au/Abelardo.Pardo Project Lead of OnTaskLearning.org https://ontasklearning.org/

-- ABELARDO PARDO | Professor and Dean Academic Division of Information Technology, Engineering and the Environment Honorary Associate, School of Electrical and Information Engineering, The University of Sydney Research Fellow, University of Texas at Arlington UNIVERSITY OF SOUTH AUSTRALIA Mawson Lakes Campus (IPC MLK-08) GPO Box 2471 | Adelaide | SA | 5001 T +61 8 8302 3200 | Twitter @abelardopardo E abelardo.pardo@unisa.edu.au abelardo.pardo@sydney.edu.au | W people.unisa.edu.au/Abelardo.Pardo http://people.unisa.edu.au/Abelardo.Pardo Project Lead of OnTaskLearning.org https://ontasklearning.org/

scmc commented 6 years ago

Hi Abelardo,

That worked I reverted to the 2.5.1 things are looking good.

-Scott

abelardopardo commented 6 years ago

Great to hear.

Incidentally, version 2.6.0 is now up in the repository. There are a few improvements in the UI as well as some additional functionality (table can now be downloaded as CSV). Let me know if you encounter any additional issues.

Best.

On Sat, May 12, 2018 at 7:44 AM Scott McMillan notifications@github.com wrote:

Hi Abelardo,

That worked I reverted to the 2.5.1 things are looking good.

-Scott

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/abelardopardo/ontask_b/issues/19#issuecomment-388498808, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnIiInCB3be37GUWHTVrlF5VsW0BcJ6ks5txg0rgaJpZM4T6rz_ .

-- Abelardo Pardo