search

jswope00 / s3-uploader-downloader

An uploader and downloader for Open EdX that works with AWS S3 storage.
6 stars 8 forks source link

Cannot install on Ironwood #2

Closed joshoftenly closed 4 years ago

joshoftenly commented 4 years ago

After following the instructions here I tried to install and ran into issues. I checked my venv files and I also tried running as not Edxapp user, I also tried running this as =production as opposed to =aws, still same.

I am still receiving this error:

~/edx-platform$ python manage.py lms makemigrations s3uploader_downloader --settings=aws WARNING:py.warnings:/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/init.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)

WARNING:root:DEPRECATION WARNING: aws.py has been deprecated, you should use production.py instead. Traceback (most recent call last): File "manage.py", line 120, in startup.run() File "/edx/app/edxapp/edx-platform/lms/startup.py", line 19, in run django.setup() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/init.py", line 1, in from .s3uploader_downloader import UploaderDownloaderXBlock File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/s3uploader_downloader.py", line 18, in from xmodule.modulestore.django import modulestore File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py", line 35, in from xblock_django.user_service import DjangoXBlockUserService File "/edx/app/edxapp/edx-platform/common/djangoapps/xblock_django/user_service.py", line 4, in from django.contrib.auth.models import User File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in class AbstractBaseUser(models.Model): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 110, in new app_config = apps.get_containing_app_config(module) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 247, in get_containing_app_config self.check_apps_ready() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 125, in check_apps_ready raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

jswope00 commented 4 years ago

Hi Josh, Thanks for your email. I think the last version we tested this for was Hawthorne. So I'm not surprised it isn't working on Ironwood (though I'd guess it's close).

@Atif Atiq aatiq@knysys.com can you try to install this on one of our Ironwood instances and troubleshoot it? If you get to the fix, push an Ironwood-compatible version to the repo.

Best, John

On Mon, Apr 6, 2020 at 4:46 AM Josh Hoke notifications@github.com wrote:

After following the instructions here I tried to install and ran into issues. I checked my venv files and I also tried running as not Edxapp user, I also tried running this as =production as opposed to =aws, still same.

I am still receiving this error:

~/edx-platform$ python manage.py lms makemigrations s3uploader_downloader --settings=aws WARNING:py.warnings:/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/ init.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)

WARNING:root:DEPRECATION WARNING: aws.py has been deprecated, you should use production.py instead. Traceback (most recent call last): File "manage.py", line 120, in startup.run() File "/edx/app/edxapp/edx-platform/lms/startup.py", line 19, in run django.setup() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/ init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/ init.py", line 1, in from .s3uploader_downloader import UploaderDownloaderXBlock File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/s3uploader_downloader.py", line 18, in from xmodule.modulestore.django import modulestore File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py", line 35, in from xblock_django.user_service import DjangoXBlockUserService File "/edx/app/edxapp/edx-platform/common/djangoapps/xblock_django/user_service.py", line 4, in from django.contrib.auth.models import User File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in class AbstractBaseUser(models.Model): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 110, in new app_config = apps.get_containing_app_config(module) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 247, in get_containing_app_config self.check_apps_ready() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 125, in check_apps_ready raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jswope00/s3-uploader-downloader/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALK22PGFJUNXPDE5E2BVUDRLGJGFANCNFSM4MCCPE4A .

-- John Swope curricu.me m. 603.490.7853

jswope00 commented 4 years ago

Hi John,

We are looking into it right away and get back to you with updates.

Regards,

Atif Atiq | Project Manager [image: KNYSYS] http://www.knysys.com/

PK W +92-21-34972358 | PK M +92-333-3341745

Skype: aatiq.knysys | Hangout: aatiq.knysys | aatiq@knysys.com

www.knysys.com | www.knyapps.com

On Mon, Apr 6, 2020 at 11:16 PM John Swope john@curricu.me wrote:

Hi Josh, Thanks for your email. I think the last version we tested this for was Hawthorne. So I'm not surprised it isn't working on Ironwood (though I'd guess it's close).

@Atif Atiq aatiq@knysys.com can you try to install this on one of our Ironwood instances and troubleshoot it? If you get to the fix, push an Ironwood-compatible version to the repo.

Best, John

On Mon, Apr 6, 2020 at 4:46 AM Josh Hoke notifications@github.com wrote:

After following the instructions here I tried to install and ran into issues. I checked my venv files and I also tried running as not Edxapp user, I also tried running this as =production as opposed to =aws, still same.

I am still receiving this error:

~/edx-platform$ python manage.py lms makemigrations s3uploader_downloader --settings=aws

WARNING:py.warnings:/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/ init.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)

WARNING:root:DEPRECATION WARNING: aws.py has been deprecated, you should use production.py instead. Traceback (most recent call last): File "manage.py", line 120, in startup.run() File "/edx/app/edxapp/edx-platform/lms/startup.py", line 19, in run django.setup() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/ init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/ init.py", line 1, in from .s3uploader_downloader import UploaderDownloaderXBlock File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/s3uploader_downloader.py", line 18, in from xmodule.modulestore.django import modulestore File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py", line 35, in from xblock_django.user_service import DjangoXBlockUserService File "/edx/app/edxapp/edx-platform/common/djangoapps/xblock_django/user_service.py", line 4, in from django.contrib.auth.models import User File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in class AbstractBaseUser(models.Model): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 110, in new app_config = apps.get_containing_app_config(module) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 247, in get_containing_app_config self.check_apps_ready() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 125, in check_apps_ready raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jswope00/s3-uploader-downloader/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALK22PGFJUNXPDE5E2BVUDRLGJGFANCNFSM4MCCPE4A .

-- John Swope curricu.me m. 603.490.7853

atif commented 4 years ago

Kindly tag Atif Atiq - you mistakenly tagged me

jswope00 commented 4 years ago

We started with installing S3-uploader-downloader.

We set the key in cms.env.json and restarted CMS. "ALLOW_ALL_ADVANCED_COMPONENTS": true,

Then, cloned repository of s3-uploader-downloader in /edx/app/edxapp. git clone https://github.com/jswope00/s3-uploader-downloader.git https://github.com/jswope00/s3-uploader-downloader.git

For installing and upgrading Xblock:

pip install s3-uploader-downloader/pip install -U --no-deps s3-uploader-downloader/

For Xblock path in site-packages: nano /edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/s3uploader_downloader.py

Then, added xblock (i.e 's3uploader_downloader') into the INSTALLED_APPS:

cd /edx/app/edxapp/edx-platformnano lms/envs/common.pynano cms/envs/common.py

Reproduced the same error on Ironwood instance that you reported by executing the following command: python manage.py lms makemigrations s3uploader_downloader –settings=aws

[image: image.png]

Error: django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

In Ironwood, the version of Django is 1.11.28 while in Hawthorn, it is 1.11.18. https://docs.djangoproject.com/en/1.11/ref/applications/#troubleshooting

We found that this happens when importing an application configuration or a models module triggers code that depends on the app registry.

We first identified the line in s3-uploader-downloader Xblock that shows error.

Then, we tried to change the order of this line that changed the execution order, but didn't work.

Also, tried by changing the importing user: edx-platform/common/djangoapps/xblock_django/user_service.py

From: from django.contrib.auth.models import User

To:

from django.contrib.auth import get_user_modelUser = get_user_model()

But, yet not got the success.

We'll continue working on it.

Note: Currently, we are working to configure s3-uploader-downloader on Ironwood. After that we will be requiring to test the Xblock and for this we'll need AWS Secret keys (i.e Access Key & Secret access key) for S3.

Regards,

Atif Atiq | Project Manager [image: KNYSYS] http://www.knysys.com/

PK W +92-21-34972358 | PK M +92-333-3341745

Skype: aatiq.knysys | Hangout: aatiq.knysys | aatiq@knysys.com

www.knysys.com | www.knyapps.com

On Tue, Apr 7, 2020 at 10:56 AM Atif Atiq aatiq@knysys.com wrote:

Hi John,

We are looking into it right away and get back to you with updates.

Regards,

Atif Atiq | Project Manager [image: KNYSYS] http://www.knysys.com/

PK W +92-21-34972358 | PK M +92-333-3341745

Skype: aatiq.knysys | Hangout: aatiq.knysys | aatiq@knysys.com

www.knysys.com | www.knyapps.com

On Mon, Apr 6, 2020 at 11:16 PM John Swope john@curricu.me wrote:

Hi Josh, Thanks for your email. I think the last version we tested this for was Hawthorne. So I'm not surprised it isn't working on Ironwood (though I'd guess it's close).

@Atif Atiq aatiq@knysys.com can you try to install this on one of our Ironwood instances and troubleshoot it? If you get to the fix, push an Ironwood-compatible version to the repo.

Best, John

On Mon, Apr 6, 2020 at 4:46 AM Josh Hoke notifications@github.com wrote:

After following the instructions here I tried to install and ran into issues. I checked my venv files and I also tried running as not Edxapp user, I also tried running this as =production as opposed to =aws, still same.

I am still receiving this error:

~/edx-platform$ python manage.py lms makemigrations s3uploader_downloader --settings=aws

WARNING:py.warnings:/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/ init.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)

WARNING:root:DEPRECATION WARNING: aws.py has been deprecated, you should use production.py instead. Traceback (most recent call last): File "manage.py", line 120, in startup.run() File "/edx/app/edxapp/edx-platform/lms/startup.py", line 19, in run django.setup() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/ init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/ init.py", line 1, in from .s3uploader_downloader import UploaderDownloaderXBlock File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/s3uploader_downloader.py", line 18, in from xmodule.modulestore.django import modulestore File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py", line 35, in from xblock_django.user_service import DjangoXBlockUserService File "/edx/app/edxapp/edx-platform/common/djangoapps/xblock_django/user_service.py", line 4, in from django.contrib.auth.models import User File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in class AbstractBaseUser(models.Model): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 110, in new app_config = apps.get_containing_app_config(module) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 247, in get_containing_app_config self.check_apps_ready() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 125, in check_apps_ready raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jswope00/s3-uploader-downloader/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALK22PGFJUNXPDE5E2BVUDRLGJGFANCNFSM4MCCPE4A .

-- John Swope curricu.me m. 603.490.7853

jswope00 commented 4 years ago

Hello Josh, A fix has been implemented that should allow you to install on Ironwood. Give it a shot and see if it works. John

On Mon, Apr 6, 2020 at 4:46 AM Josh Hoke notifications@github.com wrote:

After following the instructions here I tried to install and ran into issues. I checked my venv files and I also tried running as not Edxapp user, I also tried running this as =production as opposed to =aws, still same.

I am still receiving this error:

~/edx-platform$ python manage.py lms makemigrations s3uploader_downloader --settings=aws WARNING:py.warnings:/edx/app/edxapp/edx-platform/lms/djangoapps/courseware/ init.py:5: DeprecationWarning: Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported warnings.warn("Importing 'lms.djangoapps.courseware' as 'courseware' is no longer supported", DeprecationWarning)

WARNING:root:DEPRECATION WARNING: aws.py has been deprecated, you should use production.py instead. Traceback (most recent call last): File "manage.py", line 120, in startup.run() File "/edx/app/edxapp/edx-platform/lms/startup.py", line 19, in run django.setup() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/ init.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module import(name) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/ init.py", line 1, in from .s3uploader_downloader import UploaderDownloaderXBlock File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/s3uploader_downloader/s3uploader_downloader.py", line 18, in from xmodule.modulestore.django import modulestore File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/modulestore/django.py", line 35, in from xblock_django.user_service import DjangoXBlockUserService File "/edx/app/edxapp/edx-platform/common/djangoapps/xblock_django/user_service.py", line 4, in from django.contrib.auth.models import User File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in class AbstractBaseUser(models.Model): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 110, in new app_config = apps.get_containing_app_config(module) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 247, in get_containing_app_config self.check_apps_ready() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/apps/registry.py", line 125, in check_apps_ready raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jswope00/s3-uploader-downloader/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALK22PGFJUNXPDE5E2BVUDRLGJGFANCNFSM4MCCPE4A .

-- John Swope curricu.me m. 603.490.7853

joshoftenly commented 4 years ago

Aplogies guys, I am still getting an error, I will post it or the resolution once I finish later today.

Thanks for your alls time and attention, I will be on this everyday until it's working from today on, I had some other things to work.

joshoftenly commented 4 years ago

I got it. It's important to add the s3uploader/downloader to the right place in common files. Erg sorry that took so long. You guys did a good job getting the issue fixed.