search

modoboa / modoboa-imap-migration

An extension to ease the migration between 2 IMAP servers using offlineimap
ISC License
17 stars 12 forks source link

OSError on modoboa_imap_migration #99

Closed DEK4 closed 1 year ago

DEK4 commented 3 years ago

Hello,

I found this issue #69 . I have the same problem. Is there anyone who can help me to resolve this problem?

Thanks

tonioo commented 3 years ago

Can you describe your issue a bit more please?

Whalehunter commented 3 years ago

I assume, because I have the same problem, is that the static directory is not generated as https://github.com/modoboa/modoboa-imap-migration/issues/69#issuecomment-764721305 mentioned.

The output of the python manage.py collectstatic is the following:

You have requested to collect static files at the destination
location as specified in your settings:

    /srv/modoboa/instance/sitestatic

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes
Found another file with the destination path 'index.html'. It will be ignored since only the first encountered file is collected. If this is not what you want, make sure every static file has a unique path.

0 static files copied to '/srv/modoboa/instance/sitestatic', 3150 unmodified.

I run the latest Modoboa release (1.17.0), with Python 3.7 and modoboa imap migration version 1.3.4

Please mention what kind of information you need, I'll be happy to provide it.

tonioo commented 3 years ago

Actually, the plugin based on vue.js (radicale, contacts and migration) won't work when the DEBUG flag is set to True. Try to set it to False (which is the recommanded value for a production server) and try again

Whalehunter commented 3 years ago

I have DEBUG = False in instance/settings.py. Here's a dump, hopefully without identifying information:

Internal Server Error: /modoboa_imap_migration/

OSError at /modoboa_imap_migration/
Error reading /srv/modoboa/env/lib/python3.7/site-packages/modoboa_imap_migration-1.3.4-py3.7.egg/modoboa_imap_migration/static/modoboa_imap_migration/webpack-stats.json. Are you sure webpack has generated the file and the path is correct?

Request Method: GET
Django Version: 2.2.24
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.7.3
Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/srv/modoboa/env/lib/python3.7/site-packages', '/srv/modoboa/env/lib/python3.7/site-packages/modoboa_imap_migration-1.3.4-py3.7.egg']
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.sites',
 'django.contrib.staticfiles',
 'reversion',
 'ckeditor',
 'ckeditor_uploader',
 'rest_framework',
 'rest_framework.authtoken',
 'phonenumber_field',
 'django_otp',
 'django_otp.plugins.otp_totp',
 'django_otp.plugins.otp_static',
 'modoboa',
 'modoboa.core',
 'modoboa.lib',
 'modoboa.admin',
 'modoboa.transport',
 'modoboa.relaydomains',
 'modoboa.limits',
 'modoboa.parameters',
 'modoboa.dnstools',
 'modoboa.policyd',
 'modoboa.maillog',
 'modoboa_amavis',
 'modoboa_pdfcredentials',
 'modoboa_postfix_autoreply',
 'modoboa_sievefilters',
 'modoboa_webmail',
 'modoboa_contacts',
 'modoboa_radicale',
 'modoboa_imap_migration',
 'webpack_loader')
Installed Middleware:
('x_forwarded_for.middleware.XForwardedForMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django_otp.middleware.OTPMiddleware',
 'modoboa.core.middleware.TwoFAMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'modoboa.core.middleware.LocalConfigMiddleware',
 'modoboa.lib.middleware.AjaxLoginRedirect',
 'modoboa.lib.middleware.CommonExceptionCatcher',
 'modoboa.lib.middleware.RequestCatcherMiddleware')

Traceback:

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/loader.py" in load_assets
  25.             with open(self.config['STATS_FILE'], encoding="utf-8") as f:

During handling of the above exception ([Errno 2] No such file or directory: '/srv/modoboa/env/lib/python3.7/site-packages/modoboa_imap_migration-1.3.4-py3.7.egg/modoboa_imap_migration/static/modoboa_imap_migration/webpack-stats.json'), another exception occurred:

File "/srv/modoboa/env/lib/python3.7/site-packages/django/core/handlers/exception.py" in inner
  34.             response = get_response(request)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/core/handlers/base.py" in _get_response
  145.                 response = self.process_exception_by_middleware(e, request)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/core/handlers/base.py" in _get_response
  143.                 response = response.render()

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/response.py" in render
  106.             self.content = self.rendered_content

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/response.py" in rendered_content
  83.         content = template.render(context, self._request)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/backends/django.py" in render
  61.             return self.template.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render
  171.                     return self._render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in _render
  163.         return self.nodelist.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render
  937.                 bit = node.render_annotated(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render_annotated
  904.             return self.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/loader_tags.py" in render
  150.             return compiled_parent._render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in _render
  163.         return self.nodelist.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render
  937.                 bit = node.render_annotated(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render_annotated
  904.             return self.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/loader_tags.py" in render
  150.             return compiled_parent._render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in _render
  163.         return self.nodelist.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render
  937.                 bit = node.render_annotated(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render_annotated
  904.             return self.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/loader_tags.py" in render
  62.                 result = block.nodelist.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render
  937.                 bit = node.render_annotated(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render_annotated
  904.             return self.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/defaulttags.py" in render
  309.                 return nodelist.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render
  937.                 bit = node.render_annotated(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/base.py" in render_annotated
  904.             return self.render(context)

File "/srv/modoboa/env/lib/python3.7/site-packages/django/template/library.py" in render
  192.         output = self.func(*resolved_args, **resolved_kwargs)

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/templatetags/webpack_loader.py" in render_bundle
  12.     tags = utils.get_as_tags(bundle_name, extension=extension, config=config, attrs=attrs)

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/utils.py" in get_as_tags
  62.     bundle = _get_bundle(bundle_name, extension, config)

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/utils.py" in _get_bundle
  40.     bundle = get_loader(config).get_bundle(bundle_name)

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/loader.py" in get_bundle
  59.         assets = self.get_assets()

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/loader.py" in get_assets
  36.                 self._assets[self.name] = self.load_assets()

File "/srv/modoboa/env/lib/python3.7/site-packages/webpack_loader/loader.py" in load_assets
  31.                     self.config['STATS_FILE']))

Exception Type: OSError at /modoboa_imap_migration/
Exception Value: Error reading /srv/modoboa/env/lib/python3.7/site-packages/modoboa_imap_migration-1.3.4-py3.7.egg/modoboa_imap_migration/static/modoboa_imap_migration/webpack-stats.json. Are you sure webpack has generated the file and the path is correct?
Request information:
USER: admin

GET: No GET data

POST: No POST data

FILES: No FILES data

META:
DOCUMENT_ROOT = '/srv/modoboa/instance'
PATH_INFO = '/modoboa_imap_migration/'
QUERY_STRING = ''
REQUEST_METHOD = 'GET'
REQUEST_URI = '/modoboa_imap_migration/'
SCRIPT_NAME = ''
SERVER_PROTOCOL = 'HTTP/1.1'

Settings:
Using settings module instance.settings
ABSOLUTE_URL_OVERRIDES = {}
AMAVIS_DEFAULT_DATABASE_ENCODING = 'LATIN1'
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'modoboa_imap_migration.auth_backends.IMAPBackend')"
AUTH_USER_MODEL = 'core.User'
BASE_DIR = '/srv/modoboa/instance'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_SECONDS = 600
CKEDITOR_ALLOW_NONIMAGE_FILES = False
CKEDITOR_BROWSE_SHOW_DIRS = True
CKEDITOR_CONFIGS = {'default': {'allowedContent': True, 'toolbar': 'Modoboa', 'width': None, 'toolbar_Modoboa': [['Bold', 'Italic', 'Underline'], ['JustifyLeft', 'JustifyCenter', 'JustifyRight', 'JustifyBlock'], ['BidiLtr', 'BidiRtl', 'Language'], ['NumberedList', 'BulletedList', '-', 'Outdent', 'Indent'], ['Undo', 'Redo'], ['Link', 'Unlink', 'Anchor', '-', 'Smiley'], ['TextColor', 'BGColor', '-', 'Source'], ['Font', 'FontSize'], ['Image'], ['SpellChecker']]}}
CKEDITOR_IMAGE_BACKEND = 'pillow'
CKEDITOR_RESTRICT_BY_USER = True
CKEDITOR_UPLOAD_PATH = 'uploads/'
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SECURE = True
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASE_ROUTERS = ['modoboa_amavis.dbrouter.AmavisRouter']
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISABLE_DASHBOARD_EXTERNAL_QUERIES = False
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.sites', 'django.contrib.staticfiles', 'reversion', 'ckeditor', 'ckeditor_uploader', 'rest_framework', 'rest_framework.authtoken', 'phonenumber_field', 'django_otp', 'django_otp.plugins.otp_totp', 'django_otp.plugins.otp_static', 'modoboa', 'modoboa.core', 'modoboa.lib', 'modoboa.admin', 'modoboa.transport', 'modoboa.relaydomains', 'modoboa.limits', 'modoboa.parameters', 'modoboa.dnstools', 'modoboa.policyd', 'modoboa.maillog', 'modoboa_amavis', 'modoboa_pdfcredentials', 'modoboa_postfix_autoreply', 'modoboa_sievefilters', 'modoboa_webmail', 'modoboa_contacts', 'modoboa_radicale', 'modoboa_imap_migration', 'webpack_loader')"
INTERNAL_IPS = []
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = '/accounts/profile/'
LOGIN_URL = '/accounts/login/'
LOGOUT_REDIRECT_URL = None
MANAGERS = []
MEDIA_ROOT = '/srv/modoboa/instance/media'
MEDIA_URL = '/media/'
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MIDDLEWARE = "('x_forwarded_for.middleware.XForwardedForMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django_otp.middleware.OTPMiddleware', 'modoboa.core.middleware.TwoFAMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'modoboa.core.middleware.LocalConfigMiddleware', 'modoboa.lib.middleware.AjaxLoginRedirect', 'modoboa.lib.middleware.CommonExceptionCatcher', 'modoboa.lib.middleware.RequestCatcherMiddleware')"
MIGRATION_MODULES = {}
MODOBOA_APPS = "('modoboa', 'modoboa.core', 'modoboa.lib', 'modoboa.admin', 'modoboa.transport', 'modoboa.relaydomains', 'modoboa.limits', 'modoboa.parameters', 'modoboa.dnstools', 'modoboa.policyd', 'modoboa.maillog', 'modoboa_amavis', 'modoboa_pdfcredentials', 'modoboa_postfix_autoreply', 'modoboa_sievefilters', 'modoboa_webmail', 'modoboa_contacts', 'modoboa_radicale', 'modoboa_imap_migration')"
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PHONENUMBER_DB_FORMAT = 'INTERNATIONAL'
PREPEND_WWW = False
REDIS_HOST = 'localhost'
REDIS_PORT = 6379
REDIS_QUOTA_DB = 0
REDIS_URL = 'redis://localhost:6379/0'
REST_FRAMEWORK = {'DEFAULT_AUTHENTICATION_CLASSES': ('rest_framework.authentication.TokenAuthentication', 'rest_framework.authentication.SessionAuthentication')}
ROOT_URLCONF = 'instance.urls'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = None
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SAMESITE = 'Lax'
SESSION_COOKIE_SECURE = True
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'instance.settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = ['security.W019', 'fields.W342']
SITE_ID = 1
STATICFILES_DIRS = "('/srv/modoboa/env/lib/python3.7/site-packages/modoboa/bower_components',)"
STATICFILES_FINDERS = ['django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder']
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/srv/modoboa/instance/sitestatic'
STATIC_URL = '/sitestatic/'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.contrib.messages.context_processors.messages', 'modoboa.core.context_processors.top_notifications'], 'debug': False}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'Europe/Paris'
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = False
USE_X_FORWARDED_PORT = False
WEBPACK_LOADER = {'CONTACTS': {'CACHE': True, 'BUNDLE_DIR_NAME': 'modoboa_contacts/', 'STATS_FILE': '/srv/modoboa/env/lib/python3.7/site-packages/modoboa_contacts/static/modoboa_contacts/webpack-stats.json', 'IGNORE': ['.+\\.hot-update.js', '.+\\.map']}, 'CALENDAR': {'CACHE': True, 'BUNDLE_DIR_NAME': 'modoboa_radicale/', 'STATS_FILE': '/srv/modoboa/env/lib/python3.7/site-packages/modoboa_radicale/static/modoboa_radicale/webpack-stats.json', 'IGNORE': ['.+\\.hot-update.js', '.+\\.map']}, 'IMAP_MIGRATION': {'CACHE': True, 'BUNDLE_DIR_NAME': 'modoboa_imap_migration/', 'STATS_FILE': '/srv/modoboa/env/lib/python3.7/site-packages/modoboa_imap_migration-1.3.4-py3.7.egg/modoboa_imap_migration/static/modoboa_imap_migration/webpack-stats.json', 'IGNORE': ['.+\\.hot-update.js', '.+\\.map']}}
WSGI_APPLICATION = 'instance.wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'
tonioo commented 3 years ago

Make sure to reload uwsgi

Whalehunter commented 3 years ago

Thank for the reply, I've reloaded uwsgi before replying. This is the dump I got in my mailbox from Django. If you have any advice on how to debug this problem, I'm all ears!