search

coiled / feedback

A place to provide Coiled feedback
14 stars 3 forks source link

'ClusterManager' object has no attribute 'list_dask_clusters' #17

Closed ericdill closed 1 year ago

ericdill commented 4 years ago

@necaris clicking on an individual cluster in this view image

results in the following stack trace:

Environment:

Request Method: GET
Request URL: http://<URL>/organizations/1/cluster_types/ericdill-0a89fe2e-6

Django Version: 3.0.7
Python Version: 3.8.2
Installed Applications:
['whitenoise.runserver_nostatic',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.messages',
 'django.contrib.sites',
 'django.contrib.sessions',
 'django.contrib.staticfiles',
 'allauth',
 'allauth.account',
 'allauth.socialaccount',
 'allauth.socialaccount.providers.github',
 'bootstrap4',
 'channels',
 'dj_rest_auth',
 'rest_framework',
 'rest_framework.authtoken',
 'sass_processor',
 'api.apps.ApiConfig',
 'resources.apps.ResourcesConfig',
 'users.apps.UsersConfig',
 'cc',
 'clusters']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/contrib/auth/mixins.py", line 52, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/views/generic/base.py", line 97, in dispatch
    return handler(request, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/views/generic/detail.py", line 107, in get
    context = self.get_context_data(object=self.object)
  File "./users/views.py", line 142, in get_context_data
    context["clusters"] = ClusterQuerySet(
  File "./resources/models/aws.py", line 68, in __init__
    response = ResourcesConfig.backend.list_dask_clusters(self.organization)

Exception Type: AttributeError at /organizations/1/cluster_types/ericdill-0a89fe2e-6
Exception Value: 'ClusterManager' object has no attribute 'list_dask_clusters'

This is running off of the deployment-dtn branch plus the local changes that you already know about

ericdill commented 4 years ago

Similarly, trying to delete the cluster results in 

Environment:

Request Method: POST
Request URL: http://coiled.dev.dsci.zones.dtn.com/organizations/1/cluster_types/ericdill-0a89fe2e-6/_delete

Django Version: 3.0.7
Python Version: 3.8.2
Installed Applications:
['whitenoise.runserver_nostatic',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.messages',
 'django.contrib.sites',
 'django.contrib.sessions',
 'django.contrib.staticfiles',
 'allauth',
 'allauth.account',
 'allauth.socialaccount',
 'allauth.socialaccount.providers.github',
 'bootstrap4',
 'channels',
 'dj_rest_auth',
 'rest_framework',
 'rest_framework.authtoken',
 'sass_processor',
 'api.apps.ApiConfig',
 'resources.apps.ResourcesConfig',
 'users.apps.UsersConfig',
 'cc',
 'clusters']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/contrib/auth/mixins.py", line 52, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/django/views/generic/base.py", line 97, in dispatch
    return handler(request, *args, **kwargs)
  File "./users/views.py", line 189, in post
    ResourcesConfig.backend.delete_dask_cluster_type(org.aws_name(), typename)
  File "./backends/core.py", line 138, in delete_dask_cluster_type
    self._delete_dask_cluster_type, organization=organization, typename=typename

Exception Type: AttributeError at /organizations/1/cluster_types/ericdill-0a89fe2e-6/_delete
Exception Value: 'ClusterManager' object has no attribute '_delete_dask_cluster_type'
necaris commented 4 years ago

@ericdill thanks! Looking at the stacktrace our refactors broke more than we thought :-(

On Wed, Jun 10, 2020, 08:18 Eric Dill notifications@github.com wrote:

Similarly, trying to delete the cluster results in

Environment:

Request Method: POST Request URL: http://coiled.dev.dsci.zones.dtn.com/organizations/1/cluster_types/ericdill-0a89fe2e-6/_delete

Django Version: 3.0.7 Python Version: 3.8.2 Installed Applications: ['whitenoise.runserver_nostatic', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.messages', 'django.contrib.sites', 'django.contrib.sessions', 'django.contrib.staticfiles', 'allauth', 'allauth.account', 'allauth.socialaccount', 'allauth.socialaccount.providers.github', 'bootstrap4', 'channels', 'dj_rest_auth', 'rest_framework', 'rest_framework.authtoken', 'sass_processor', 'api.apps.ApiConfig', 'resources.apps.ResourcesConfig', 'users.apps.UsersConfig', 'cc', 'clusters'] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/conda/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, callback_kwargs) File "/opt/conda/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, *kwargs) File "/opt/conda/lib/python3.8/site-packages/django/contrib/auth/mixins.py", line 52, in dispatch return super().dispatch(request, args, kwargs) File "/opt/conda/lib/python3.8/site-packages/django/views/generic/base.py", line 97, in dispatch return handler(request, *args, **kwargs) File "./users/views.py", line 189, in post ResourcesConfig.backend.delete_dask_cluster_type(org.aws_name(), typename) File "./backends/core.py", line 138, in delete_dask_cluster_type self._delete_dask_cluster_type, organization=organization, typename=typename

Exception Type: AttributeError at /organizations/1/cluster_types/ericdill-0a89fe2e-6/_delete Exception Value: 'ClusterManager' object has no attribute '_delete_dask_cluster_type'

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coiled/cloud-issues/issues/17#issuecomment-641965501, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADQPYLNQTF7ZTQ64B6QJPLRV522FANCNFSM4N2J56SQ .

ericdill commented 4 years ago

Final issue from this morning:

tl;dr, I can see >10 cluster types in the web ui, can only get 5 of them via the python API and can't delete any of them.

I can see a bunch of cluster types in the web UI image

But when I use the python api, I only see 5 clusters. I also can't delete them: (oh and I renamed Exception -> CloudException in core.py so I could make sure I'm only catching client-side errors)

from coiled_cloud import Cloud, CoiledCluster, CloudException
cloud = Cloud(server, user, pw)
name = 'dask-2.16.0'

cluster_names = [cluster for cluster in cloud.list_clusters('ds')]
print(cluster_names)

for cluster_name in cluster_names:
    print(f'trying to delete {cluster_name}')
    try:
        cloud.delete_cluster_configuration(cluster_name)
    except CloudException:
        print(f'Cannot delete {cluster_name}')

This is the output of that script:

['ericdill-bbc42ba8-e', 'ericdill-9a6ca1d8-3', 'ericdill-3f7ad7c6-0', 'ericdill-406544fb-9', 'ericdill-f796db18-4']
trying to delete ericdill-bbc42ba8-e
Cannot delete ericdill-bbc42ba8-e
trying to delete ericdill-9a6ca1d8-3
Cannot delete ericdill-9a6ca1d8-3
trying to delete ericdill-3f7ad7c6-0
Cannot delete ericdill-3f7ad7c6-0
trying to delete ericdill-406544fb-9
Cannot delete ericdill-406544fb-9
trying to delete ericdill-f796db18-4
Cannot delete ericdill-f796db18-4
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f955fe43850>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7f955fe44820>, 99013.150182726)]']
connector: <aiohttp.connector.TCPConnector object at 0x7f955fe43a60>

And, If I don't catch the CloudException, here's the stack trace:

trying to delete ericdill-bbc42ba8-e
Traceback (most recent call last):
  File "run.py", line 10, in <module>
    cloud.delete_cluster_configuration(cluster_name)
  File "/home/ericdill/dev/coiled/cloud/python-api/coiled_cloud/core.py", line 501, in delete_cluster_configuration
    return self._sync(
  File "/home/ericdill/dev/coiled/cloud/python-api/coiled_cloud/core.py", line 198, in _sync
    return sync(
  File "/home/ericdill/miniconda/envs/django/lib/python3.8/site-packages/distributed/utils.py", line 347, in sync
    raise exc.with_traceback(tb)
  File "/home/ericdill/miniconda/envs/django/lib/python3.8/site-packages/distributed/utils.py", line 331, in f
    result[0] = yield future
  File "/home/ericdill/miniconda/envs/django/lib/python3.8/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/home/ericdill/dev/coiled/cloud/python-api/coiled_cloud/core.py", line 514, in _delete_cluster_configuration
    raise CloudException(text)
coiled_cloud.core.CloudException: {"detail":"Not found."}
necaris commented 4 years ago

thank you @ericdill !

mrocklin commented 4 years ago

Cluster types are removed, but stuck around in UI for a bit. This is now also removed in a current PR (good timing on your part).

Deletion not working is mildly surprising. I'll take a look.

necaris commented 4 years ago

@mrocklin pretty sure the delete UI element works on cluster types not cluster configurations, so likely just needs updating

On Wed, Jun 10, 2020, 10:43 Matthew Rocklin notifications@github.com wrote:

Cluster types are removed, but stuck around in UI for a bit. This is now also removed in a current PR (good timing on your part).

Deletion not working is mildly surprising. I'll take a look.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coiled/cloud-issues/issues/17#issuecomment-642056310, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADQPYOXKXOIM7QBHYIQVCLRV6L2DANCNFSM4N2J56SQ .

mrocklin commented 4 years ago

We don't currently have any web UI around cluster configurations. I'm inclined to wait for the frontend work to pick this up rather than implement this in Django-land. We could do this though if it's blocking someone. My guess is that this isn't on @ericdill 's critical path though, or that he's comfortable using the Python API (which works fine) as a temporary stand in.

We don't currently have a delete element on the Schedulers table shown in the dashboard. We could add one though. Same deal as above that we might also wait for the proper frontend work for this. Really this is just a question on how important this is short term for early users and how long it takes to turn this out properly (cc @dantheman39 )

shughes-uk commented 1 year ago

no longer an issue