ResourceNotFound on CloudSearch operations

[SamCyanide]

All operations that require the "DomainName" kwarg do not work, besides create_domain.

Any that do not require these return blank responses. csc = boto3.client('cloudsearch) Example:

pprint(csc.list_domain_names())
{'DomainNames': {'cloudsearchdomain1': '2011-02-01'},
 'ResponseMetadata': {'HTTPStatusCode': 200,
                      'RequestId': '983bdce5-3a07-11e5-b28e-bdb6cd8bee0c'}}

pprint(csc.describe_index_fields(DomainName='cloudsearchdomain1'))
---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
<ipython-input-16-01168a1f4334> in <module>()
----> 1 pprint(csc.describe_index_fields(DomainName='cloudsearchdomain1'))

/usr/local/lib/python3.4/dist-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    267                     "%s() only accepts keyword arguments." % py_operation_name)
    268             # The "self" in this scope is referring to the BaseClient.
--> 269             return self._make_api_call(operation_name, kwargs)
    270 
    271         _api_call.__name__ = str(py_operation_name)

/usr/local/lib/python3.4/dist-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    321 
    322         if http.status_code >= 300:
--> 323             raise ClientError(parsed_response, operation_name)
    324         else:
    325             return parsed_response

ClientError: An error occurred (ResourceNotFound) when calling the DescribeIndexFields operation: Domain not found: cloudsearchdomain1

pprint(csc.describe_expressions(DomainName='cloudsearchdomain1'))
---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
<ipython-input-24-4f571ab0e52d> in <module>()
----> 1 pprint(csc.describe_expressions(DomainName='cloudsearchdomain1'))

/usr/local/lib/python3.4/dist-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    267                     "%s() only accepts keyword arguments." % py_operation_name)
    268             # The "self" in this scope is referring to the BaseClient.
--> 269             return self._make_api_call(operation_name, kwargs)
    270 
    271         _api_call.__name__ = str(py_operation_name)

/usr/local/lib/python3.4/dist-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    321 
    322         if http.status_code >= 300:
--> 323             raise ClientError(parsed_response, operation_name)
    324         else:
    325             return parsed_response

ClientError: An error occurred (ResourceNotFound) when calling the DescribeExpressions operation: Domain not found: cloudsearchdomain1

pprint(csc.describe_domains(DomainNames=['cloudsearchdomain1']))
{'DomainStatusList': [],
 'ResponseMetadata': {'HTTPStatusCode': 200,
                      'RequestId': '0661307e-3a08-11e5-a69f-5d5e2552c255'}}

The domain exists in the console, and works just fine no matter what I do to it. What's going on? (I tried this with multiple domains, the name 'cloudsearchdomain1' is nonexistent, and used just as a placeholder, I have many more domains).

[jamesls]

I'm not able to repro. Are you sure that your client is using the same region in which you've created your domains? I did notice that if I specify a different region, I'll get an empty response:

In [15]: c = boto3.client('cloudsearch', 'us-west-2')

In [16]: c.list_domain_names()
Out[16]:
{u'DomainNames': {'testdomain': '2013-01-01'},
 'ResponseMetadata': {'HTTPStatusCode': 200,
  'RequestId': 'id'}}

c.describe_index_fields(DomainName='testdomain')['IndexFields']
Out[18]:
[{u'Options': {u'IndexFieldName': 'actors',
   u'IndexFieldType': 'text-array',
   u'TextArrayOptions': {u'AnalysisScheme': '_en_default_',
    u'HighlightEnabled': True,
    u'ReturnEnabled': True}},
   ...

In [19]: us_east_1 = boto3.client('cloudsearch', 'us-east-1')
In [20]: us_east_1.list_domain_names()
Out[20]:
{u'DomainNames': {'testdomain': '2013-01-01'},
 'ResponseMetadata': {'HTTPStatusCode': 200,
  'RequestId': 'id'}}

# Shows up in list_domain_names(), but we aren't able to describe the domain's index fields because it's in us-west-2.
In [21]: us_east_1.describe_index_fields(DomainName='testdomain')
Out[21]:
{u'IndexFields': [],
 'ResponseMetadata': {'HTTPStatusCode': 200,
  'RequestId': 'id1'}}

[SamCyanide]

Nope, this does not work either.

The domain in question - cloudsearchdomain1 - is in 'us-east-1' (at least according to the AWS console) and I tried the following: In:

csc = boto3.client('cloudsearch', 'us-east-1')

In:

pprint(csc.list_domain_names())

Output:

{'DomainNames': {'cloudsearchdomain1': '2011-02-01'},
 'ResponseMetadata': {'HTTPStatusCode': 200,
                      'RequestId': '6c356097-44f2-11e5-9ac1-352c9572c8dc'}}

In:

pprint(csc.describe_index_fields(DomainName='cloudsearchdomain1'))

Output:

---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
<ipython-input-9-624f59aa01cc> in <module>()
----> 1 pprint(csc.describe_index_fields(DomainName='cloudsearchdomain1')
      2 )

/usr/local/lib/python3.4/dist-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    267                     "%s() only accepts keyword arguments." % py_operation_name)
    268             # The "self" in this scope is referring to the BaseClient.
--> 269             return self._make_api_call(operation_name, kwargs)
    270 
    271         _api_call.__name__ = str(py_operation_name)

/usr/local/lib/python3.4/dist-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    321 
    322         if http.status_code >= 300:
--> 323             raise ClientError(parsed_response, operation_name)
    324         else:
    325             return parsed_response

ClientError: An error occurred (ResourceNotFound) when calling the DescribeIndexFields operation: Domain not found: cloudsearchdomain1

And running pprint(csc.describe_index_fields(DomainName='cloudsearchdomain1'))['IndexFields'] gives the same output.

[JordonPhillips]

I'm also not able to reproduce. Could you please attach debug logs so I can dig deeper? To enable the logs, put the following command before any other boto code:

boto3.set_stream_logger('botocore')

Thanks

[kyleknap]

Ping. I actually have a feeling that it may be due to the fact that your cloudsearch is the old version of cloudsearch '2011-02-01' because after looking at the docs: http://docs.aws.amazon.com/cloudsearch/latest/developerguide/migrating.html there is a lot of functionality changes. We are currently calling 2013-01-01 operations and based on the docs it seems that the 2011-02-01 API is needed to interact with 2011-02-01 domains. You can probably check by registering a handler to use the old version (note that I cannot confirm this because I am unable to create domains of that version. It is a legacy thing):

import boto3

client = boto3.client('cloudsearch', 'us-east-1')
def switch_api_version(request, **kwargs):
    request.data['Version'] = '2011-02-01'

client.meta.events.register('before-sign.cloudsearch', switch_api_version)
client.describe_index_fields(DomainName='cloudsearchdomain1')

How feasible is it to move off of the old cloudsearch version? It may be difficult to get a model for the old version of cloudsearch especially since it was major versioned bumped before the creation of boto3, but we can look into it.

[SamCyanide]

I think I ended up determining that the issue was indeed caused by the fact that we were using the old version.

I do not enjoy using older systems (because of lack of support like this!) but at my company we have many pieces of outdated infrastructure which doesn't get updated and ends up causing trouble. I'm not in a position to change this at the moment, so I have to just deal with it.

I think I ended up using some scraping method, not positive though.

[kyleknap]

Makes sense. Were you using the old boto.cloudsearch module (which uses 2011-02-01 before the cloudsearch2 package was created to use 2013-01-01 API) before? In the meantime, we can look to see if we can get a model of the old API version into botocore if you still need it?

[JordonPhillips]

@SamCyanide I think we've all been there. I'm sorry we couldn't help out more, hopefully this issue will help you get an update in a more timely manner.

[kyleknap]

So I actually was able to find an old model of this cloudsearch version (thanks javascript SDK) opened a PR to support it. To use it on the client, all you have to do is specify api_version='2011-02-01' when creating the cloudsearch client.