Import of groups doesn't work

[jomeier]

Hi,

My Inventory contains groups and hosts which belong to this groups.

Today I tried to import an inventory file I previously exported with:

tower-cli receive --inventory all > inventory.json

I imported the inventory.json again with:

tower-cli send inventory.json

During the import I got error messages like this:

Created inventory azure_system-management_hosts
Schedules are up to date
Unable to manage group due to name field missing
Unable to manage group due to name field missing
Unable to manage group due to name field missing
Unable to manage group due to name field missing
Unable to manage group due to name field missing

It seems as if a field 'name' is missing in the exported inventory.json file.

Best regards,

Josef

Versions: Tower CLI 3.3.0 API v2 AWX 1.0.6.41 Ansible 2.6.1

[jomeier]

PR: https://github.com/ansible/tower-cli/pull/583

[john-westcott-iv]

@jomeier Again, can you let me know what version you are exporting out of and trying building tower-cli from the source? When I export an inventory with groups it properly adds the group name.

[Gianlu]

Hi, I had the same problem as @jomeier with tower-cli 3.3.0. I had also another issue: in tower-cli receive --inventory, the variables associated to groups are non exported (I suppose also variables associated to hosts)

[john-westcott-iv]

@Gianlu, what version of AWX were you exporting from? Can you also try the latest tower-cli compiled from GitHub to see if the issue persists?

[Gianlu]

@john-westcott-iv I'm exporting from awx 1.0.5 to 1.0.7. I tried with pip version, github version and finally I cloned @jomeier's repo and it worked. For group variables, I added a line after @jomeier's line to resolve group name issue and it exported (and imported) also variables.

[jomeier]

@john-westcott-iv The versions are the ones from my initial issue report:

Tower CLI 3.3.0
API v2
AWX 1.0.6.41
Ansible 2.6.1

I exported the inventories and also users again from AWX 1.0.6.41 and it still shows the same behaviour as described initially with tower-cli compiled from sources with

sudo make install

tower-cli receive --inventory kubernetes_3-masters_2-workers

shows:

[
  {
    "organization": "Default",
    "variables": "---\nansible_connection: local",
    "asset_relation": {
      "inventory_source": [],
      "host": [
        {
          "name": "master1"
        },
        {
          "name": "master2"
        },
        {
          "name": "master3"
        },
        {
          "name": "worker1"
        },
        {
          "name": "worker2"
        }
      ],
      "group": [
        {
          "hosts": [
            "master1",
            "master2",
            "master3"
          ],
          "sub_groups": []
        },
        {
          "hosts": [
            "worker1",
            "worker2"
          ],
          "sub_groups": []
        }
      ],
      "roles": [
        {
          "team": [],
          "name": "Ad Hoc",
          "user": []
        },
        {
          "team": [],
          "name": "Admin",
          "user": []
        },
        {
          "team": [],
          "name": "Read",
          "user": []
        },
        {
          "team": [],
          "name": "Use",
          "user": []
        },
        {
          "team": [],
          "name": "Update",
          "user": []
        }
      ]
    },
    "asset_type": "inventory",
    "name": "kubernetes_3-masters_2-workers"
  }
]

The group shows only hosts but not the group names.

On a tower-cli with my patch the same export shows the proper group names:

[
  {
    "organization": "Default",
    "variables": "---\nansible_connection: local",
    "asset_relation": {
      "inventory_source": [],
      "host": [
        {
          "name": "master1"
        },
        {
          "name": "master2"
        },
        {
          "name": "master3"
        },
        {
          "name": "worker1"
        },
        {
          "name": "worker2"
        }
      ],
      "group": [
        {
          "hosts": [
            "master1",
            "master2",
            "master3"
          ],
          "name": "masters",
          "sub_groups": []
        },
        {
          "hosts": [
            "worker1",
            "worker2"
          ],
          "name": "workers",
          "sub_groups": []
        }
      ],
      "roles": [
        {
          "team": [],
          "name": "Ad Hoc",
          "user": []
        },
        {
          "team": [],
          "name": "Admin",
          "user": []
        },
        {
          "team": [],
          "name": "Read",
          "user": []
        },
        {
          "team": [],
          "name": "Use",
          "user": []
        },
        {
          "team": [],
          "name": "Update",
          "user": []
        }
      ]
    },
    "asset_type": "inventory",
    "name": "kubernetes_3-masters_2-workers"
  }
]

If I import the last one with send into my AWX 1.0.6.41 everything works as expected.

[jomeier]

To be more precise: I cloned the tower-cli from the master branch.

[Gianlu]

@jomeier you're still missing variables (group_vars). After your line, I added another one in order to export variables

def process_inventory_groups(group_json):
    group_post_options = get_api_options('group')
    group_to_return = {}
    map_node_to_post_options(group_post_options, group_json, group_to_return)
    name_to_id_map = {}

    group_to_return['name'] = group_json['name']
    group_to_return['variables'] = group_json['variables'] ## <=== I added this one

Thank you.

[TwoTwenty]

Having a similar issue with groups having no 'name:' field Tower CLI 3.3.0 ansible awx 2.01 docker

  "group": [
    {
      "sub_groups": [],
      "hosts": [
        "uni-swsusp01.domain.ca"
      ]
    },
    {
      "sub_groups": [],
      "hosts": [
        "uni-scmdbp01.domain.ca",
        "uni-sricp01.domain.ca",
        "uni-ssophp01.domain.ca"
      ]
    },
    {
      "sub_groups": [],
      "hosts": [
        "uni-solmp01.domain.ca",
        "uni-ssacp01.domain.ca",
        "uni-svcenp02.domain.ca",
        "w88.domain.ca"
      ]
    },
    {
      "sub_groups": [],
      "hosts": [
        "uni-sbmsp01.domain.ca",
        "uni-sbmsp02.domain.ca",
        "uni-sclop01.domain.ca",
        "uni-sclop02.domain.ca",
        "uni-sgenp01.domain.ca",
        "uni-sgenp02.domain.ca"
      ]
    }
  ],

[john-westcott-iv]

@TwoTwenty try going to /api/v2/groups/ and click on the options button in the upper right. Post the JSON from that page to this list.

[TwoTwenty]

HTTP 200 OK Allow: GET, POST, HEAD, OPTIONS Content-Type: application/json Vary: Accept X-API-Node: awx X-API-Time: 0.056s

{ "name": "Group List", "description": "# List Groups:\n\nMake a GET request to this resource to retrieve the list of\ngroups.\n\nThe resulting data structure contains:\n\n {\n \"count\": 99,\n \"next\": null,\n \"previous\": null,\n \"results\": [\n ...\n ]\n }\n\nThe count field indicates the total number of groups\nfound for the given query. The next and previous fields provides links to\nadditional results if there are more than will fit on a single page. The\nresults list contains zero or more group records. \n\n## Results\n\nEach group data structure includes the following fields:\n\n id: Database ID for this group. (integer)\n type: Data type for this group. (choice)\n url: URL for this group. (string)\n related: Data structure with URLs of related resources. (object)\n summary_fields: Data structure with name/description for related resources. (object)\n created: Timestamp when this group was created. (datetime)\n modified: Timestamp when this group was last modified. (datetime)\n name: Name of this group. (string)\n description: Optional description of this group. (string)\n inventory: (field)\n variables: Group variables in JSON or YAML format. (string)\n has_active_failures: Flag indicating whether this group has any hosts with active failures. (boolean)\n total_hosts: Total number of hosts directly or indirectly in this group. (integer)\n hosts_with_active_failures: Number of hosts in this group with active failures. (integer)\n total_groups: Total number of child groups contained within this group. (integer)\n groups_with_active_failures: Number of child groups within this group that have active failures. (integer)\n has_inventory_sources: Flag indicating whether this group was created/updated from any external inventory sources. (boolean)\n\n\n\n## Sorting\n\nTo specify that groups are returned in a particular\norder, use the order_by query string parameter on the GET request.\n\n ?order_by=name\n\nPrefix the field name with a dash - to sort in reverse:\n\n ?order_by=-name\n\nMultiple sorting fields may be specified by separating the field names with a\ncomma ,:\n\n ?order_by=name,some_other_field\n\n## Pagination\n\nUse the page_size query string parameter to change the number of results\nreturned for each request. Use the page query string parameter to retrieve\na particular page of results.\n\n ?page_size=100&page=2\n\nThe previous and next links returned with the results will set these query\nstring parameters automatically.\n\n## Searching\n\nUse the search query string parameter to perform a case-insensitive search\nwithin all designated text fields of a model.\n\n ?search=findme\n\nAdded in AWX 1.4\n\n(Added in Ansible Tower 3.1.0) Search across related fields:\n\n ?relatedsearch=findme\n\nNote: If you want to provide more than one search term, multiple\nsearch fields with the same key, like `?relatedsearch=foo&relatedsearch=bar,\nwill be ORed together. Terms separated by commas, like?relatedsearch=foo,bar\nwill be ANDed together.\n\n## Filtering\n\nAny additional query string parameters may be used to filter the list of\nresults returned to those matching a given value. Only fields and relations\nthat exist in the database may be used for filtering. Any special characters\nin the specified value should be url-encoded. For example:\n\n ?field=value%20xyz\n\nFields may also span relations, only for fields and relationships defined in\nthe database:\n\n ?other__field=value\n\nTo exclude results matching certain criteria, prefix the field parameter with\nnot`:\n\n ?notfield=value\n\n(Added in AWX 1.4) By default, all query string filters are AND'ed together, so\nonly the results matching all filters will be returned. To combine results\nmatching any one of multiple criteria, prefix each query string parameter\nwith or__:\n\n ?orfield=value&orfield=othervalue\n ?ornotfield=value&orfield=othervalue\n\n(Added in Ansible Tower 1.4.5) The default AND filtering applies all filters\nsimultaneously to each related object being filtered across database\nrelationships. The chain filter instead applies filters separately for each\nrelated object. To use, prefix the query string parameter with `chain:\n\n ?chain__related__field=value&chain__related__field2=othervalue\n ?chain__not__related__field=value&chain__related__field2=othervalue\n\nIf the first query above were written as\n?relatedfield=value&relatedfield2=othervalue`, it would return only the\nprimary objects where the same related object satisfied both conditions. As\nwritten using the chain filter, it would return the intersection of primary\nobjects matching each condition.\n\nField lookups may also be used for more advanced queries, by appending the\nlookup to the field name:\n\n ?field__lookup=value\n\nThe following field lookups are supported:\n\n exact: Exact match (default lookup if not specified).\n iexact: Case-insensitive version of exact.\n contains: Field contains value.\n icontains: Case-insensitive version of contains.\n startswith: Field starts with value.\n istartswith: Case-insensitive version of startswith.\n endswith: Field ends with value.\n iendswith: Case-insensitive version of endswith.\n regex: Field matches the given regular expression.\n iregex: Case-insensitive version of regex.\n gt: Greater than comparison.\n gte: Greater than or equal to comparison.\n lt: Less than comparison.\n lte: Less than or equal to comparison.\n isnull: Check whether the given field or related object is null; expects a\n boolean value.\n in: Check whether the given field's value is present in the list provided;\n expects a list of items.\n\nBoolean values may be specified as True or 1 for true, False or 0 for\nfalse (both case-insensitive).\n\nNull values may be specified as None or Null (both case-insensitive),\nthough it is preferred to use the isnull lookup to explicitly check for null\nvalues.\n\nLists (for the in lookup) may be specified as a comma-separated list of\nvalues.\n\n(Added in Ansible Tower 3.1.0) Filtering based on the requesting user's\nlevel of access by query string parameter.\n\n role_level: Level of role to filter on, such as admin_role\n\n\n\n\n# Create a Group:\n\nMake a POST request to this resource with the following group\nfields to create a new group:\n\n\n\n\n\n\n\n\n\n name: Name of this group. (string, required)\n description: Optional description of this group. (string, default=\"\")\n inventory: (field, required)\n variables: Group variables in JSON or YAML format. (string, default=\"\")", "renders": [ "application/json", "text/html" ], "parses": [ "application/json" ], "actions": { "POST": { "name": { "type": "string", "required": true, "label": "Name", "max_length": 512, "help_text": "Name of this group.", "filterable": true }, "description": { "type": "string", "required": false, "label": "Description", "help_text": "Optional description of this group.", "filterable": true, "default": "" }, "inventory": { "type": "field", "required": true, "label": "Inventory", "filterable": true }, "variables": { "type": "string", "required": false, "label": "Variables", "help_text": "Group variables in JSON or YAML format.", "filterable": true, "default": "" } }, "GET": { "id": { "type": "integer", "label": "ID", "help_text": "Database ID for this group.", "filterable": true }, "type": { "type": "choice", "label": "Type", "help_text": "Data type for this group.", "choices": [ [ "group", "Group" ] ] }, "url": { "type": "string", "label": "Url", "help_text": "URL for this group." }, "related": { "type": "object", "label": "Related", "help_text": "Data structure with URLs of related resources." }, "summary_fields": { "type": "object", "label": "Summary fields", "help_text": "Data structure with name/description for related resources." }, "created": { "type": "datetime", "label": "Created", "help_text": "Timestamp when this group was created.", "filterable": true }, "modified": { "type": "datetime", "label": "Modified", "help_text": "Timestamp when this group was last modified.", "filterable": true }, "name": { "type": "string", "label": "Name", "help_text": "Name of this group.", "filterable": true }, "description": { "type": "string", "label": "Description", "help_text": "Optional description of this group.", "filterable": true }, "inventory": { "type": "field", "label": "Inventory", "filterable": true }, "variables": { "type": "string", "label": "Variables", "help_text": "Group variables in JSON or YAML format.", "filterable": true }, "has_active_failures": { "type": "boolean", "label": "Has active failures", "help_text": "Flag indicating whether this group has any hosts with active failures.", "filterable": true }, "total_hosts": { "type": "integer", "label": "Total hosts", "help_text": "Total number of hosts directly or indirectly in this group.", "min_value": 0, "filterable": true }, "hosts_with_active_failures": { "type": "integer", "label": "Hosts with active failures", "help_text": "Number of hosts in this group with active failures.", "min_value": 0, "filterable": true }, "total_groups": { "type": "integer", "label": "Total groups", "help_text": "Total number of child groups contained within this group.", "min_value": 0, "filterable": true }, "groups_with_active_failures": { "type": "integer", "label": "Groups with active failures", "help_text": "Number of child groups within this group that have active failures.", "min_value": 0, "filterable": true }, "has_inventory_sources": { "type": "boolean", "label": "Has inventory sources", "help_text": "Flag indicating whether this group was created/updated from any external inventory sources.", "filterable": true } } }, "types": [ "group" ], "search_fields": [ "description", "name" ], "related_search_fields": [ "modified_bysearch", "inventorysearch", "inventory_sourcessearch", "parentssearch", "hostssearch", "childrensearch", "created_by__search" ], "max_page_size": 200 }