A null value cannot be used as the collection in a 'for' expression

[electriquo]

Describe the bug After installing steampipe-mod-aws-well-architected, steampipe cannot be started with a dashboard.

Steampipe version (steampipe -v)

$ steampipe --version
Steampipe v0.20.2

Plugin version (steampipe plugin list)

$ steampipe plugin list
+---------------------------------------------+---------+-------------+
| Installed Plugin                            | Version | Connections |
+---------------------------------------------+---------+-------------+
| hub.steampipe.io/plugins/turbot/aws@0.103.0 | 0.103.0 | aws         |
+---------------------------------------------+---------+-------------+

To reproduce

$ steampipe mod install github.com/turbot/steampipe-mod-aws-well-architected

Installed 2 mods:

local
└── github.com/turbot/steampipe-mod-aws-well-architected@v0.6.0
    └── github.com/turbot/steampipe-mod-aws-compliance@v0.63.0

$ steampipe service start --foreground --dashboard
Error: failed to load workspace: Failed to decode all mod hcl files:
Invalid function argument: Invalid value for "list" parameter: argument must not be null.
(/workspace/.steampipe/mods/github.com/turbot/steampipe-mod-aws-compliance@v0.65.0/mod.sp:35,18-39)
Iteration over null value: A null value cannot be used as the collection in a 'for' expression.
(/workspace/.steampipe/mods/github.com/turbot/steampipe-mod-aws-compliance@v0.65.0/mod.sp:48,17-35)

Expected behavior Steampipe should start

Additional context From looking at the mod list and the error above, it seems that steampipe loads steampipe-mod-aws-compliance version 0.65.0 (which is already installed) rather than 0.63.0 (the version installed by the module).

$ steampipe mod list

local
├── github.com/turbot/steampipe-mod-aws-compliance@v0.65.0
├── github.com/turbot/steampipe-mod-aws-insights@v0.12.0
├── github.com/turbot/steampipe-mod-aws-perimeter@v0.3.0
├── github.com/turbot/steampipe-mod-aws-tags@v0.9.0
└── github.com/turbot/steampipe-mod-aws-well-architected@v0.6.0
    └── github.com/turbot/steampipe-mod-aws-compliance@v0.63.0

[khushboo9024]

@foolioo Thanks for using Steampipe & raising this issue.

While I am checking whether the mod install is supported directly with the WellArchited mod, I would suggest you proceed with the git cloneapproach as mentioned here

git clone https://github.com/turbot/steampipe-mod-aws-well-architected.git
cd steampipe-mod-aws-well-architected
steampipe service start --foreground --dashboard

It may ask you to install it if it does not find the dependent AWS mod i.e. Error: failed to load workspace: not all dependencies are installed - run 'steampipe mod install'

In the same steampipe-mod-aws-well-architected, execute steampipe mod install >> This will install the required AWS compliance mod inside the same directory

Now you can execute > steampipe service start --foreground --dashboard Access the dashboard with http://localhost:9194/

Let us know if this helps you to unblock.

[electriquo]

@khushboo9024

1. I do not follow you, could you please explain why do you think that using the mod with git clone rather than steampipe mod install?
2. Plus, I shared everything so that you could reproduce the issue to verify it happens to you either, than try the fix with your offer to see whether it resolves the issue

[khushboo9024]

@foolioo, I apologize if the above was not enough to unblock you, and I provided an alternative for the time being

I am unable to reproduce the issue; I tried with

CLI v0.20.2

$ steampipe plugin list
+---------------------------------------------+---------+-------------+
| Installed Plugin                            | Version | Connections |
+---------------------------------------------+---------+-------------+
| hub.steampipe.io/plugins/turbot/aws@0.103.0 | 0.103.0 | aws         |
+---------------------------------------------+---------+-------------+

Installed the following mods in a folder called test-wta

test-wta$ steampipe mod install github.com/turbot/steampipe-mod-aws-perimeter@v0.3.0
Installed 1 mod:
local
└── github.com/turbot/steampipe-mod-aws-perimeter@v0.3.0

test-wta$ steampipe mod install github.com/turbot/steampipe-mod-aws-tags@v0.9.0
Installed 1 mod:
local
└── github.com/turbot/steampipe-mod-aws-tags@v0.9.0

test-wta$ steampipe mod install github.com/turbot/steampipe-mod-aws-insights@v0.12.0
Installed 1 mod:
local
└── github.com/turbot/steampipe-mod-aws-insights@v0.12.0

**Note**: I didn;t install AWS compliance specifically, considering it will be part of WellArchitected

Listing the mod

test-wta$ steampipe mod list
**local**
├── github.com/turbot/steampipe-mod-aws-insights@v0.12.0
├── github.com/turbot/steampipe-mod-aws-perimeter@v0.3.0
├── github.com/turbot/steampipe-mod-aws-tags@v0.9.0

Installed WellArchitected****

test-wta$ steampipe mod install github.com/turbot/steampipe-mod-aws-well-architected Installed 2 mods: local └── github.com/turbot/steampipe-mod-aws-well-architected@v0.6.0 └──github.com/turbot/steampipe-mod-aws-compliance@v0.63.0

Listing the mod

test-wta raj$ steampipe mod list
local
├──  github.com/turbot/steampipe-mod-aws-insights@v0.12.0
├── github.com/turbot/steampipe-mod-aws-perimeter@v0.3.0
├── github.com/turbot/steampipe-mod-aws-tags@v0.9.0
└── github.com/turbot/steampipe-mod-aws-well-architected@v0.6.0
    └── github.com/turbot/steampipe-mod-aws-compliance@v0.63.0

Executing the mod

test-wta$ steampipe service start --foreground --dashboard It loads up all the required mods. Are there any steps I missed out on reproducing? cc: @cbruno10 @rajlearner17

[cbruno10]

Hi @foolioo , thanks for raising this issue! As we were trying to reproduce behaviour you were seeing, we noticed that in Steampipe v0.20.0, there was a fix for mod dependency version resolution. In mod.sp, we previously had:

  require {
    mod "github.com/turbot/steampipe-mod-aws-compliance" {
      version = "^0.63.0"
      args = {
        common_dimensions = var.common_dimensions,
        tag_dimensions    = var.tag_dimensions
      }
    }
  }

In Steampipe v0.19.5, running steampipe mod install github.com/turbot/steampipe-mod-aws-well-architected or cloning the mod repo and running steampipe mod install would result in AWS Compliance v0.65.0 (the latest version) being installed.

However, running the same commands in v0.20.2 would result in AWS Compliance v0.63.0 being installed, not v0.65.0.

This is actually a good fix in Steampipe v0.20.2, and semver matching is now working correctly (even though we're using the carat, in v0.y.x of semver, each minor version can have breaking changes, so the caret only matches the latest minor version of pre-v1.0.0 versions).

So can you please try:

• Creating a new directory and starting fresh
• Run steampipe mod install github.com/turbot/steampipe-mod-aws-well-architected in that directory
• Run steampipe service start --foreground --dashboard

And let us know if this fixes your issue?

Also, please note that in the same directory, if you run steampipe mod install github.com/turbot/steampipe-mod-aws-compliance in the same directory and then try to run steampipe dashboard, you will get an error:

cbruno@M1P testinstall % steampipe dashboard
[ Wait    ] Loading Workspace
Error: failed to load workspace: more than one mod found which satisfies dependency github.com/turbot/steampipe-mod-aws-compliance@*

This error is expected since Steampipe isn't sure how to resolve the AWS Compliance mod dependency for the AWS Well-Architected mod, so if you want to install the AWS Compliance mod at top level, you will need to do so in another directory.

[electriquo]

Hi guys, thanks for your reply.

@khushboo9024 You forgot to install steampipe-mod-aws-compliance@0.65.0 directly along side steampipe-mod-aws-well-architected@0.6.0. only then the issue occurs.

add the following to your steps and see the issue occur.

$ steampipe mod install github.com/turbot/steampipe-mod-aws-compliance@0.65.0

@cbruno10 Maybe I am missing something. I would like to enjoy whatever steampipe offers and keep it fresh to benefit from all plugins and modules update. That was the motivation behind specifying the latest version of each plugin and module. While doing so, I have noticed the issue which I raised here.

After reading your suggestion, it is still unclear to me how can one use all steampipe plugins and modules in their latest version. Would you care to explain?

[cbruno10]

@foolioo If you're interested in using the AWS Compliance and AWS Well-Architected mod, you can now install them both in the same directory, as the AWS Well-Architected mod will pull the latest version of the AWS Compliance mod as part of installation/update, e.g.,

cbruno@M1P testinstall % steampipe mod install github.com/turbot/steampipe-mod-aws-compliance

Installed 1 mod:

local
└── github.com/turbot/steampipe-mod-aws-compliance@v0.65.0

cbruno@M1P testinstall % steampipe mod install github.com/turbot/steampipe-mod-aws-well-architected

Installed 2 mods:

local
└── github.com/turbot/steampipe-mod-aws-well-architected@v0.7.0
    └── github.com/turbot/steampipe-mod-aws-compliance@v0.65.0

cbruno@M1P testinstall % steampipe service start --foreground --dashboard
Steampipe service is running:

Database:

  Host(s):            localhost, 127.0.0.1, 192.168.1.9, 192.168.1.19
  Port:               9193
  Database:           steampipe
  User:               steampipe
  Password:           ********* [use --show-password to reveal]
  Connection string:  postgres://steampipe@localhost:9193/steampipe

Dashboard:

  Host(s):  localhost, 127.0.0.1, 192.168.1.9, 192.168.1.19
  Port:     9194
  URL:      http://localhost:9194/

Managing the Steampipe service:

  # Get status of the service
  steampipe service status

  # View database password for connecting from another machine
  steampipe service status --show-password

  # Restart the service
  steampipe service restart

  # Stop the service
  steampipe service stop

Hit Ctrl+C to stop the service
Stopping Steampipe service.
Steampipe service stopped.
cbruno@M1P testinstall % steampipe mod list

local
├── github.com/turbot/steampipe-mod-aws-compliance@v0.65.0
└── github.com/turbot/steampipe-mod-aws-well-architected@v0.7.0
    └── github.com/turbot/steampipe-mod-aws-compliance@v0.65.0

If you want to work in your current directory, you can run steampipe mod update, which will update the mod dependencies.

[electriquo]

@cbruno10 After upgrading, the error is gone.