Build failing with 'cmake' not found

[hoIIer]

Hello, my previously working codebuild for my project began to fail just now with 'cmake' not found for awscrt (py 3.8). It looks like the last time I ran the build was 4 days ago, and it looks like the latest release for this repo was 4 days ago too. Any ideas on how to remediate?

Collecting awsiotsdk
--
49 | Downloading awsiotsdk-1.5.1-py3-none-any.whl (18 kB)

Building wheels for collected packages: awscrt
--
174 | Building wheel for awscrt (setup.py): started
175 | Building wheel for awscrt (setup.py): finished with status 'error'
176 | ERROR: Command errored out with exit status 1:
177 | command: /codebuild/output/src602545874/src/ve/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hpczhxw1/awscrt/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hpczhxw1/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-jj56xkwk
178 | cwd: /tmp/pip-install-hpczhxw1/awscrt/
179 | Complete output (61 lines):
180 | running bdist_wheel
181 | running build
182 | running build_py
183 | creating build
184 | creating build/lib.linux-x86_64-3.8
185 | creating build/lib.linux-x86_64-3.8/awscrt
186 | copying awscrt/http.py -> build/lib.linux-x86_64-3.8/awscrt
187 | copying awscrt/auth.py -> build/lib.linux-x86_64-3.8/awscrt
188 | copying awscrt/io.py -> build/lib.linux-x86_64-3.8/awscrt
189 | copying awscrt/mqtt.py -> build/lib.linux-x86_64-3.8/awscrt
190 | copying awscrt/exceptions.py -> build/lib.linux-x86_64-3.8/awscrt
191 | copying awscrt/crypto.py -> build/lib.linux-x86_64-3.8/awscrt
192 | copying awscrt/awsiot_mqtt_connection_builder.py -> build/lib.linux-x86_64-3.8/awscrt
193 | copying awscrt/__init__.py -> build/lib.linux-x86_64-3.8/awscrt
194 | running build_ext
195 | Traceback (most recent call last):
196 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 83, in check_cmake_installed
197 | subprocess.check_call(['cmake', '--version'])
198 | File "/usr/local/lib/python3.8/subprocess.py", line 359, in check_call
199 | retcode = call(*popenargs, **kwargs)
200 | File "/usr/local/lib/python3.8/subprocess.py", line 340, in call
201 | with Popen(*popenargs, **kwargs) as p:
202 | File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
203 | self._execute_child(args, executable, preexec_fn, close_fds,
204 | File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
205 | raise child_exception_type(errno_num, err_msg, err_filename)
206 | FileNotFoundError: [Errno 2] No such file or directory: 'cmake'
207 |  
208 | During handling of the above exception, another exception occurred:
209 |  
210 | Traceback (most recent call last):
211 | File "<string>", line 1, in <module>
212 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 245, in <module>
213 | setuptools.setup(
214 | File "/codebuild/output/src602545874/src/ve/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
215 | return distutils.core.setup(**attrs)
216 | File "/usr/local/lib/python3.8/distutils/core.py", line 148, in setup
217 | dist.run_commands()
218 | File "/usr/local/lib/python3.8/distutils/dist.py", line 966, in run_commands
219 | self.run_command(cmd)
220 | File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
221 | cmd_obj.run()
222 | File "/codebuild/output/src602545874/src/ve/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 290, in run
223 | self.run_command('build')
224 | File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
225 | self.distribution.run_command(command)
226 | File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
227 | cmd_obj.run()
228 | File "/usr/local/lib/python3.8/distutils/command/build.py", line 135, in run
229 | self.run_command(cmd_name)
230 | File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
231 | self.distribution.run_command(command)
232 | File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
233 | cmd_obj.run()
234 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 179, in run
235 | self._build_dependency(lib)
236 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 120, in _build_dependency
237 | check_cmake_installed()
238 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 86, in check_cmake_installed
239 | raise Exception("'cmake' not found. cmake must be installed to build from source.")
240 | Exception: 'cmake' not found. cmake must be installed to build from source.
241 | ----------------------------------------
242 | ERROR: Failed building wheel for awscrt
243 | Running setup.py clean for awscrt
244 | Failed to build awscrt
245 | Installing collected packages: PyJWT, six, python-dateutil, arrow, attrs, awscrt, awsiotsdk, webencodings, pyparsing, packaging, bleach, urllib3, jmespath, botocore, s3transfer, boto3, starlette, brotli, brotli-asgi, cached-property, certifi, elasticsearch, elasticsearch-dsl, MarkupSafe, jinja2, typing-extensions, mangum, marshmallow, marshmallow-jsonapi, cssselect, sgmllib3k, feedparser, lxml, PyYAML, soupsieve, beautifulsoup4, idna, chardet, requests, feedfinder2, regex, joblib, tqdm, click, nltk, Pillow, requests-file, tldextract, tinysegmenter, jieba3k, newspaper3k, passlib, cssutils, cachetools, premailer, pynamodb, requests-aws4auth, sentry-sdk, tld, pluggy, py, toml, iniconfig, pytest, pycodestyle, mccabe, pyflakes, flake8, pytest-flake8
246 | Running setup.py install for awscrt: started
247 | Running setup.py install for awscrt: finished with status 'error'
248 | ERROR: Command errored out with exit status 1:
249 | command: /codebuild/output/src602545874/src/ve/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hpczhxw1/awscrt/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hpczhxw1/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-xh76bofo/install-record.txt --single-version-externally-managed --compile --install-headers /codebuild/output/src602545874/src/ve/include/site/python3.8/awscrt
250 | cwd: /tmp/pip-install-hpczhxw1/awscrt/
251 | Complete output (63 lines):
252 | running install
253 | running build
254 | running build_py
255 | creating build
256 | creating build/lib.linux-x86_64-3.8
257 | creating build/lib.linux-x86_64-3.8/awscrt
258 | copying awscrt/http.py -> build/lib.linux-x86_64-3.8/awscrt
259 | copying awscrt/auth.py -> build/lib.linux-x86_64-3.8/awscrt
260 | copying awscrt/io.py -> build/lib.linux-x86_64-3.8/awscrt
261 | copying awscrt/mqtt.py -> build/lib.linux-x86_64-3.8/awscrt
262 | copying awscrt/exceptions.py -> build/lib.linux-x86_64-3.8/awscrt
263 | copying awscrt/crypto.py -> build/lib.linux-x86_64-3.8/awscrt
264 | copying awscrt/awsiot_mqtt_connection_builder.py -> build/lib.linux-x86_64-3.8/awscrt
265 | copying awscrt/__init__.py -> build/lib.linux-x86_64-3.8/awscrt
266 | running build_ext
267 | Traceback (most recent call last):
268 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 83, in check_cmake_installed
269 | subprocess.check_call(['cmake', '--version'])
270 | File "/usr/local/lib/python3.8/subprocess.py", line 359, in check_call
271 | retcode = call(*popenargs, **kwargs)
272 | File "/usr/local/lib/python3.8/subprocess.py", line 340, in call
273 | with Popen(*popenargs, **kwargs) as p:
274 | File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
275 | self._execute_child(args, executable, preexec_fn, close_fds,
276 | File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
277 | raise child_exception_type(errno_num, err_msg, err_filename)
278 | FileNotFoundError: [Errno 2] No such file or directory: 'cmake'
279 |  
280 | During handling of the above exception, another exception occurred:
281 |  
282 | Traceback (most recent call last):
283 | File "<string>", line 1, in <module>
284 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 245, in <module>
285 | setuptools.setup(
286 | File "/codebuild/output/src602545874/src/ve/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
287 | return distutils.core.setup(**attrs)
288 | File "/usr/local/lib/python3.8/distutils/core.py", line 148, in setup
289 | dist.run_commands()
290 | File "/usr/local/lib/python3.8/distutils/dist.py", line 966, in run_commands
291 | self.run_command(cmd)
292 | File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
293 | cmd_obj.run()
294 | File "/codebuild/output/src602545874/src/ve/lib/python3.8/site-packages/setuptools/command/install.py", line 61, in run
295 | return orig.install.run(self)
296 | File "/usr/local/lib/python3.8/distutils/command/install.py", line 545, in run
297 | self.run_command('build')
298 | File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
299 | self.distribution.run_command(command)
300 | File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
301 | cmd_obj.run()
302 | File "/usr/local/lib/python3.8/distutils/command/build.py", line 135, in run
303 | self.run_command(cmd_name)
304 | File "/usr/local/lib/python3.8/distutils/cmd.py", line 313, in run_command
305 | self.distribution.run_command(command)
306 | File "/usr/local/lib/python3.8/distutils/dist.py", line 985, in run_command
307 | cmd_obj.run()
308 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 179, in run
309 | self._build_dependency(lib)
310 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 120, in _build_dependency
311 | check_cmake_installed()
312 | File "/tmp/pip-install-hpczhxw1/awscrt/setup.py", line 86, in check_cmake_installed
313 | raise Exception("'cmake' not found. cmake must be installed to build from source.")
314 | Exception: 'cmake' not found. cmake must be installed to build from source.
315 | ----------------------------------------
316 | ERROR: Command errored out with exit status 1: /codebuild/output/src602545874/src/ve/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-hpczhxw1/awscrt/setup.py'"'"'; __file__='"'"'/tmp/pip-install-hpczhxw1/awscrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-xh76bofo/install-record.txt --single-version-externally-managed --compile --install-headers /codebuild/output/src602545874/src/ve/include/site/python3.8/awscrt Check the logs for full command output.

[bretambrose]

It's likely that a change in the crt switched you from consuming a pre-built wheel to having to explicitly build the native code component. That may due to a mistake on our part, but until we track it down, adding a cmake install to your build image/script may be a temporary workaround for the issue.

[bretambrose]

Following up, I believe this issue is related to https://github.com/awslabs/aws-crt-python/issues/191 Something has happened with the pre-built wheels to force a rebuild of the native code. When that is fixed I believe this issue will go away.

[elockman]

Issue #191 claims to be resolved, but I'm still having the same issue with the latest awsiotsdk (1.5.4) using

pip 21.0.1 from /usr/lib/python3.8/site-packages/pip (python 3.8)

[rpcme]

Hello @elockman can we close this issue due to https://github.com/aws/meta-aws/issues/109 ?

[elockman]

I have worked around this by adding it to my yocto build with aws/meta, but I think this is still an issue for others. I think this is still a pip install issue per @bretambrose's comment.

[graebm]

This is bizarre. Glad you found a workaround by installing cmake just in case.

If you could install with --verbose and share the full logs when this happens, we could see which wheels it sees in pypi, and perhaps see a reason that it's still choosing to build from source.

[hoIIer]

@rpcme @graebm not sure if this is still an issue? I pinned awsiotsdk==1.5.0 to get around it and haven't revisited https://github.com/awslabs/aws-crt-python/issues/192