adafruit / circuitpython-build-tools

Build scripts for CircuitPython libraries and the bundle
MIT License
28 stars 17 forks source link

circuitpython-build-tools will not run on Windows #75

Closed lesamouraipourpre closed 2 years ago

lesamouraipourpre commented 3 years ago

Following comments from @jepler on #72, I've looked further into why I can't get circuitpython-build-tools to run on Windows.

The problem is usage of tempfile.NamedTemporaryFile(), for example at https://github.com/adafruit/circuitpython-build-tools/blob/716a38c713eddb9d2d713bcfbe95e5f0572791d6/circuitpython_build_tools/build.py#L221

The problem is documented at https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile, specifically

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

The error is triggered later in the method at a call to shutil.copyfile(temp_file.name, output_file) because copyfile tries to re-open the file.

Error on Windows:

Traceback (most recent call last):
  File "c:\users\pc\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\pc\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\PC\AppData\Local\Programs\Python\Python39\Scripts\circuitpython-build-bundles.exe\__main__.py", line 7, in <module>
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\circuitpython_build_tools\scripts\build_bundles.py", line 246, in build_bundles
    build_bundle(libs, bundle_version, zip_filename, package_folder_prefix,
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\circuitpython_build_tools\scripts\build_bundles.py", line 153, in build_bundle
    build.library(library_path, build_lib_dir,  package_folder_prefix,
  File "c:\users\pc\appdata\local\programs\python\python39\lib\site-packages\circuitpython_build_tools\build.py", line 234, in library
    shutil.copyfile(temp_file.name, output_file)
  File "c:\users\pc\appdata\local\programs\python\python39\lib\shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\PC\\AppData\\Local\\Temp\\tmp_73dy5j9'
python --version
Python 3.9.5

I don't use Windows for development so this bug is not of concern to me. That it has not been reported before suggests that almost nobody has come across this or at least have not reported it.

2bndy5 commented 3 years ago

It has been reported. This is a duplicate of #48. I was able to solve the error you're posting, but I abandoned trying to build mpy-cross with NMake.

Since then I've learned how to use CMake, so maybe its worth adjusting mpy-cross to build using

cmake -G "NMake Makefiles" <path-to-mpycross-src-with-CMakeLists.txt>

In the meantime, anyone can use WSL on windows... See also https://github.com/adafruit/circuitpython-build-tools/issues/48#issuecomment-692011657

2bndy5 commented 2 years ago

Need the AWS URI to bin/mpy-cross-... (pre-built for Windows)

I took another whack at this since mpy-cross builds are now downloaded from AWS (which wasn't an option last I tried solving this). However, I don't know the proper URL to use when downloading a pre-built executable of mpy-cross for Windows.

Meanwhile

I have been able to solve the copyfile() problem on Windows. which also led to an AttributeError since os.uname() does not exist in python on Windows. AFAIK, we can use platform.uname() instead without losing any required info (best practice would be to mandate uniform string casing as well).

I can submit a PR that simply fixes this issue, but full compatibility on Windows requires a pre-built executable of mpy-cross from AWS because building from source still uses make and other Unix specific cmds.

jepler commented 2 years ago

The listing for a path on our s3 is always viewable -- try https://adafruit-circuit-python.s3.amazonaws.com/index.html?prefix=bin/mpy-cross/

looks like a typical filename for mpy-cross for windows would be https://adafruit-circuit-python.s3.amazonaws.com/bin/mpy-cross/mpy-cross.static-x64-windows-7.1.1.exe

2bndy5 commented 2 years ago

@jepler Thank you! I'm kinda embarrassed to think that I didn't check with AWS search query (I don't often work with AWS because 💰).

With the correct URL, this pkg now runs on Windows (x64 only). I'll open a PR after checking contrib guidelines...

jepler commented 2 years ago

No worries, the "index.html?prefix=" trick is not something you would be expected to know. I only know it due to being in my browser history.