samuelpowell commented 5 years ago

Platforms

[x] Linux (64) (CI only, not with hardware)
[ ] Linux (ARMv7)
[x] Linux (ARMv8) (CI only, not with hardware (?) )
[x] Win64
[x] Win32 (CI only, not with hardware)
[x] MacOS (CI only, not with hardware)

samuelpowell commented 5 years ago

@ReubenHill

Run the build , it will fail, update the hash in build.jl with that reported from the download for the Windows target
Look at the directory structure created under usr/, locate the library and update the product specification under the Sys.Windows guard. Modify the library names (excluding the extension) in the same gaurd to refer to it (I think for windows its the ftd2xx64 and ftd2xx without the lib prefix)
Profit?

ReubenHill commented 5 years ago

Struggling to work out what the "reported hash" is.

Output from build:

(v1.0) pkg> build LibFTD2XX
  Building LibFTD2XX → `C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.log`
 Resolving package versions...
┌ Error: Error building `LibFTD2XX`:
│ ┌ Warning: On Windows, creating file symlinks requires Administrator privileges
│ └ @ Base.Filesystem file.jl:789
│ [ Info: Directory C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\release\build does not exist!
│ ┌ Warning: Could not extract the platform key of https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip; continuing...
│ └ @ BinaryProvider C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:185
│ [ Info: Downloading https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip to C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip...
│ ERROR: LoadError: Hash must be 256 bits (64 characters) long, given hash is 0 characters long
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #verify#144(::Bool, ::Bool, ::String, ::Function, ::String, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:498
│  [3] #verify at .\none:0 [inlined]
│  [4] #download_verify#94(::Bool, ::Bool, ::Bool, ::Function, ::String, ::String, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\PlatformEngines.jl:569
│  [5] #download_verify at .\none:0 [inlined]
│  [6] #install#133(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:314
│  [7] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at .\none:0
│  [8] top-level scope at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:46
│  [9] include at .\boot.jl:317 [inlined]
│  [10] include_relative(::Module, ::String) at .\loading.jl:1044
│  [11] include(::Module, ::String) at .\sysimg.jl:29
│  [12] include(::String) at .\client.jl:392
│  [13] top-level scope at none:0
│ in expression starting at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:42
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Operations.jl:1097

samuelpowell commented 5 years ago

Forgot to push, try again

ReubenHill commented 5 years ago

Updated the hash but looks like the .zip isn't being unzipped. I presume you don't want me to update the product spec to point to the downloaded zip file?

build.log:

┌ Warning: On Windows, creating file symlinks requires Administrator privileges
└ @ Base.Filesystem file.jl:789
[ Info: Directory C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\release\build does not exist!
┌ Warning: Could not extract the platform key of https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip; continuing...
└ @ BinaryProvider C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:185
[ Info: Downloading https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip to C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip...
[ Info: No hash cache found
[ Info: Calculated hash 82db36f089d391f194c8ad6494b0bf44c508b176f9d3302777c041dad1ef7fe6 for file C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip
[ Info: Installing C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip into C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr
ERROR: LoadError: Could not list contents of tarball C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] #list_tarball_files#143(::Bool, ::Function, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:468
 [3] list_tarball_files at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:457 [inlined]
 [4] macro expansion at .\logging.jl:307 [inlined]
 [5] #install#133(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:319
 [6] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at .\none:0
 [7] top-level scope at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:46
 [8] include at .\boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at .\loading.jl:1044
 [10] include(::Module, ::String) at .\sysimg.jl:29
 [11] include(::String) at .\client.jl:392
 [12] top-level scope at none:0
in expression starting at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:42

ERROR: The pipe is being closed.

7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

Listing archive: 

--
Path = 
Type = tar
Code Page = UTF-8

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
------------------- ----- ------------ ------------  ------------------------
                                     0               0 files

samuelpowell commented 5 years ago

Can you manually confirm the zip file is valid? It seems to be failing to unzip

ReubenHill commented 5 years ago

Manually unzips fine in explorer for me

samuelpowell commented 5 years ago

Try deleting the usr subdirectory (the thing created by BinaryProvider) and re-running

ReubenHill commented 5 years ago

Deleting usr just seems to cause the process to repeat itself:

(v1.0) pkg> build LibFTD2XX
  Building LibFTD2XX → `C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.log`
 Resolving package versions...
┌ Error: Error building `LibFTD2XX`:
│ ┌ Warning: On Windows, creating file symlinks requires Administrator privileges
│ └ @ Base.Filesystem file.jl:789
│ [ Info: Directory C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\release\build does not exist!
│ ┌ Warning: Could not extract the platform key of https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip; continuing...
│ └ @ BinaryProvider C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:185
│ [ Info: Downloading https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip to C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip...
│ [ Info: No hash cache found
│ [ Info: Calculated hash 82db36f089d391f194c8ad6494b0bf44c508b176f9d3302777c041dad1ef7fe6 for file C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip
│ [ Info: Installing C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip into C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr
│ ERROR: LoadError: Could not list contents of tarball C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #list_tarball_files#143(::Bool, ::Function, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:468
│  [3] list_tarball_files at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:457 [inlined]
│  [4] macro expansion at .\logging.jl:307 [inlined]
│  [5] #install#133(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:319
│  [6] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at .\none:0
│  [7] top-level scope at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:46
│  [8] include at .\boot.jl:317 [inlined]
│  [9] include_relative(::Module, ::String) at .\loading.jl:1044
│  [10] include(::Module, ::String) at .\sysimg.jl:29
│  [11] include(::String) at .\client.jl:392
│  [12] top-level scope at none:0
│ in expression starting at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:42
│
│ ERROR: The pipe is being closed.
│
│ 7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
│
│
│ Listing archive:
│
│ --
│ Path =
│ Type = tar
│ Code Page = UTF-8
│
│    Date      Time    Attr         Size   Compressed  Name
│ ------------------- ----- ------------ ------------  ------------------------
│ ------------------- ----- ------------ ------------  ------------------------
│                                      0               0 files
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Operations.jl:1097

julia> # delete usr

(v1.0) pkg> build LibFTD2XX
  Building LibFTD2XX → `C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.log`
 Resolving package versions...
┌ Error: Error building `LibFTD2XX`:
│ ┌ Warning: On Windows, creating file symlinks requires Administrator privileges
│ └ @ Base.Filesystem file.jl:789
│ [ Info: Directory C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\release\build does not exist!
│ ┌ Warning: Could not extract the platform key of https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip; continuing...
│ └ @ BinaryProvider C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:185
│ [ Info: Downloading https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip to C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip...
│ [ Info: No hash cache found
│ [ Info: Calculated hash 82db36f089d391f194c8ad6494b0bf44c508b176f9d3302777c041dad1ef7fe6 for file C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip
│ [ Info: Installing C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip into C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr
│ ERROR: LoadError: Could not list contents of tarball C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\usr\downloads\CDM%20v2.12.28%20WHQL%20Certified.zip
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #list_tarball_files#143(::Bool, ::Function, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:468
│  [3] list_tarball_files at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:457 [inlined]
│  [4] macro expansion at .\logging.jl:307 [inlined]
│  [5] #install#133(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at C:\Users\Reuben\.julia\packages\BinaryProvider\4F5Hq\src\Prefix.jl:319
│  [6] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at .\none:0
│  [7] top-level scope at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:46
│  [8] include at .\boot.jl:317 [inlined]
│  [9] include_relative(::Module, ::String) at .\loading.jl:1044
│  [10] include(::Module, ::String) at .\sysimg.jl:29
│  [11] include(::String) at .\client.jl:392
│  [12] top-level scope at none:0
│ in expression starting at C:\Users\Reuben\Documents\GitHub\LibFTD2XX\deps\build.jl:42
│
│ 7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
│
│
│ Listing archive:
│
│ --
│ Path =
│ Type = tar
│ Code Page = UTF-8
│
│    Date      Time    Attr         Size   Compressed  Name
│ ------------------- ----- ------------ ------------  ------------------------
│ ------------------- ----- ------------ ------------  ------------------------
│                                      0               0 files
│
│ ERROR: The pipe is being closed.
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Operations.jl:1097

samuelpowell commented 5 years ago

Okay, so 7zip is failing.

Can you try and see what the problem is when unzipping it with 7zip?

samuelpowell commented 5 years ago

And check the filename is correct?!

ReubenHill commented 5 years ago

Are you sure it's not that it's trying to treat a zip file as a tarball..?

samuelpowell commented 5 years ago

No, not sure! Would be good if you could try using 7zip - perhaps do exactly what it does (see the BinaryProvider source)

ReubenHill commented 5 years ago

Having looked through the BinaryProvider source, I can find no evidence that it supports the installation of anything other than tarball .tar.gz binaries. Given that for windows all we have are zips, I suspect that BinaryProvider is not appropriate for us.

samuelpowell commented 5 years ago

I don't agree, we just need to get past the unzipping stage which should be manageable. The rest can be automated.

From: https://discourse.julialang.org/t/julia-0-7-package-development-workflow-where-are-git-repositories-of-packages/12455

Can you run the following, replacing the URLs and files, and see what command it is issuing, we can then modify to do the unzipping, and the rest should be fine.

import Pkg
Pkg.BinaryProvider.probe_platform_engines!(;verbose=true)
cmd = Pkg.BinaryProvider.gen_download_cmd("https://github.com/KristofferC/Crayons.jl/archive/v0.5.1.tar.gz", "Crayons.tar.gz")
run(cmd)
unpack = Pkg.BinaryProvider.gen_unpack_cmd("Crayons.tar.gz", "Crayons")
mkdir("Crayons")
run(unpack)

ReubenHill commented 5 years ago

Looks to be some issue with the unpack variable which is of type pipeline. I run

import Pkg
Pkg.BinaryProvider.probe_platform_engines!(;verbose=true)
bin_prefix = "https://www.ftdichip.com/Drivers"
cmd = Pkg.BinaryProvider.gen_download_cmd("$bin_prefix/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip", "unzipme.zip")
run(cmd)
unpack = Pkg.BinaryProvider.gen_unpack_cmd("unzipme.zip", "unzipped")
mkdir("unzipped")
run(unpack)

and get:

julia> import Pkg

julia> Pkg.BinaryProvider.probe_platform_engines!(;verbose=true)
[ Info: Probing for download engine...
[ Info: Probing powershell as a possibility...
[ Info:   Probe successful for powershell
[ Info: Found download engine powershell
[ Info: Probing for compression engine...
[ Info: Probing C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe as a possibility...
[ Info:   Probe successful for C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe
[ Info: Found compression engine C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe
true

julia> bin_prefix = "https://www.ftdichip.com/Drivers"
"https://www.ftdichip.com/Drivers"

julia> cmd = Pkg.BinaryProvider.gen_download_cmd("$bin_prefix/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip", "unzipme.zip")
`powershell -NoProfile -Command '[System.Net.ServicePointManager]::SecurityProtocol =
    [System.Net.SecurityProtocolType]::Tls12;
$webclient = (New-Object System.Net.Webclient);
$webclient.UseDefaultCredentials = $true;
$webclient.Proxy.Credentials = $webclient.Credentials;
$webclient.Headers.Add("user-agent", "Pkg.jl (https://github.com/JuliaLang/Pkg.jl)");
$webclient.DownloadFile("https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip", "unzipme.zip")
'`

julia> run(cmd)
Process(`powershell -NoProfile -Command '[System.Net.ServicePointManager]::SecurityProtocol =
    [System.Net.SecurityProtocolType]::Tls12;
$webclient = (New-Object System.Net.Webclient);
$webclient.UseDefaultCredentials = $true;
$webclient.Proxy.Credentials = $webclient.Credentials;
$webclient.Headers.Add("user-agent", "Pkg.jl (https://github.com/JuliaLang/Pkg.jl)");
$webclient.DownloadFile("https://www.ftdichip.com/Drivers/CDM/CDM%20v2.12.28%20WHQL%20Certified.zip", "unzipme.zip")
'`, ProcessExited(0))

julia> unpack = Pkg.BinaryProvider.gen_unpack_cmd("unzipme.zip", "unzipped")
pipeline(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x unzipme.zip -y -so`, stdout=`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x -si -y -ttar -ounzipped`)

julia> mkdir("unzipped")
ERROR: SystemError (with unzipped): mkdir: File exists
Stacktrace:
 [1] #systemerror#39(::String, ::Function, ::Symbol, ::Bool) at .\error.jl:106
 [2] #systemerror at .\none:0 [inlined]
 [3] #mkdir#7(::UInt16, ::Function, ::String) at .\file.jl:167
 [4] mkdir(::String) at .\file.jl:162
 [5] top-level scope at none:0

julia> run(unpack)

7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

Extracting archive:
--
Path =
Type = tar
Code Page = UTF-8

Everything is Ok

Files: 0
Size:       0
Compressed: 0

ERROR: The pipe is being closed.
ERROR: failed process: Process(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x unzipme.zip -y -so`, ProcessExited(2)) [2]
Stacktrace:
 [1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at .\error.jl:42
 [2] pipeline_error at .\process.jl:705 [inlined]
 [3] pipeline_error(::Base.ProcessChain) at .\process.jl:718
 [4] #run#504 at .\process.jl:663 [inlined]
 [5] run(::Base.OrCmds) at .\process.jl:661
 [6] top-level scope at none:0

if I get rid of the arguments to the first part of the pipeline:

unpack2 = pipeline(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x unzipme.zip`, stdout=`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x -si -y -ttar -ounzipped`)
run(unpack2)

the zip unzips to into the same directory as unzipme.zip:

julia> unpack2 = pipeline(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x unzipme.zip`, stdout=`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x -si -y -ttar -ounzipped`)
pipeline(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x unzipme.zip`, stdout=`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x -si -y -ttar -ounzipped`)julia> run(unpack2)

7-Zip [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04

Extracting archive:
--
Path =
Type = tar
Code Page = UTF-8

Everything is Ok

Files: 0
Size:       0
Compressed: 0
Base.ProcessChain(Base.Process[Process(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x unzipme.zip`, ProcessExited(0)), Process(`'C:\Users\Reuben\AppData\Local\Julia-1.0.2\bin\7z.exe' x -si -y -ttar -ounzipped`, ProcessExited(0))], RawFD(0x00000000), RawFD(0x00000001), RawFD(0x00000002))

samuelpowell commented 5 years ago

@ReubenHill so you happy to put the bits we discussed into the script and push to this branch? To summarise,

Use download_verify from BinaryProvider to grab the file (1 liner)
Unzip by building a suitable command using environment variables (see BinaryProviders source as the command it provides almost works) and directly running it.
Make sure directories in the Windows target match what is downloaded, and that the library names are correct.
Should just work!

ReubenHill commented 5 years ago

@samuelpowell I've made changes - all seems to work on windows. Please test on Linux. I also moved the write_deps_file call to make sure its run whenever the build script is run. It looks like that was what you intended in the first place? Feel free to change if not.

ReubenHill commented 5 years ago

CI needs to be updated to reflect whatever platform support we want this to have btw

samuelpowell commented 5 years ago

@ReubenHill thank you, have added OS X support, testing Win32, just leavers ARM targets

samuelpowell commented 5 years ago

@ReubenHill can you confirm library directory and name for Win32 in the archive it downloads so we can get AppVeyor 32 bit to pass?

codecov[bot] commented 5 years ago

Codecov Report

Merging #45 into master will not change coverage. The diff coverage is 75%.

@@           Coverage Diff           @@
##           master      #45   +/-   ##
=======================================
  Coverage   63.55%   63.55%           
=======================================
  Files           3        3           
  Lines         236      236           
=======================================
  Hits          150      150           
  Misses         86       86

Impacted Files	Coverage Δ
src/wrapper.jl	`73.33% <100%> (ø)`	:arrow_up:
src/LibFTD2XX.jl	`46.46% <66.66%> (ø)`	:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ec6ca18...10dfe7e. Read the comment docs.

samuelpowell commented 5 years ago

@ReubenHill thanks for the push but I don't think that functionally changes anything. The problem is that

[ Info: Directory C:\projects\libftd2xx-jl\deps\usr\i386 does not exist!

but presumably the same path with the 64-bit suffix does work, since it can find it. So ideally could you look at what gets unpacked and make sure that directory is indeed present?

ReubenHill commented 5 years ago

I'm a bit baffled. The i386 directory is definitely there and in the usr/ directory

ReubenHill commented 5 years ago

Perhaps it's not actually unzipping on win32? Can't see why though

samuelpowell commented 5 years ago

Confused++

Could you try stetting the 32/64-bit guard the wrong way around so your system pretends to be 32-bit, and see what happens in the filesystem?

samuelpowell commented 5 years ago

It was the comma???

All but one tests pass on Win32, any insight?

samuelpowell commented 5 years ago

Tests pass locally on AArch64

ReubenHill commented 5 years ago

The issue was a missing download location for 32 bit windows. I'm now looking into why tests are not passing on 32 bit windows...

samuelpowell commented 5 years ago

@tdowrick are you able to check out this branch on an ARMv7 device and ]build ]test to ensure things are still working on this architecture?

samuelpowell commented 5 years ago

@ReubenHill worth looking into as it could reveals deeper problems. I'm not that bothered about Win32 support so if it's an irrelevant ratsnest, please feel free just guard that particular test against Win32 and we can look into it if it every comes up!

ReubenHill commented 5 years ago

NB: ede8803 is a small fix to a more general issue outlined in #46

samuelpowell commented 5 years ago

Thanks @ReubenHill

ReubenHill commented 5 years ago

@samuelpowell CI is failing on linux exactly as described in #35 . Does it work locally?

samuelpowell commented 5 years ago

Yep, passes locally

samuelpowell commented 5 years ago

Is there anything useful we can do without making that call to create the device list on Linux?

samuelpowell commented 5 years ago

And I take it we never got to the bottom of what it meant?

ReubenHill commented 5 years ago

Is there anything useful we can do without making that call to create the device list on Linux?

Not really.

And I take it we never got to the bottom of what it meant?

No. It could be a device clash on the virtual machines or some issue with the wrong library being loaded... Hard to say if we can't reproduce it!

tdowrick commented 5 years ago

I tried this out on the ARM7 board and get the following build error:

root@colibri-imx6ull:~# julia-1.0.4/bin/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.4 (2019-05-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.0) pkg> dev /home/root/LibFTD2XX.jl/
   Cloning default registries into /home/root/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
 Resolving package versions...
┌ Warning: failed to extract archive downloaded from https://api.github.com/repos/JuliaLang/Compat.jl/tarball/84aa74986c5b9b898b0d1acaf3258741ee64754f
└ @ Pkg.Operations /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:477
┌ Warning: failed to extract archive downloaded from https://api.github.com/repos/JuliaPackaging/BinaryProvider.jl/tarball/c7361ce8a2129f20b0e05a89f7070820cfed6648
└ @ Pkg.Operations /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:477
   Cloning [34da2185-b29b-5c13-b0c7-acf172513d20] Compat from https://github.com/JuliaLang/Compat.jl.git
[ Info: Installed Compat ───────── v2.1.0
   Cloning [b99e7846-7c00-51b0-8f62-c81ae34c0232] BinaryProvider from https://github.com/JuliaPackaging/BinaryProvider.jl.git
[ Info: Installed BinaryProvider ─ v0.5.4
  Updating `~/.julia/environments/v1.0/Project.toml`
  [85e77ee8] + LibFTD2XX v0.1.0 [`~/LibFTD2XX.jl`]
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [b99e7846] + BinaryProvider v0.5.4
  [34da2185] + Compat v2.1.0
  [85e77ee8] + LibFTD2XX v0.1.0 [`~/LibFTD2XX.jl`]
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

(v1.0) pkg> build LibFTD2XX
  Building LibFTD2XX → `~/LibFTD2XX.jl/deps/build.log`
 Resolving package versions...
┌ Error: Error building `LibFTD2XX`:
│ [ Info: Directory /home/root/LibFTD2XX.jl/deps/usr/release/build does not exist!
│ ┌ Warning: Could not extract the platform key of https://www.ftdichip.com/Drivers/D2XX/Linux/libftd2xx-arm-v7-hf-1.4.8.gz; continuing...
│ └ @ BinaryProvider ~/.julia/packages/BinaryProvider/TcAwt/src/Prefix.jl:185
│ [ Info: Downloading https://www.ftdichip.com/Drivers/D2XX/Linux/libftd2xx-arm-v7-hf-1.4.8.gz to /home/root/LibFTD2XX.jl/deps/usr/downloads/libftd2xx-arm-v7-hf-1.4.8.gz...
│ [ Info: No hash cache found
│ [ Info: Calculated hash 81c8556184e9532a3a19ee6915c3a43110dc208116967a4d3e159f00db5d16e1 for file /home/root/LibFTD2XX.jl/deps/usr/downloads/libftd2xx-arm-v7-hf-1.4.8.gz
│ [ Info: Installing /home/root/LibFTD2XX.jl/deps/usr/downloads/libftd2xx-arm-v7-hf-1.4.8.gz into /home/root/LibFTD2XX.jl/deps/usr
│ ERROR: LoadError: Could not unpack /home/root/LibFTD2XX.jl/deps/usr/downloads/libftd2xx-arm-v7-hf-1.4.8.gz into /home/root/LibFTD2XX.jl/deps/usr
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] #unpack#94(::Bool, ::Function, ::String, ::String) at /home/root/.julia/packages/BinaryProvider/TcAwt/src/PlatformEngines.jl:660
│  [3] #unpack at ./tuple.jl:0 [inlined]
│  [4] #install#129(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at /home/root/.julia/packages/BinaryProvider/TcAwt/src/Prefix.jl:347
│  [5] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at ./tuple.jl:0
│  [6] top-level scope at /home/root/LibFTD2XX.jl/deps/build.jl:51
│  [7] include at ./boot.jl:317 [inlined]
│  [8] include_relative(::Module, ::String) at ./loading.jl:1044
│  [9] include(::Module, ::String) at ./sysimg.jl:29
│  [10] include(::String) at ./client.jl:392
│  [11] top-level scope at none:0
│ in expression starting at /home/root/LibFTD2XX.jl/deps/build.jl:46
[08:26:46] ######################################################################## 100.0%
│ [08:26:52] tar: unrecognized option '--directory=/home/root/LibFTD2XX.jl/deps/usr'
│ [08:26:52] BusyBox v1.24.1 (2019-04-01 10:26:50 UTC) multi-call binary.
│ [08:26:52]
│ [08:26:52] Usage: tar -[cxtZzJjahmvO] [-X FILE] [-T FILE] [-f TARFILE] [-C DIR] [FILE]...
└ @ Pkg.Operations /buildworker/worker/package_linuxarmv7l/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096

samuelpowell commented 5 years ago

Ahhh, okay, this is because it uses busybox instead of including the real tar, I is have a think

ReubenHill commented 5 years ago

@samuelpowell I'm keen to get this merged. Can you make an issue about ARM v7 and tidy CI support ready for merging?

samuelpowell commented 5 years ago

Agreed

Gowerlabs / LibFTD2XX.jl

Migrate to BinaryProvider, fix #41, fix #43 #45

Codecov Report