search

JuliaCrypto / ToyFHE.jl

Toy implementation of FHE algorithms
Other
93 stars 12 forks source link

MNIST example does not work on Julia master #13

Open DilumAluthge opened 4 years ago

DilumAluthge commented 4 years ago

I think the issue is that the manifest uses an old version of CuArrays that does not work on Julia master.

cc: @Keno 

julia> cd("ToyFHE.jl/examples/encrypted_mnist")
julia rm("mnist_conv.bson"; force = true)
julia> include("train.jl")
[ Info: Precompiling CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae]
┌ Warning: Assignment to `name` in soft scope is ambiguous because a global variable by the same name exists: `name` will be treated as a new local. Disambiguate by using `local name` to suppress this warning or `global name` to assign to the existing global variable.
└ @ ~/.julia/packages/CUDAdrv/WqkY2/src/error.jl:83
┌ Warning: `@get!(dict, key, default)` at /users/daluthge/.julia/packages/Requires/9Jse8/src/require.jl:11 is deprecated, use `get!(()->default, dict, key)` instead.
│   caller = include(::Function, ::Module, ::String) at Base.jl:380
└ @ Base ./Base.jl:380
ERROR: LoadError: LoadError: syntax: invalid keyword argument syntax "(escape (= threads nthreads))"
Stacktrace:
 [1] top-level scope at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/array.jl:318
 [2] include(::Function, ::Module, ::String) at ./Base.jl:380
 [3] include at ./Base.jl:368 [inlined]
 [4] include(::String) at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/CuArrays.jl:1
 [5] top-level scope at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/CuArrays.jl:22
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:451
 [11] top-level scope at ./none:3
in expression starting at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/array.jl:318
in expression starting at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/CuArrays.jl:22
ERROR: LoadError: Failed to precompile CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae] to /users/daluthge/.julia/compiled/v1.5/CuArrays/7YFE0_YKowg.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1288
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::String) at ./client.jl:441
 [7] top-level scope at REPL[10]:1
in expression starting at /gpfs_home/daluthge/Desktop/ToyFHE.jl/examples/encrypted_mnist/train.jl:9

julia> versioninfo(verbose = true)
Julia Version 1.5.0-DEV.274
Commit 8eb0f9fefb (2020-02-15 12:41 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Red Hat Enterprise Linux Server release 7.3 (Maipo)"
  uname: Linux 3.10.0-957.5.1.el7.x86_64 #1 SMP Wed Dec 19 10:46:58 EST 2018 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz:
              speed         user         nice          sys         idle          irq
       #1  3601 MHz  302347066 s        170 s   53586390 s  337540460 s          0 s
       #2  3601 MHz  235401396 s        383 s   47128775 s  410325724 s          0 s
       #3  3601 MHz  103237285 s        288 s   30844130 s  559882062 s          0 s
       #4  3601 MHz   63412689 s        364 s   22978592 s  607626432 s          0 s
       #5  3601 MHz  315443735 s        113 s   49256804 s  329997494 s          0 s
       #6  3601 MHz  155708333 s        578 s   37188177 s  501770058 s          0 s
       #7  3601 MHz   54762847 s        546 s   15693504 s  624338616 s          0 s
       #8  3601 MHz   46859843 s        550 s   14616044 s  632741590 s          0 s

  Memory: 93.04103088378906 GB (74548.9140625 MB free)
  Uptime: 6.958699e6 sec
  Load Avg:  1.20654296875  1.44921875  1.22314453125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  CPLUS_INCLUDE_PATH = /gpfs/runtime/opt/gcc/8.3/include
  MANPATH = /gpfs/runtime/opt/python/3.7.4/share/man:/gpfs/runtime/opt/git/2.20.2/share/man:/gpfs/runtime/opt/gcc/8.3/share/man:/gpfs/runtime/opt/binutils/2.31/share/man:/gpfs/runtime/opt/intel/2017.0/man/common/man1:
  TERM = xterm-256color
  LIBRARY_PATH = /gpfs/runtime/opt/cuda/10.2/cuda/lib64:/gpfs/runtime/opt/cuda/10.2/cuda/lib:/gpfs/runtime/opt/cudnn/7.6.5/lib64:/gpfs/runtime/opt/python/3.7.4/lib:/gpfs/runtime/opt/binutils/2.31/lib:/gpfs/runtime/opt/intel/2017.0/lib/intel64:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64
  INCLUDE_PATH = /gpfs/runtime/opt/cudnn/7.6.5/include
  CUDA_HOME = /gpfs/runtime/opt/cuda/10.2/cuda
  LD_LIBRARY_PATH = /gpfs/runtime/opt/cuda/10.2/src/lib64:/gpfs/runtime/opt/cuda/10.2/src/lib:/gpfs/runtime/opt/cudnn/7.6.5/lib64:/gpfs/runtime/opt/python/3.7.4/lib:/gpfs/runtime/opt/gcc/8.3/lib64:/gpfs/runtime/opt/binutils/2.31/lib:/gpfs/runtime/opt/intel/2017.0/lib/intel64:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64:/gpfs/runtime/opt/java/8u111/jre/lib/amd64
  CPATH = /gpfs/runtime/opt/cuda/10.2/cuda/include:/gpfs/runtime/opt/cudnn/7.6.5/include:/gpfs/runtime/opt/python/3.7.4/include:/gpfs/runtime/opt/gcc/8.3/include:/gpfs/runtime/opt/binutils/2.31/include:/gpfs/runtime/opt/intel/2017.0/mkl/include
  NLSPATH = /gpfs/runtime/opt/intel/2017.0/lib/intel64/locale/en_US:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64/locale/en_US
  PATH = /gpfs/runtime/opt/cuda/10.2/cuda/bin:/users/daluthge/bin:/gpfs/runtime/opt/python/3.7.4/bin:/gpfs/runtime/opt/git/2.20.2/bin:/gpfs/runtime/opt/gcc/8.3/bin:/gpfs/runtime/opt/binutils/2.31/bin:/gpfs/runtime/opt/intel/2017.0/bin:/gpfs/runtime/opt/matlab/R2017b/bin:/gpfs/runtime/opt/java/8u111/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/usr/lpp/mmfs/sbin:/opt/ibutils/bin:/gpfs/runtime/bin
  C_INCLUDE_PATH = /gpfs/runtime/opt/cudnn/7.6.5/include:/gpfs/runtime/opt/gcc/8.3/include
  LD_RUN_PATH = /gpfs/runtime/opt/cuda/10.2/cuda/lib64:/gpfs/runtime/opt/cuda/10.2/cuda/lib:/gpfs/runtime/opt/cudnn/7.6.5/lib64:/gpfs/runtime/opt/cudnn/7.6.5/lib:/gpfs/runtime/opt/python/3.7.4/lib:/gpfs/runtime/opt/gcc/8.3/lib64:/gpfs/runtime/opt/binutils/2.31/lib:/gpfs/runtime/opt/intel/2017.0/lib/intel64:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64
  JAVA_HOME = /gpfs/runtime/opt/java/8u111
  MODULEPATH = /gpfs/runtime/modulefiles
  HOME = /users/daluthge
  IPP_PATH = /gpfs/runtime/opt/intel/2017.0/ipp
  MODULEHOME = /gpfs/runtime/pymodules
  PKG_CONFIG_PATH = /gpfs/runtime/opt/python/3.7.4/lib/pkgconfig
  QT_PLUGIN_PATH = /usr/lib64/kde4/plugins:/usr/lib/kde4/plugins

(ToyFHE) pkg> st
Project ToyFHE v0.1.0
Status `/gpfs_home/daluthge/Desktop/ToyFHE.jl/Project.toml`
  [c3fe647b] AbstractAlgebra v0.7.1 #master (https://github.com/Nemocas/AbstractAlgebra.jl.git)
  [15f4f7f2] AutoHashEquals v0.2.0
  [fbb218c0] BSON v0.2.4
  [c3b6d118] BitIntegers v0.2.0 #kf/div (https://github.com/Keno/BitIntegers.jl)
  [3a865a2d] CuArrays v1.3.0 #master (https://github.com/JuliaGPU/CuArrays.jl.git)
  [ab62b9b5] DeepDiffs v1.1.0
  [31c24e10] Distributions v0.21.1 #kf/discretenormal (https://github.com/Keno/Distributions.jl)
  [7a1cc6ca] FFTW v1.0.1
  [587475ba] Flux v0.9.0
  [65ccaadd] FourierTransforms v0.0.0 #master (https://github.com/JuliaComputing/FourierTransforms.jl)
  [0c68f7d7] GPUArrays v1.0.4 #master (https://github.com/JuliaGPU/GPUArrays.jl.git)
  [8d0d7f98] GaloisFields v0.4.0 #master (https://github.com/tkluck/GaloisFields.jl.git)
  [3e1990a7] Hecke v0.6.5 #kf/discard2 (https://github.com/Keno/Hecke.jl)
  [7869d1d1] IRTools v0.3.0 #master (https://github.com/MikeInnes/IRTools.jl.git)
  [aa1ae85d] JuliaInterpreter v0.7.4
  [7475f97c] Mods v0.1.0
  [2edaba10] Nemo v0.15.1 #master (https://github.com/Nemocas/Nemo.jl.git)
  [6fe1bfb0] OffsetArrays v0.11.1
  [f27b6e38] Polynomials v0.5.3
  [27ebfcd6] Primes v0.4.0+ #master (https://github.com/JuliaMath/Primes.jl.git)
  [09ab397b] StructArrays v0.4.0 #kf/fhe (https://github.com/Keno/StructArrays.jl)
  [700de1a5] ZygoteRules v0.2.0 #master (https://github.com/FluxML/ZygoteRules.jl.git)
  [37e2e46d] LinearAlgebra
  [9a3f8284] Random

(ToyFHE) pkg> st -m
Project ToyFHE v0.1.0
Status `/gpfs_home/daluthge/Desktop/ToyFHE.jl/Manifest.toml`
  [c3fe647b] AbstractAlgebra v0.7.1 #master (https://github.com/Nemocas/AbstractAlgebra.jl.git)
  [621f4979] AbstractFFTs v0.4.1
  [1520ce14] AbstractTrees v0.2.1
  [79e6a3ab] Adapt v1.0.0
  [7d9fca2a] Arpack v0.3.1
  [15f4f7f2] AutoHashEquals v0.2.0
  [fbb218c0] BSON v0.2.4
  [9e28174c] BinDeps v0.8.10
  [b99e7846] BinaryProvider v0.5.8
  [c3b6d118] BitIntegers v0.2.0 #kf/div (https://github.com/Keno/BitIntegers.jl)
  [fa961155] CEnum v0.2.0
  [3895d2a7] CUDAapi v1.2.0
  [c5f51814] CUDAdrv v4.0.3
  [be33ccc6] CUDAnative v2.5.4
  [da1fd8a2] CodeTracking v0.5.8
  [944b1d66] CodecZlib v0.6.0
  [3da002f7] ColorTypes v0.8.0
  [5ae59095] Colors v0.9.6
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v2.2.0
  [8f4d0f93] Conda v1.3.0
  [3a865a2d] CuArrays v1.3.0 #master (https://github.com/JuliaGPU/CuArrays.jl.git)
  [9a962f9c] DataAPI v1.1.0
  [864edb3b] DataStructures v0.17.5
  [ab62b9b5] DeepDiffs v1.1.0
  [163ba53b] DiffResults v0.0.4
  [b552c78f] DiffRules v0.0.10
  [31c24e10] Distributions v0.21.1 #kf/discretenormal (https://github.com/Keno/Distributions.jl)
  [7a1cc6ca] FFTW v1.0.1
  [1a297f60] FillArrays v0.7.4
  [53c48c17] FixedPointNumbers v0.6.1
  [587475ba] Flux v0.9.0
  [f6369f11] ForwardDiff v0.10.6
  [65ccaadd] FourierTransforms v0.0.0 #master (https://github.com/JuliaComputing/FourierTransforms.jl)
  [0c68f7d7] GPUArrays v1.0.4 #master (https://github.com/JuliaGPU/GPUArrays.jl.git)
  [8d0d7f98] GaloisFields v0.4.0 #master (https://github.com/tkluck/GaloisFields.jl.git)
  [cd3eb016] HTTP v0.8.8
  [3e1990a7] Hecke v0.6.5 #kf/discard2 (https://github.com/Keno/Hecke.jl)
  [7869d1d1] IRTools v0.3.0 #master (https://github.com/MikeInnes/IRTools.jl.git)
  [83e8ac13] IniFile v0.5.0
  [682c06a0] JSON v0.21.0
  [aa1ae85d] JuliaInterpreter v0.7.4
  [e5e0dc1b] Juno v0.7.2
  [929cbde3] LLVM v1.3.2
  [1914dd2f] MacroTools v0.5.2
  [739be429] MbedTLS v0.7.0
  [e89f7d12] Media v0.5.0
  [e1d29d7a] Missings v0.4.3
  [7475f97c] Mods v0.1.0
  [872c559c] NNlib v0.6.0
  [77ba4419] NaNMath v0.3.2
  [2edaba10] Nemo v0.15.1 #master (https://github.com/Nemocas/Nemo.jl.git)
  [6fe1bfb0] OffsetArrays v0.11.1
  [bac558e1] OrderedCollections v1.1.0
  [90014a1f] PDMats v0.9.10
  [69de0a69] Parsers v0.3.8
  [f27b6e38] Polynomials v0.5.3
  [2dfb63ee] PooledArrays v0.5.2
  [27ebfcd6] Primes v0.4.0+ #master (https://github.com/JuliaMath/Primes.jl.git)
  [1fd47b50] QuadGK v2.1.1
  [3cdcf5f2] RecipesBase v0.7.0
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v0.5.2
  [79098fc4] Rmath v0.6.0
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.7.2
  [90137ffa] StaticArrays v0.12.1
  [2913bbd2] StatsBase v0.32.0
  [4c63d2b9] StatsFuns v0.8.0
  [09ab397b] StructArrays v0.4.0 #kf/fhe (https://github.com/Keno/StructArrays.jl)
  [a759f4b9] TimerOutputs v0.5.3
  [9f7883ad] Tracker v0.2.5
  [3bb67fe8] TranscodingStreams v0.9.5
  [30578b45] URIParser v0.4.0
  [81def892] VersionParsing v1.1.3
  [a5390f91] ZipFile v0.8.3
  [700de1a5] ZygoteRules v0.2.0 #master (https://github.com/FluxML/ZygoteRules.jl.git)
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [44cfe95a] Pkg
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [4607b0f0] SuiteSparse
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
Keno commented 4 years ago

I can look into updating the manifest