search

chengchingwen / Transformers.jl

Julia Implementation of Transformer models
MIT License
526 stars 75 forks source link

Transformer not working with NeuralNetDiffEq #13

Closed zzj0402 closed 4 years ago

zzj0402 commented 4 years ago

I am trying to develop a transformer model for NeuralNetDiffEq. Seems like there is a dependency issue: https://stackoverflow.com/questions/59765831/julia-flux-issue-undefvarerror-tracker-not-defined

[ Info: Precompiling Transformers [21ca0261-441d-5938-ace7-c90938fde4d4]
ERROR: LoadError: LoadError: UndefVarError: Tracker not defined
Stacktrace:
 [1] include(::Module, ::String) at .\Base.jl:377
 [2] include(::String) at C:\Users\zzj04\.julia\packages\Transformers\2BZJs\src\Transformers.jl:1
 [3] top-level scope at C:\Users\zzj04\.julia\packages\Transformers\2BZJs\src\Transformers.jl:33
 [4] include(::Module, ::String) at .\Base.jl:377
 [5] top-level scope at none:2
 [6] eval at .\boot.jl:331 [inlined]
 [7] eval(::Expr) at .\client.jl:449
 [8] top-level scope at .\none:3
in expression starting at C:\Users\zzj04\.julia\packages\Transformers\2BZJs\src\fix\batchedmul.jl:3
in expression starting at C:\Users\zzj04\.julia\packages\Transformers\2BZJs\src\Transformers.jl:33
ERROR: Failed to precompile Transformers [21ca0261-441d-5938-ace7-c90938fde4d4] to C:\Users\zzj04\.julia\compiled\v1.4\Transformers\gPQmm_KBlm6.ji.
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1272
 [3] _require(::Base.PkgId) at .\loading.jl:1029
 [4] require(::Base.PkgId) at .\loading.jl:927
 [5] require(::Module, ::Symbol) at .\loading.jl:922
chengchingwen commented 4 years ago

We have switched from Tracker.jl to Zygote.jl in the latest version. You need to make sure NeuralNetDiffEq.jl is compatible with Zygote.jl and the latest version of Flux.

zzj0402 commented 4 years ago

We have switched from Tracker.jl to Zygote.jl in the latest version. You need to make sure NeuralNetDiffEq.jl is compatible with Zygote.jl and the latest version of Flux.

I am new to Julia. How to do that?

dboyliao commented 4 years ago

My suggestion is that you switch to pkg mode (by pressing ]) and then run status to see the exact packages you installed in your environment. It looks like NeuralNetDiffEq.jl is using out-dated Flux api.

ChrisRackauckas commented 4 years ago

The problem is due to https://github.com/chengchingwen/Transformers.jl/blob/master/Project.toml#L33-L37 bounding CUDA packages and Adapt < 2 not allowing any recent Flux version to be used with this package. I added a PR for CompatHelper so this library can get some maintenance help, but this should probably be brought into the same fold as the other ML core infrastructure so that this doesn't happen in the future (and so you can get some help!)

It looks like NeuralNetDiffEq.jl is using out-dated Flux api.

No, Transformers is using the out-dated Flux API.

chengchingwen commented 4 years ago

No, Transformers is using the out-dated Flux API.

Ah, my bad. I will try to update it ASAP

chengchingwen commented 4 years ago

This problem should be fixed with the latest version.