Open danielalcalde opened 1 year ago
That sounds really strange. The compat entries of PastaQ.jl have barely changed between those versions: https://github.com/GTorlai/PastaQ.jl/blob/v0.0.20/Project.toml https://github.com/GTorlai/PastaQ.jl/blob/v0.0.24/Project.toml
so I don't see why you would be seeing differences between those versions, since if something changed in ChainRules or Zygote, PastaQ v0.0.20 and v0.0.24 should be using the same versions of ChainRules or Zygote. From the error messages it looks like a Zygote or ChainRules issue. I have seen regressions in some Zygote operations so I could believe that an operations that we are using in PastaQ used to be differentiable but may not in a or earlier version of Zygote. Are you certain that in the comparison between the cases that work and don't work that you are using the same versions of Zygote and ChainRules? I would try using a clean project environment and doing a minimal installation of the package you need to run your script (PastaQ, ITensors, and Zygote), make sure everything is updated, and then compare between PastaQ v0.0.20 and PastaQ v0.0.24 (and check the versions of ChainRules and Zygote that get installed), as a sanity check.
Another possibility is that we are using some operations in PastaQ v0.0.24 that are no longer differentiable in recent versions of ChainRules/Zygote, but in PastaQ v0.0.20 we aren't using those operations. So you may be detecting a regression in ChainRules/Zygote, and the solution from our side or your side would be to pin ChainRules or Zygote to an older version where PastaQ v0.0.24 works (or detect which operations had a regression and rewrite those parts of the code to be differentiable with the latest versions of ChainRules/Zygote).
I have installed PastaQ in two different projects, one with V0.0.20 and another with v0.0.24. Both projects have identical Manifest files, except for the PastaQ entry and the same behavior I wrote about above is happening. As a sanity check I did the same on another machine with the same effect. This makes me think that the issue is not related to Zygote.
I also tried implementing the ChainRulesCore.ProjectTo function, but it only resulted in more errors.
Implementation:
function (b::ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:elements, :axes), Tuple{Vector{Any}, Tuple{Base.OneTo{Int64}}}}})(a::Tuple{})
return Array{Number}(collect(a))
end
after which it wants me to implement
function (b::ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:elements, :axes), Tuple{Vector{ChainRulesCore.ProjectTo{AbstractArray, NamedTuple{(:elements, :axes), Tuple{Vector{ChainRulesCore.ProjectTo}, Tuple{Base.OneTo{Int64}}}}}}, Tuple{Base.OneTo{Int64}}}}})(a::Tuple{Vector{ChainRulesCore.AbstractTangent}})
which honestly I do not know what it is expecting here.
Alright, so as I expected this is caused by a regression in Zygote. I tried running your code but pinning Zygote to an older version (Zygote v0.6.43, while the latest version is v0.6.60):
julia> ] add Zygote@0.6.43
and then running your code works.
I see the error message is originating from this line: https://github.com/GTorlai/PastaQ.jl/blob/v0.0.24/src/circuits/noise.jl#L193
So somewhere between Zygote v0.6.43 and v0.6.60, there was a regression in differentiating either that line, or some combination of code in the insertnoise
function that it's in. The insertnoise
is quite complicated so I'm not surprised Zygote might be having trouble differentiating it.
Anyway, that's a minimal fix that you can try from your end, let me know if that works. It would be good to know at what version Zygote starting failing to differentiate the insertnoise
function, and then try to raise an issue over at Zygote about it.
Alright, some more information. It looks like Zygote versions v0.6.44 and below work, and v0.6.45 and above fail at this same line of code in insertnoise
.
In the release notes of Zygote v0.6.45:
https://github.com/FluxML/Zygote.jl/releases/tag/v0.6.45
I see this PR was merged and included in Zygote v0.6.45: https://github.com/FluxML/Zygote.jl/pull/1277 that switched the implementation of the derivative rules for hvcat
, hcat
, and vcat
over to ChainRules. I would guess that is causing the issue in differentiating insertnoise
, since it makes use of vcat
.
Yes, you are right, this fixes it. Thank you for the help! I wonder in which way vcat was broken since I used it in my code without issue. If I have time this week I will investigate insertnoise
in more detail to find the unsupported operation.
Thanks! It would be nice to try to make as minimal of a code example as possible to reproduce the bug so then we can report it to Zygote. For example, a starting place could be writing a function that just depends on insertnoise
but doesn't do the circuit evolution, and then try to differentiate that with Zygote. It could be a function where you make a noisy circuit with a certain gate angle and then just extract that gate angle back out of the function. Then we can try to remove any PastaQ-specific functions or types from insertnoise
, which would make it easier to report to Zygote. That shouldn't be too hard do since it is basically just manipulating Vectors of Tuples. I've seen similar kinds of regressions from Zygote before, and it can be quite subtle, for example some specific combination of operations, operations that involve some control flow, etc.
I encountered a peculiar bug where parts of my code that utilize noise have suddenly stopped working, despite the fact that I have not updated PastaQ or any other package. This can be confirmed by referring to my Manifest file, which is tracked by git. Interestingly, rolling back to version 0.0.20 of PastaQ resolves the issue.
I'm using Julia 1.8.5 but observed the same error with julia 1.7.3
Below is a minimal example illustrating the problem, adapted from the official documentation with minor modifications to incorporate noise:
for which I get this error for version 0.0.24
and for versions 0.0.21-0.0.23 I get the error:
Installed packages:
Hope someone can make sense of this.