JuliaDynamics / Associations.jl

Algorithms for quantifying associations, independence testing and causal inference from data.
https://juliadynamics.github.io/Associations.jl/stable/
Other
154 stars 13 forks source link

CausalityTools should NOT install a plotting package #125

Closed Datseris closed 3 years ago

Datseris commented 4 years ago

At the moment doing ]add CausalityTools installs Plots, GR, FFMPEG, and many other packages. It even installs Documenter (?!?!?):

 Updating `C:\Users\m300808\.julia\environments\v1.3\Project.toml`
  [5520caf5] + CausalityTools v0.3.0
  Updating `C:\Users\m300808\.julia\environments\v1.3\Manifest.toml`
  [13072b0f] + AxisAlgorithms v1.0.0
  [e28b5b4c] + Bootstrap v2.2.0
  [96374032] + CRlibm v0.8.0
  [324d7699] + CategoricalArrays v0.7.7
  [5520caf5] + CausalityTools v0.3.0
  [b1f25513] + CausalityToolsBase v0.7.0
  [861a8166] ↓ Combinatorics v1.0.0 ⇒ v0.7.0
  [d38c429a] + Contour v0.5.1
  [7011085d] + CrossMappings v0.3.4
  [a93c6f00] + DataFrames v0.20.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [e30172f5] + Documenter v0.24.4
  [35a29f4d] + DocumenterTools v0.1.4
  [90fa49ef] + ErrorfreeArithmetic v0.4.1
  [c87230d0] + FFMPEG v0.2.4
  [fa42c844] + FastRounding v0.2.0
  [6a86dc24] + FiniteDiff v2.1.0
  [28b8d3ca] + GR v0.46.0
  [4d00f742] + GeometryTypes v0.7.7
  [09f84164] + HypothesisTests v0.8.0
  [505f98c9] + InplaceOps v0.3.0
  [a98d9a8b] + Interpolations v0.12.8
  [d1acc4aa] + IntervalArithmetic v0.16.3
  [41ab1584] + InvertedIndices v1.0.0
  [5ab0869b] + KernelDensity v0.5.1
  [d3d80556] + LineSearches v7.0.1
  [442fdcdd] + Measures v0.3.1
  [d41bc354] + NLSolversBase v7.6.0
  [6fe1bfb0] + OffsetArrays v1.0.2
  [4536629a] ↑ OpenBLAS_jll v0.3.7+4 ⇒ v0.3.7+5
  [429524aa] + Optim v0.20.0
  [260eed61] + PerronFrobenius v0.6.0
  [ccf2f8ad] + PlotThemes v1.0.1
  [995b91a9] + PlotUtils v0.6.2
  [91a5bcdd] + Plots v0.27.1
  [2dfb63ee] + PooledArrays v0.5.3
  [85a6dd25] + PositiveFactorizations v0.2.3
  [c84ed2f1] + Ratios v0.3.1
  [322a6be2] + Sass v0.1.0
  [3cc68bcd] + SetRounding v0.2.0
  [1277b4bf] + ShiftedArrays v1.0.0
  [992d4aef] + Showoff v0.3.1
  [d5428e67] + Simplices v0.4.1
  [1441a9f6] + StateSpaceReconstruction v0.4.2
  [3eaba693] + StatsModels v0.6.7
  [bd369af6] + Tables v0.2.11
  [c804724b] + TimeseriesSurrogates v0.3.1
  [ea221983] + TransferEntropy v0.4.3
  [dcd9ba68] + UncertainData v0.6.0
  [29a6e085] + Wavelets v0.9.0
  [efce3f68] + WoodburyMatrices v0.5.0
  [9fa8497b] + Future 

Notice also that this dependency list is excluding the dependencies of DynamicalSystems.jl. I think this large, (in fact too large) and many things have to be dropped before 1.0. For example, plotting must absolutely not be installed with this package, as this makes it unusable in clusters (where plotting is typically impossible to install). The plotting can use RecipesBase or be in a different repo altogether, or behind a requires block.

I guess having Documenter as a dependency is a mistake which I'll fix when I make the documenter PR.

It is also evident that parts of the dependencies have not been updated to their 1.0 version, e.g. Combinatorics. I'd recommend to use CompatHelper.jl to make this process easier.

Datseris commented 4 years ago

About TimeseriesSurrogates: are they used in any of the methods or is it installed only because of the documentation (because I can also fix this; in general the documentation production is done in a separate project where you can install whatever you want without affecting the front-end user).

kahaaga commented 4 years ago

At the moment doing ]add CausalityTools installs Plots, GR, FFMPEG, and many other packages. It even installs Documenter (?!?!?):

This should have been taken care of already. As you say, plotting should be handled by RecipesBase.

In the latest tagged version, neither Plots nor Documenter are dependencies. Currently, the last tagged version of the package is v0.9.9, in which the dependencies are as follows

[deps]
CausalityToolsBase = "b1f25513-c649-51ee-8a1c-0e98d01ae897"
Conda = "8f4d0f93-b110-5947-807f-2305c1781a2d"
CrossMappings = "7011085d-2e9b-5a7b-b0c2-f102c05e50c2"
DelayEmbeddings = "5732040d-69e3-5649-938a-b6b4f237613f"
Distances = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
DynamicalSystems = "61744808-ddfa-5f27-97ff-6e42cc95d634"
HypothesisTests = "09f84164-cd44-5f33-b23f-e6b0d136a0d5"
Interpolations = "a98d9a8b-a2ab-59e6-89dd-64a1c18fca59"
NearestNeighbors = "b8a86587-4115-5ab1-83bc-aa920d37bbce"
PerronFrobenius = "260eed61-d0e8-5f1e-b040-a9756a401c97"
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
RecipesBase = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
Requires = "ae029012-a4dd-5104-9daa-d747884805df"
SimpleDiffEq = "05bca326-078c-5bf0-a5bf-ce7c7982d7fd"
Simplices = "d5428e67-3037-59ba-9ab1-57a04f0a3b6a"
StateSpaceReconstruction = "1441a9f6-6a74-5418-a591-cdf1d78a07f0"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
TimeseriesSurrogates = "c804724b-8c18-5caa-8579-6025a0767c70"
TransferEntropy = "ea221983-52f3-5440-99c7-13ea201cd633"
UncertainData = "dcd9ba68-c27b-5cea-ae21-829cd07325bf"

In your update message it says that v0.3.0 was installed. I'm not sure why it isn't upgrading to the latest version.

kahaaga commented 4 years ago

About TimeseriesSurrogates: are they used in any of the methods or is it installed only because of the documentation (because I can also fix this; in general the documentation production is done in a separate project where you can install whatever you want without affecting the front-end user).

TimeseriesSurrogates is installed because it is included used by the cross mapping routines. However, I have realised that this is unnecessary - the user can do this while preparing the input data - so was planning to remove this option from crossmap and convergentcrossmapping. This reminds me to do so, so I'm adding that to the 1.0 milestone!

However, surrogate testing will explicitly be part of a high-level causality test at some points, so it still needs to be included then. But for now, I think it is sufficient to just remove any explicit dependencies on TimeseriesSurrogates by updating CrossMappings.

Datseris commented 4 years ago

However, surrogate testing will explicitly be part of a high-level causality test at some points, so it still needs to be included then. But for now, I think it is sufficient to just remove any explicit dependencies on TimeseriesSurrogates by updating CrossMappings.

Packages used for only for testing should not be part of the normal dependency tree. For example, have a look here: https://github.com/JuliaDynamics/ChaosTools.jl/blob/master/Project.toml at the extras . OrdinaryDiffEq is used for testing, but is not a dependency of ChaosTools and thus it is not installed when a user adds it. it is only necessary when running the tests.

kahaaga commented 4 years ago

Packages used for only for testing should not be part of the normal dependency tree.

Sorry, I didn't mean testing as in julia code tests. Surrogates are used for hypothesis testing in the context of time series causality analysis. For example, one way to tell if transfer entropy (TE) is significant or not, for example the nonzero-TE in the non-causal direction we discussed in #122, is to compare the obtained TE value with that obtained for an ensemble of surrogate time series.

If there are any dependencies used only for testing left, they are remnants from some old version. In the latest tagged version (v0.9.9), I think I should have gotten rid of all testing-related packages.

kahaaga commented 3 years ago

All plotting is removed in v1 (#129).