search

scicloj / tablecloth.time

Tools for the processing and manipulation of time-series data in Clojure.
Other
18 stars 1 forks source link

Add ability to optionally return indexes instead of subset of dataset for `slice` and `slice-index` #7

Closed ezmiller closed 3 years ago

ezmiller commented 3 years ago

Goal/Purpose

We sometimes want to just have the indexes when we slice.

Solution

This PR allows you to pass an options hash to both slice-index and slice to get the row indexes for the slice/subset instead of a dataset. That looks like:

(slice-index ds 1 10 {:result-type :as-indexes})

This syntax matches that used by tablecloth, e.g. for group-by: https://scicloj.github.io/tablecloth/index.html#Group-by

How to test

Run the new automated tests in your emacs editor if possible. (For some reason lein test is not working right now, but we can fix it in a separate PR.)

daslu commented 3 years ago

Regarding making lein test work, see this PR: https://github.com/scicloj/tablecloth.time/pull/11

daslu commented 3 years ago

Tried locally:

Then lein test works.

daslu commented 3 years ago

Except for that, the PR looks great to me.

rsiddharthan commented 3 years ago

I can pick up this thread tomorrow to review the PR.

On Mon, Feb 1, 2021 at 7:10 AM Daniel Slutsky notifications@github.com wrote:

Except for that, the PR looks great to me.

— You are receiving this because your review was requested. Reply to this email directly, view it on GitHub https://github.com/scicloj/tablecloth.time/pull/7#issuecomment-770810537, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR2HB3TEGF7KCPLBU5RVBTS42KZ3ANCNFSM4W4DZJXA .

ezmiller commented 3 years ago

Tried locally:

* change `slice_test.clj` so that the namespace matches the file name

* move `slice_test.clj` and `index_test.clj` under the `test/tablecloth/time` directory

Then lein test works.

@daslu I think I fixed these issues that you mentioned re namespace naming and file location for the test files (in this commit), but I'm still getting this very strange error:

Exception in thread "main" java.lang.RuntimeException: No such var: tech.v3.datatype.export-symbols/requiring-resolve, compiling:(tech/v3/datatype/functional.clj:269:1)
        at clojure.lang.Compiler.analyze(Compiler.java:6792)
        at clojure.lang.Compiler.analyze(Compiler.java:6729)
        at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3813)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:7005)
        at clojure.lang.Compiler.analyze(Compiler.java:6773)
        at clojure.lang.Compiler.access$300(Compiler.java:38)
        at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6368)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:7003)
        at clojure.lang.Compiler.analyze(Compiler.java:6773)
        at clojure.lang.Compiler.analyze(Compiler.java:6729)
        at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6100)
        at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5460)
        at clojure.lang.Compiler$FnExpr.parse(Compiler.java:4022)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:7001)
        at clojure.lang.Compiler.analyze(Compiler.java:6773)
        at clojure.lang.Compiler.eval(Compiler.java:7059)
        at clojure.lang.Compiler.eval(Compiler.java:7052)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:3894)
        at tech.v3.datatype$eval476$loading__6434__auto____477.invoke(datatype.clj:1)
        at tech.v3.datatype$eval476.invokeStatic(datatype.clj:1)
        at tech.v3.datatype$eval476.invoke(datatype.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at tablecloth.api$eval470.invokeStatic(api.clj:5)
        at tablecloth.api$eval470.invoke(api.clj:5)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:482)
        at tablecloth.time.api.slice_test$eval437$loading__6434__auto____438.invoke(slice_test.clj:1)
        at tablecloth.time.api.slice_test$eval437.invokeStatic(slice_test.clj:1)
        at tablecloth.time.api.slice_test$eval437.invoke(slice_test.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$apply.invoke(core.clj:652)
        at user$eval317.invokeStatic(form-init8936462340375699323.clj:1)
        at user$eval317.invoke(form-init8936462340375699323.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7052)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.Compiler.loadFile(Compiler.java:7452)
        at clojure.main$load_script.invokeStatic(main.clj:278)
        at clojure.main$init_opt.invokeStatic(main.clj:280)
        at clojure.main$init_opt.invoke(main.clj:280)
        at clojure.main$initialize.invokeStatic(main.clj:311)
        at clojure.main$null_opt.invokeStatic(main.clj:345)
        at clojure.main$null_opt.invoke(main.clj:342)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: java.lang.RuntimeException: No such var: tech.v3.datatype.export-symbols/requiring-resolve
        at clojure.lang.Util.runtimeException(Util.java:221)
        at clojure.lang.Compiler.resolveIn(Compiler.java:7273)
        at clojure.lang.Compiler.resolve(Compiler.java:7243)
        at clojure.lang.Compiler.analyzeSymbol(Compiler.java:7204)
        at clojure.lang.Compiler.analyze(Compiler.java:6752)
        ... 141 more
Tests failed.
daslu commented 3 years ago

@ezmiller interesting. I am using that commit, and all tests pass!

ezmiller commented 3 years ago

@ezmiller interesting. I am using that commit, and all tests pass!

Well must be something in my environment then. I'll somehow try to refresh the dependencies or something.

ezmiller commented 3 years ago

Funny, something strange is definitely going on with my computer. This repo fails on my computer with the same error when I run lein test, but when I ran it on a different machine, it was fine.

In any case, I'll merge this PR since this seems to be a problem on my machine.

daslu commented 3 years ago

@ezmiller interesting.

Some things that may affect the local behaviour are: