Consider evaluating source-code docstring tests in their namespace

[MIJOTHY]

Typically the examples you find in docstrings refer to the form they are defined in. However, currently to test a docstring, you need to require the namespace in that docstring. For example,

(ns example)

(defn fn5
  "
  ```Clojure
  (fn5 1)
  => 2

" [x] (inc x))

produces

clj -M:isolated/kaocha generated --- generated (clojure.test) --------------------------- ERROR ERROR in generated (example_cljc_test.clj:27) Failed loading tests: Exception: clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: Unable to resolve symbol: fn5 in this context, compiling:(example_cljc_test.clj:27:24)

Whereas the following

````clojure
(ns example)

(defn fn5
  "
  ```Clojure
  (require '[example :refer [fn5]])
  (fn5 1)
  => 2

" [x] (inc x))

produces 

clj -M:isolated/kaocha generated --- generated (clojure.test) --------------------------- example-cljc-test block-0001 doc/example.cljc - line 5 - fn5 docstring

1 tests, 1 assertions, 0 failures.

I think to make doctesting more ergonomic, doctests should automatically have the current namespace's vars brought into scope.

[lread]

Thanks for the feedback @MIJOTHY. I very much appreciate the time you take to make the issues you raise easy to understand.

For me personally, I like including the require because I am explaining how a public API would be typically used. It might be wordier but it is complete and unambiguous. There are many examples from the rewrite-clj node API.

Is the API you are describing prescribing end-users use :refer to bring in API vars?

[MIJOTHY]

Is the API you are describing prescribing end-users use :refer to bring in API vars?

Ideally the API wouldn't make any prescriptions around how users bring in vars. The end-result of what I'm describing would be consumers of the test-doc-blocks library being able to write functions with docstrings like

(defn fn5
  "
  ```Clojure
  (fn5 1)
  => 2

" [x] (inc x))

and these would be testable without needing any additional `require`s from the user. Though the option to `require ... :as` and use that alias would still be present. I'll give an example solution to elucidate the sort of behaviour I mean, though I haven't considered the implications too fully so I don't necessarily suggest going down this route:

### Example solution

For docstrings in source code (`.clj`, `.cljc`, `.cljs`), 
* Tests are generated as they are now
* An additional `require` is added to the generated test ns, which `:refer :all`s the tested namespace

#### `example.cljc`
````clj
(ns example)

(defn fn-public
  "
  We can omit any `require`s if we are referring to vars in the current namespace

  ```Clojure
  (fn-public 1)
  => 2

" [x] (inc x))

(defn- fn-private " We can refer to private vars in the current namespace (no we can't but it would be a nice-to-have)

  (fn-private 1)
  => 2

" [x] (inc x))

(defn fn-public-2 " But we can require if we want to

  (require '[example :as ex])
  (ex/fn-public-2 1)
  => 2

" [x] (fn-private x))

#### `target/test-doc-blocks/test/example_cljc_test.clj`

````clj
(ns example-cljc-test
  (:require clojure.test
            clojure.string
            lread.test-doc-blocks.runtime
            [example :refer :all] ;; automatically injected
            [example :as ex]))

(clojure.test/deftest block-0001
  (clojure.test/testing  "doc/example.cljc - line 7 - fn-public docstring"
    (clojure.test/is (= '2 (fn-public 1)))))

(clojure.test/deftest block-0002
  (clojure.test/testing  "doc/example.cljc - line 19 - fn-private docstring"
(clojure.test/is (= '2 (fn-private 1))))) ;; Doesn't actually work, as we're not using `in-ns` or anything fancy like that

(clojure.test/deftest block-0003
  (clojure.test/testing  "doc/example.cljc - line 30 - fn-public-2 docstring"
nil
(clojure.test/is (= '2 (ex/fn-public-2 1)))))

(defn test-ns-hook [] (block-0001) (block-0002) (block-0003))

Hope that helps clear up the sort of thing I'm thinking of? Essentially having docstrings be tested as if the tests are defined in the same namespace, or at least in a context where requiring the namespace the form is defined in is unnecessary.

[lread]

Thanks @MIJOTHY!

When I wrote:

I very much appreciate the time you take to make the issues you raise easy to understand.

I meant your original issue was nice and clear. That said, your follow-up is equally clear too.

The implementation is not the tricky part. Deciding to bring a feature into test-doc-blocks is the tricky part, because yes, as you know, is forever! 🙂

To make sure I got my point across on the rewrite-clj API docstrings, I'll elaborate. As I write the docstrings, I am doing my best to sit in the shoes of the API user. The docstring is describing how the end-user would typically use the rewrite-clj API. For rewrite-clj this is not typically via any :refer :all usage.

That is why I asked you about how you think the consumers of your library typically use your fns, and if you thought they were typically going to use :refer to bring in vars.

But... I might have missed your use case. Maybe including markdown doc blocks in your docstrings is more for the tests they generate than as end-user documentation?

And without thinking too hard, I'm wondering about conflicts. What if your namespace included some fns that have the same name as some of the clojure core fns?

Please be aware that I'm not necessarily disagreeing with you, just exploring with you.

As we work through this, a work-around for you might be to include the following in your namespace docstring:

The examples in this namespace's docstrings assume:
```Clojure
(require '[example :refer :all])
```

[MIJOTHY]

Thanks @MIJOTHY!

When I wrote:

I very much appreciate the time you take to make the issues you raise easy to understand.

I meant your original issue was nice and clear. That said, your follow-up is equally clear too.

Much appreciated.

The implementation is not the tricky part. Deciding to bring a feature into test-doc-blocks is the tricky part, because yes, as you know, is forever! 🙂

To make sure I got my point across on the rewrite-clj API docstrings, I'll elaborate. As I write the docstrings, I am doing my best to sit in the shoes of the API user. The docstring is describing how the end-user would typically use the rewrite-clj API. For rewrite-clj this is not typically via any :refer :all usage.

That is why I asked you about how you think the consumers of your library typically use your fns, and if you thought they were typically going to use :refer to bring in vars.

But... I might have missed your use case. Maybe including markdown doc blocks in your docstrings is more for the tests they generate than as end-user documentation?

Ah yes, I see what you mean now. Our use-case is as follows: there are some internal functions that we document usage of and provide examples for, for developers' sake. So it's a way to have enhanced internal documentation that doesn't get out of sync with the implementation. This notably isn't a substitute for a test suite, which we'd still expect to be written.

And without thinking too hard, I'm wondering about conflicts. What if your namespace included some fns that have the same name as some of the clojure core fns?

Yeh, I think there's a lot of fiddly parts to be considered here, and I'm not entirely sure what the best way of going about this would be. But I suppose the first point of order would be to decide if this is a use-case that you'd like to support in the library.

[lread]

Thanks for explaining your use case. It seems like it is the same as mine.

As far as I can tell, there are not a huge number of test-doc-block users yet. Maybe waiting to see if such a feature is crucial to multiple people would make sense?

In the meantime, does my proposed workaround work for you?

[MIJOTHY]

In the meantime, does my proposed workaround work for you?

Yes somewhat; unfortunately private vars still have to be var-quoted as they aren't in scope by default. I'm fine with waiting to see if others find this useful. Thanks for discussing.

[lread]

unfortunately private vars still have to be var-quoted as they aren't in scope by default

@MIJOTHY I'm wondering why your code block examples would include private vars? These are not part of your public API, right?

[MIJOTHY]

unfortunately private vars still have to be var-quoted as they aren't in scope by default

@MIJOTHY I'm wondering why your code block examples would include private vars? These are not part of your public API, right?

Right, but we don't only document our public API. When I speak about 'internal functions' I'm referring to things that may be private (perhaps I worded this too ambiguously before). The motivation here is to have internal documentation to make it easier for developers working on the project, in the same vein as why you might spec/instrument certain critical internal methods (not just for testing, but also for documentation). Given it can be sensible to have private functions documented, it can also be sensible to provide examples in those private functions to enhance the documentation for collaborators IMO.

[lread]

Ah gotcha, thanks for the clarification, much appreciated!

Onto my next question! 🙂

I'm curious how your developers consume the docstrings. Given that code blocks in docstrings can sometimes require somewhat challenging escaping requirements, do you use markdown rendering to visualize the docstrings? The docstrings I wrote for rewrite-clj are intended to be viewed through cljdoc, but was wondering what your strategy here is.

[MIJOTHY]

I'm curious how your developers consume the docstrings. Given that code blocks in docstrings can sometimes require somewhat challenging escaping requirements, do you use markdown rendering to visualize the docstrings? The docstrings I wrote for rewrite-clj are intended to be viewed through cljdoc, but was wondering what your strategy here is.

Currently just by navigating the source-code. We don't have codox or anything set up yet, but it's something we'd like to do eventually.

[lread]

Heya, I think I'll close this one up for now.

Future reader: please do re-open if you'd like to discuss further.