search

ponylang / pony-tutorial

:horse: Tutorial for the Pony programming language
http://tutorial.ponylang.io
BSD 2-Clause "Simplified" License
310 stars 111 forks source link

[3/?] Add ability to run code examples in the playground: Fetch tutorial snippet from URL by file name #542

Open shaedrich opened 4 months ago

shaedrich commented 4 months ago

See https://github.com/ponylang/pony-tutorial/issues/340

[!WARNING] Will only work, after #544 has been merged. Works in combination with https://github.com/ponylang/pony-playground/pull/205. Contains changes from #544 until db1eb1183d0746535db719cd537ad8c00a9cc1d1

Example: https://deploy-preview-542--pony-tutorial.netlify.app/getting-started/hello-world#the-code

Open questions

Files

Getting Started

Types

Expressions

Reference capabilities

Object capabilities

Generics

Packages

Testing

C-FFI

Gotchas

Appendices

jemc commented 4 months ago

What to do with the non-pony files?

As discussed in the previous PR, please leave those ones out of the code-samples subdirectory - leave out anything we don't plan to test by compiling.

What to do with partial examples that won't run on their own?

We suggest using the ability to show a partial snippet of a code sample for a given set of lines, so that we can show only the partial example, but the outer code (hidden) would make the file in the code-samples directory compilable.

no Main actor found in package 'main'

The same idea applies - we can put actor Main in one of the hidden lines of the code sample file, outside the partial snippet that is shown.

shaedrich commented 4 months ago

What to do with partial examples that won't run on their own?

We suggest using the ability to show a partial snippet of a code sample for a given set of lines, so that we can show only the partial example, but the outer code (hidden) would make the file in the code-samples directory compilable.

no Main actor found in package 'main'

The same idea applies - we can put actor Main in one of the hidden lines of the code sample file, outside the partial snippet that is shown.

Would you like this to be addressed here or in #544?

The problem with wrapping every code sample in an actor and calling it is, that it creates a whole lot of boilerplate code. This could possibly be done by the playground instead in https://github.com/ponylang/pony-playground/pull/205

shaedrich commented 3 months ago

Okay, I tested an implementation into the playground, and while it is possible to implement, it will a) not work in all cases and b) not be versatile enough to be beneficial. However, I don't think, this should be done in #544, since this will be an additional huge amount of work, as every snippet needs to be tested and additional hidden code parts must be determined, separately. Maybe, this could even be done in another PR to keep track of the changes and their progress. Furthermore, technically, some pending draft PRs could be done in the meantime already, then.

SeanTAllen commented 3 months ago

I don't think wrapping should be done by the playground. Having in sample but only displaying some in tutorial seems appropriate. I'm ok with that being a separate PR.

@jemc thoughts?

shaedrich commented 3 months ago

@SeanTAllen I added said PR just in case and listed my progress there alongside some questions: #545

netlify[bot] commented 3 months ago

Deploy request for pony-tutorial pending review.

Visit the deploys page to approve it

Name Link
Latest commit 469dabf4f93db66f034f38441d117d79ef5977d7
jemc commented 3 months ago

additional huge amount of work, as every snippet needs to be tested and additional hidden code parts must be determined, separately

I don't think the snippets need to be tested in terms of verifying their runtime behavior - that would indeed be a huge amount of work.

We just need to validate that they compile successfully.

This was discussed briefly in the sync call.

shaedrich commented 3 months ago

I don't think the snippets need to be tested in terms of verifying their runtime behavior - that would indeed be a huge amount of work.

We just need to validate that they compile successfully.

Well, if they unexpectedly throw an error in the playground, they are no use. But you'll see that reflected in the PR description and comments in #545

jemc commented 3 months ago

unexpectedly throw an error

They won't throw a compilation error, right? That's what we want to test for, so that should indeed be covered by this work.

If you mean a runtime error, well, in idiomatic Pony (and these samples are definitely supposed to be idiomatic Pony), errors are handled so tightly that nothing really "unexpectedly throws an error" at runtime. What you have at most are flaws in program logic,.

It's not impossible that some breaking change in the compiler would cause a logic bug to appear in a code sample, but the probability should be vanishingly unlikely - that's the worst possible kind of breaking change we could make in the compiler or standard libraries, because it would also have that "subtle logic bug" effect on user-land code. We have an extreme aversion to introducing changes that would cause such "subtle logic bugs" so we take every effort we can to ensure that breaking changes in Pony will "fail loudly" at compile time.

Put all this together with the fact that making all these code samples behavior-testable would be a massive amount of work (likely an order of magnitude or more greater than what you've already done), and I just don't believe it's worth the effort, either for you or for people we're asking to write new tutorial content in the future.

In general, testing the runtime behavior of Pony programs for breaking changes should be done using our "runner tests" in the ponyc repo - it's not really a concern of the tutorial repo.

shaedrich commented 3 months ago

My choice of words might not have been ideal in reflecting technical details, however, what people care at the end of the day is:

That's what they will see in some cases while what they expected will be "Hello world". They will feel lied to or at least that nobody bothers if the examples work.

Tutorial users won't necessarily be experts on Pony and might not be able to fix logic errors other contributors committed and nobody checked.

would be a massive amount of work

I've already essentially documented all the expectations in #545 while doing that PR so it's more or less copy and paste. No real additional work to do here.

jemc commented 3 months ago

what people care at the end of the day is:

I agree :+1:

My position is that breaking changes from upstream that cause subtle logic flaws are vanishingly unlikely, so I don't expect that to be a problem in practice. So I think that skipping the work of running the examples (i.e. only compiling them) will still accomplish the goal that "people care at the end of the day", but without the extra burden of needing to create/curate runtime output expectations for every tutorial sample snippet. I think we should only focus on ensuring that the tutorial snippets continue to compile.