sunjay / turtle

Create Animated Drawings in Rust
http://turtle.rs
Mozilla Public License 2.0
559 stars 54 forks source link

Docs images helper #190

Open JosephLing opened 3 years ago

JosephLing commented 3 years ago

For https://github.com/sunjay/turtle/issues/179 this implements generating the svgs from the doc tests using docs_image feature flag.

There was one main slight complication in that a few examples required the user to click on the window. For those tests which where originally no_run I split them up into separate doc tests but hopefully they should all be one coherent code example...

sunjay commented 3 years ago

@JosephLing Sorry about the late reply. I really wanted to make sure I get you unstuck, so I checked out the code locally and did some digging starting on the master branch. The patch below works and does everything up to the actual SVG image generation itself. You should just have to finish off the SVG to PNG conversion, test it out, add a call to save_docs_image to the rest of the doctests that need it, and commit the updated images. Please keep in mind my review comments about not adding anything to doctests that require the window titlebar or require the turtle to be drawn. We can deal with those later.

One of the issues is in order for cfg flags to work they are separate for RUSTFLAGS and RUSTDOCFLAGS so that means you need both set if you want the doctests to run the cfg(docs_images).

Having to set both RUSTFLAGS and RUSTDOCFLAGS is not actually a big downside here. Keep in mind that this command is only meant to be used occasionally (and only by me). It's okay if the command is a bit unruly because we can document it in CONTRIBUTING.md. Cargo features are exposed to the user and they don't actually solve the whole problem anyway--I'll explain more below.

A Better Starting Point

A lot of the issues you ran into were because of my own incomplete understanding of some of Rust's features and cargo. Sorry about that! I've taken the time to fix that and I'm hoping that the patch below will be a much better starting point for your work on this.

The first detail I missed was that dev-dependencies are only available for the actual test files themselves, the crate itself is not recompiled with the dev-dependencies. That means the only way to get this to work is to list turtle_docs_helpers in the [dependencies] section. This is troublesome because we don't want this crate to be visible to users of the turtle crate. We also can't publish a crate to crates.io if it has a path dependency.

Solution: have a commented out line in [dependencies] with an explanation of what's going on. This allows us to manually run the command to generate the images without forcing us to have a permanent path dependency in the [dependencies] section. The reason using cargo features wouldn't solve this problem is because even with a cargo feature, you'd still need to list the path dependency in [dependencies] section. There is no getting around that. The only difference is passing --features ... instead of setting environment variables. However, as explained above, that difference isn't important here.

You can run the code with:

# Don't forget to uncomment the `turtle_docs_helpers` dependency in Cargo.toml
RUSTDOCFLAGS="--cfg docs_images" RUSTFLAGS="--cfg docs_images" cargo test --features "test unstable" --doc

On windows, you may need to set the environment variables at the front of the command in advance, then run the rest of the command starting with cargo test ....

diff --git a/Cargo.toml b/Cargo.toml
index 1dcfbca..b103548 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -51,6 +51,11 @@ cfg-if = "0.1"

 parking_lot = "0.10"

+# This dependency should be uncommented when rebuilding the images in the API
+# documentation. In all other cases this dependency cannot be used since having
+# a path-only dependency would make it impossible to publish this crate.
+#turtle_docs_helpers = { path = "turtle_docs_helpers" }
+
 [dependencies.futures-util]
 version = "0.3"
 default-features = false
diff --git a/src/async_drawing.rs b/src/async_drawing.rs
index 42931ea..49176fe 100644
--- a/src/async_drawing.rs
+++ b/src/async_drawing.rs
@@ -188,3 +188,10 @@ impl AsyncDrawing {
         self.client.debug_drawing().await
     }
 }
+
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for AsyncDrawing {
+    fn save_docs_svg(&self, path: std::path::PathBuf) {
+        self.client.save_docs_svg(path);
+    }
+}
diff --git a/src/async_turtle.rs b/src/async_turtle.rs
index 2602292..77f1e74 100644
--- a/src/async_turtle.rs
+++ b/src/async_turtle.rs
@@ -324,3 +324,10 @@ impl AsyncTurtle {
         self.client.debug_turtle(self.id, self.angle_unit).await
     }
 }
+
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for AsyncTurtle {
+    fn save_docs_svg(&self, path: std::path::PathBuf) {
+        self.client.save_docs_svg(path);
+    }
+}
diff --git a/src/drawing.rs b/src/drawing.rs
index 014e9eb..6548edf 100644
--- a/src/drawing.rs
+++ b/src/drawing.rs
@@ -629,6 +629,13 @@ impl Drawing {
     }
 }

+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for Drawing {
+    fn save_docs_svg(&self, path: std::path::PathBuf) {
+        self.drawing.save_docs_svg(path);
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
diff --git a/src/ipc_protocol/protocol.rs b/src/ipc_protocol/protocol.rs
index 6e13beb..df9a9b3 100644
--- a/src/ipc_protocol/protocol.rs
+++ b/src/ipc_protocol/protocol.rs
@@ -408,3 +408,12 @@ impl ProtocolClient {
         }
     }
 }
+
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for ProtocolClient {
+    fn save_docs_svg(&self, path: std::path::PathBuf) {
+        use crate::sync_runtime::block_on;
+        block_on(self.export_svg(path))
+            .expect("unable to save docs image");
+    }
+}
diff --git a/src/turtle.rs b/src/turtle.rs
index 5019456..a1e1388 100644
--- a/src/turtle.rs
+++ b/src/turtle.rs
@@ -535,7 +535,7 @@ impl Turtle {
     ///
     /// # Example
     ///
-    /// ```rust,no_run
+    /// ```rust
     /// use turtle::Turtle;
     ///
     /// fn main() {
@@ -557,6 +557,7 @@ impl Turtle {
     ///     turtle.set_pen_color("#4CAF50"); // green
     ///     turtle.set_pen_size(100.0);
     ///     turtle.forward(200.0);
+    ///     # #[cfg(docs_images)] turtle_docs_helpers::save_docs_image(&turtle, "pen_thickness");
     /// }
     /// ```
     ///
@@ -891,6 +892,13 @@ impl Turtle {
     }
 }

+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for Turtle {
+    fn save_docs_svg(&self, path: std::path::PathBuf) {
+        self.turtle.save_docs_svg(path);
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
diff --git a/turtle_docs_helpers/Cargo.toml b/turtle_docs_helpers/Cargo.toml
new file mode 100644
index 0000000..4a6a129
--- /dev/null
+++ b/turtle_docs_helpers/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "turtle_docs_helpers"
+version = "0.1.0"
+authors = ["Sunjay Varma <varma.sunjay@gmail.com>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
diff --git a/turtle_docs_helpers/src/lib.rs b/turtle_docs_helpers/src/lib.rs
new file mode 100644
index 0000000..40a9937
--- /dev/null
+++ b/turtle_docs_helpers/src/lib.rs
@@ -0,0 +1,17 @@
+use std::path::{Path, PathBuf};
+
+/// Saves the image being drawn as an SVG and panics if an error occurs
+///
+/// This is different from the `save_svg` method on `Drawing` and `AsyncDrawing`
+/// because this is only meant to be used for automation and may need to access
+/// internal APIs.
+pub trait SaveDocsSvg {
+    fn save_docs_svg(&self, path: PathBuf);
+}
+
+/// Saves the currently drawn image to `docs/assets/images/docs/{output_name}`
+pub fn save_docs_image<T: SaveDocsSvg>(drawing: &T, output_name: &str) {
+    let svg_path = Path::new("docs/assets/images/docs").join(output_name).with_extension("svg");
+    drawing.save_docs_svg(svg_path);
+    todo!()
+}

Notes:

Please make sure you document in CONTRIBUTING.md that you need to uncomment the turtle_docs_helpers dependency before running the command shown above.


I think if you base your work on this, you should be able to get it working. Please don't hesitate to let me know if you have any more questions!

sunjay commented 3 years ago

@JosephLing I saw your messages on Zulip today and sent a reply. (Sorry for the delay!)

Could you push the latest version of your code so I can see the issues you ran into trying to make PNGs from the SVG files?

sunjay commented 3 years ago

@JosephLing I just finished looking at the latest set of changes. Thanks for continuing to work on this! 😄

I noticed that the approach you're currently taking has deviated pretty significantly from what we've been discussing so far. Did you get a chance to look at the patch from my previous comment? The reason that approach works well is that it keeps this functionality largely separate and independent from the rest of the crate. That's important because these changes are just to automate a part of the crate maintanence. None of this should significantly alter any of the code that belongs to the main crate.

Could you take a look at that patch again and base your changes off of the approach taken there? You should be able to take the usvg and resvg code that you added to save_svg and move it into the save_docs_image function. After that, you just need to add calls to save_docs_image in the various doctests. The patch has an example of how that call should look. Please make sure you only add it to doctests where the updated image is still approximately the same as the previous image.

Note: The SaveDocsSvg trait and save_docs_image function are intentionally designed to make all of this as easy and clean as possible:

Hopefully that explains a few of the choices I made in structuring the code. Sorry if this stuff wasn't clear before!

If you run into any major issues, I'm happy to help you figure out how to work around them, but in general I think that approach should work very well. I'm also hoping that now that you've gotten it working once it will not be a huge amount of effort for you to move your code over. Let me know if you have any questions! 😁

JosephLing commented 3 years ago

I saw the previous comment but wanted to try and go for a more automated approach so tried this out. But I will work on putting it back to the other way and making sure that:

The SaveDocsSvg trait and save_docs_image function are intentionally designed to make all of this as easy and clean as possible

Yep that was happening with save_svg so I will make sure to future proof that

The trait method is named something that will never clash with another method on Turtle or Drawing

For save_svg changing how the opacity is saved to the svg will be required in order to create a png. The alternative would be to try and patch usvg (and maybe resvg might need to be patched as well) or just implement a save_png method that we can use instead (although dependant on the implementation of save_png it might require the same change still).

For the images generated are they good? as if they are all okay they can be added to the docs by having them reference the master branch. One future issue/difficulty might be versioning the docs although I imagine there won't be any major changes or too many people stuck on a version.

sunjay commented 3 years ago

I saw the previous comment but wanted to try and go for a more automated approach so tried this out. But I will work on putting it back to the other way

Thanks! At this point, it may be easiest to start from master, apply the patch, and then copy/paste the usvg and resvg code from this PR into save_docs_image. Here are the commands that should help you do that:

# Start on the docs_images_helper branch
git checkout docs_images_helper

# Backup your work
git branch docs_images_helper_backup

# Switch to master
git checkout master

# Add an upstream remote if you don't have one already
git remote add upstream https://github.com/sunjay/turtle.git

# Update to the latest master
git pull upstream master

# Remove the current version of the feature branch (don't worry, it is backed up!)
git branch -D docs_images_helper

# Create a new version of the feature branch based on the updated master
git checkout -b docs_images_helper

# Apply the patch
git apply pr190.patch

# ...finish the work...

# Push the branch
git push -f

This assumes you have the patch saved in a file called pr190.patch in the turtle directory.

Alternatively, I've pushed the changes onto a pr190 branch in this repo. You can avoid dealing with patch files by starting from that:

# See above for explanations of these commands
git checkout docs_images_helper
git branch docs_images_helper_backup
git checkout master
git remote add upstream https://github.com/sunjay/turtle.git
git pull upstream master
# Important: delete the previous copy of the branch (it should be backed up to docs_images_helper_backup)
git branch -D docs_images_helper

# Fetch all the branches from upstream (git pull does this, so you might not need it)
git fetch upstream

# Checkout the pr190 branch but name it `docs_images_helper`
git checkout -b docs_images_helper upstream/pr190

# ...finish the work...

# Push the branch
git push -f

Yep that was happening with save_svg so I will make sure to future proof that

Thanks! I like the save_docs_svg name because it makes it clear what this is for and we know that no name like that will ever be used in this crate.

For save_svg changing how the opacity is saved to the svg will be required in order to create a png. The alternative would be to try and patch usvg (and maybe resvg might need to be patched as well) or just implement a save_png method that we can use instead (although dependant on the implementation of save_png it might require the same change still).

Let's push this work to a future PR. I want to land this as soon as possible with just a basic amount of functionality. The nice thing about this is that it's hidden behind a cfg flag, so we don't have to worry about getting it 100% perfect right away. My main concern is setting up a good foundation so this integrates well into the rest of the code and so that we can make future changes without having to re-do all of the work.

For the images generated are they good? as if they are all okay they can be added to the docs by having them reference the master branch. One future issue/difficulty might be versioning the docs although I imagine there won't be any major changes or too many people stuck on a version.

Yes, this is definitely extra tricky because docs.rs doesn't support image hosting at all. You'll notice that all of the images are referenced by their commit ID in the docs. We can work on solving that problem later. The hardest part is regenerating the images, so let's get that right first.

For the images generated are they good?

The images are good, though please do make sure you only add this for examples that don't need the turtle shown. I also noticed while reviewing that some of the images seem to be incorrect. It's worth checking if you accidentally forgot to add a call before a wait_for_click or something.

GitHub has a rich diff mode that should make the incorrect images very clear:

image

sunjay commented 3 years ago

Hey @JosephLing, anything I can do to help you make progress with this? No worries if you won't have time to work on it anymore. We can take the awesome amount of progress you've made so far and try to finish it some other way if it comes to that. From the last comment I left it looks like there's just a few more steps to go! Appreciate all the time that you've put into this!

JosephLing commented 3 years ago

Thanks for reaching out @sunjay, life and job hunting have consumed my time/energy recently. I might be able to work on it in a few weeks. But more realisticly if someone wants to take it over I can be around to help on zulip chat 👍

Thanks for all the help! and yep it's close to be finished 😃

sunjay commented 3 years ago

No worries @JosephLing. I also went through a bunch of life and job hunting things recently, it can be really rough. 😰 Hope everything works out well for you!

If I get a chance to work on this or if someone else volunteers, we'll pass on the work, but otherwise you are welcome to come back and finish this off anytime. Thanks again for all the time/effort you've put in so far! 🎉