Open JosephLing opened 3 years ago
@JosephLing Sorry about the late reply. I really wanted to make sure I get you unstuck, so I checked out the code locally and did some digging starting on the master branch. The patch below works and does everything up to the actual SVG image generation itself. You should just have to finish off the SVG to PNG conversion, test it out, add a call to save_docs_image
to the rest of the doctests that need it, and commit the updated images. Please keep in mind my review comments about not adding anything to doctests that require the window titlebar or require the turtle to be drawn. We can deal with those later.
One of the issues is in order for cfg flags to work they are separate for
RUSTFLAGS
andRUSTDOCFLAGS
so that means you need both set if you want the doctests to run thecfg(docs_images)
.
Having to set both RUSTFLAGS
and RUSTDOCFLAGS
is not actually a big downside here. Keep in mind that this command is only meant to be used occasionally (and only by me). It's okay if the command is a bit unruly because we can document it in CONTRIBUTING.md
. Cargo features are exposed to the user and they don't actually solve the whole problem anyway--I'll explain more below.
A lot of the issues you ran into were because of my own incomplete understanding of some of Rust's features and cargo. Sorry about that! I've taken the time to fix that and I'm hoping that the patch below will be a much better starting point for your work on this.
The first detail I missed was that dev-dependencies are only available for the actual test files themselves, the crate itself is not recompiled with the dev-dependencies. That means the only way to get this to work is to list turtle_docs_helpers
in the [dependencies]
section. This is troublesome because we don't want this crate to be visible to users of the turtle crate. We also can't publish a crate to crates.io if it has a path dependency.
Solution: have a commented out line in [dependencies]
with an explanation of what's going on. This allows us to manually run the command to generate the images without forcing us to have a permanent path dependency in the [dependencies]
section. The reason using cargo features wouldn't solve this problem is because even with a cargo feature, you'd still need to list the path dependency in [dependencies]
section. There is no getting around that. The only difference is passing --features ...
instead of setting environment variables. However, as explained above, that difference isn't important here.
You can run the code with:
# Don't forget to uncomment the `turtle_docs_helpers` dependency in Cargo.toml
RUSTDOCFLAGS="--cfg docs_images" RUSTFLAGS="--cfg docs_images" cargo test --features "test unstable" --doc
On windows, you may need to set the environment variables at the front of the command in advance, then run the rest of the command starting with cargo test ...
.
diff --git a/Cargo.toml b/Cargo.toml
index 1dcfbca..b103548 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -51,6 +51,11 @@ cfg-if = "0.1"
parking_lot = "0.10"
+# This dependency should be uncommented when rebuilding the images in the API
+# documentation. In all other cases this dependency cannot be used since having
+# a path-only dependency would make it impossible to publish this crate.
+#turtle_docs_helpers = { path = "turtle_docs_helpers" }
+
[dependencies.futures-util]
version = "0.3"
default-features = false
diff --git a/src/async_drawing.rs b/src/async_drawing.rs
index 42931ea..49176fe 100644
--- a/src/async_drawing.rs
+++ b/src/async_drawing.rs
@@ -188,3 +188,10 @@ impl AsyncDrawing {
self.client.debug_drawing().await
}
}
+
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for AsyncDrawing {
+ fn save_docs_svg(&self, path: std::path::PathBuf) {
+ self.client.save_docs_svg(path);
+ }
+}
diff --git a/src/async_turtle.rs b/src/async_turtle.rs
index 2602292..77f1e74 100644
--- a/src/async_turtle.rs
+++ b/src/async_turtle.rs
@@ -324,3 +324,10 @@ impl AsyncTurtle {
self.client.debug_turtle(self.id, self.angle_unit).await
}
}
+
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for AsyncTurtle {
+ fn save_docs_svg(&self, path: std::path::PathBuf) {
+ self.client.save_docs_svg(path);
+ }
+}
diff --git a/src/drawing.rs b/src/drawing.rs
index 014e9eb..6548edf 100644
--- a/src/drawing.rs
+++ b/src/drawing.rs
@@ -629,6 +629,13 @@ impl Drawing {
}
}
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for Drawing {
+ fn save_docs_svg(&self, path: std::path::PathBuf) {
+ self.drawing.save_docs_svg(path);
+ }
+}
+
#[cfg(test)]
mod tests {
use super::*;
diff --git a/src/ipc_protocol/protocol.rs b/src/ipc_protocol/protocol.rs
index 6e13beb..df9a9b3 100644
--- a/src/ipc_protocol/protocol.rs
+++ b/src/ipc_protocol/protocol.rs
@@ -408,3 +408,12 @@ impl ProtocolClient {
}
}
}
+
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for ProtocolClient {
+ fn save_docs_svg(&self, path: std::path::PathBuf) {
+ use crate::sync_runtime::block_on;
+ block_on(self.export_svg(path))
+ .expect("unable to save docs image");
+ }
+}
diff --git a/src/turtle.rs b/src/turtle.rs
index 5019456..a1e1388 100644
--- a/src/turtle.rs
+++ b/src/turtle.rs
@@ -535,7 +535,7 @@ impl Turtle {
///
/// # Example
///
- /// ```rust,no_run
+ /// ```rust
/// use turtle::Turtle;
///
/// fn main() {
@@ -557,6 +557,7 @@ impl Turtle {
/// turtle.set_pen_color("#4CAF50"); // green
/// turtle.set_pen_size(100.0);
/// turtle.forward(200.0);
+ /// # #[cfg(docs_images)] turtle_docs_helpers::save_docs_image(&turtle, "pen_thickness");
/// }
/// ```
///
@@ -891,6 +892,13 @@ impl Turtle {
}
}
+#[cfg(docs_images)]
+impl turtle_docs_helpers::SaveDocsSvg for Turtle {
+ fn save_docs_svg(&self, path: std::path::PathBuf) {
+ self.turtle.save_docs_svg(path);
+ }
+}
+
#[cfg(test)]
mod tests {
use super::*;
diff --git a/turtle_docs_helpers/Cargo.toml b/turtle_docs_helpers/Cargo.toml
new file mode 100644
index 0000000..4a6a129
--- /dev/null
+++ b/turtle_docs_helpers/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "turtle_docs_helpers"
+version = "0.1.0"
+authors = ["Sunjay Varma <varma.sunjay@gmail.com>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
diff --git a/turtle_docs_helpers/src/lib.rs b/turtle_docs_helpers/src/lib.rs
new file mode 100644
index 0000000..40a9937
--- /dev/null
+++ b/turtle_docs_helpers/src/lib.rs
@@ -0,0 +1,17 @@
+use std::path::{Path, PathBuf};
+
+/// Saves the image being drawn as an SVG and panics if an error occurs
+///
+/// This is different from the `save_svg` method on `Drawing` and `AsyncDrawing`
+/// because this is only meant to be used for automation and may need to access
+/// internal APIs.
+pub trait SaveDocsSvg {
+ fn save_docs_svg(&self, path: PathBuf);
+}
+
+/// Saves the currently drawn image to `docs/assets/images/docs/{output_name}`
+pub fn save_docs_image<T: SaveDocsSvg>(drawing: &T, output_name: &str) {
+ let svg_path = Path::new("docs/assets/images/docs").join(output_name).with_extension("svg");
+ drawing.save_docs_svg(svg_path);
+ todo!()
+}
Notes:
Path
to PathBuf
to avoid an unnecessary allocation and since self.client.export_svg
takes PathBuf
anywayPlease make sure you document in CONTRIBUTING.md
that you need to uncomment the turtle_docs_helpers
dependency before running the command shown above.
I think if you base your work on this, you should be able to get it working. Please don't hesitate to let me know if you have any more questions!
@JosephLing I saw your messages on Zulip today and sent a reply. (Sorry for the delay!)
Could you push the latest version of your code so I can see the issues you ran into trying to make PNGs from the SVG files?
@JosephLing I just finished looking at the latest set of changes. Thanks for continuing to work on this! 😄
I noticed that the approach you're currently taking has deviated pretty significantly from what we've been discussing so far. Did you get a chance to look at the patch from my previous comment? The reason that approach works well is that it keeps this functionality largely separate and independent from the rest of the crate. That's important because these changes are just to automate a part of the crate maintanence. None of this should significantly alter any of the code that belongs to the main crate.
Could you take a look at that patch again and base your changes off of the approach taken there? You should be able to take the usvg
and resvg
code that you added to save_svg
and move it into the save_docs_image
function. After that, you just need to add calls to save_docs_image
in the various doctests. The patch has an example of how that call should look. Please make sure you only add it to doctests where the updated image is still approximately the same as the previous image.
Note: The SaveDocsSvg
trait and save_docs_image
function are intentionally designed to make all of this as easy and clean as possible:
save_docs_image
is a free function, so calling it doesn't require you to import a trait in every doctestsave_docs_image
doesn't return Result
because we can't propagate errors in the doctests anyway
Result::expect
)Turtle
or Drawing
save_png
method will be added to Drawing
at some point, so that probably isn't the best choiceturtle_docs_helpers
crate is useful because it means you can add usvg
and resvg
as dependencies without introducing an extra docs_images_save_png
feature like you have nowHopefully that explains a few of the choices I made in structuring the code. Sorry if this stuff wasn't clear before!
If you run into any major issues, I'm happy to help you figure out how to work around them, but in general I think that approach should work very well. I'm also hoping that now that you've gotten it working once it will not be a huge amount of effort for you to move your code over. Let me know if you have any questions! 😁
I saw the previous comment but wanted to try and go for a more automated approach so tried this out. But I will work on putting it back to the other way and making sure that:
The SaveDocsSvg trait and save_docs_image function are intentionally designed to make all of this as easy and clean as possible
Yep that was happening with save_svg
so I will make sure to future proof that
The trait method is named something that will never clash with another method on Turtle or Drawing
For save_svg
changing how the opacity is saved to the svg will be required in order to create a png. The alternative would be to try and patch usvg
(and maybe resvg
might need to be patched as well) or just implement a save_png
method that we can use instead (although dependant on the implementation of save_png it might require the same change still).
For the images generated are they good? as if they are all okay they can be added to the docs by having them reference the master branch. One future issue/difficulty might be versioning the docs although I imagine there won't be any major changes or too many people stuck on a version.
I saw the previous comment but wanted to try and go for a more automated approach so tried this out. But I will work on putting it back to the other way
Thanks! At this point, it may be easiest to start from master, apply the patch, and then copy/paste the usvg
and resvg
code from this PR into save_docs_image
. Here are the commands that should help you do that:
# Start on the docs_images_helper branch
git checkout docs_images_helper
# Backup your work
git branch docs_images_helper_backup
# Switch to master
git checkout master
# Add an upstream remote if you don't have one already
git remote add upstream https://github.com/sunjay/turtle.git
# Update to the latest master
git pull upstream master
# Remove the current version of the feature branch (don't worry, it is backed up!)
git branch -D docs_images_helper
# Create a new version of the feature branch based on the updated master
git checkout -b docs_images_helper
# Apply the patch
git apply pr190.patch
# ...finish the work...
# Push the branch
git push -f
This assumes you have the patch saved in a file called pr190.patch
in the turtle
directory.
Alternatively, I've pushed the changes onto a pr190
branch in this repo. You can avoid dealing with patch files by starting from that:
# See above for explanations of these commands
git checkout docs_images_helper
git branch docs_images_helper_backup
git checkout master
git remote add upstream https://github.com/sunjay/turtle.git
git pull upstream master
# Important: delete the previous copy of the branch (it should be backed up to docs_images_helper_backup)
git branch -D docs_images_helper
# Fetch all the branches from upstream (git pull does this, so you might not need it)
git fetch upstream
# Checkout the pr190 branch but name it `docs_images_helper`
git checkout -b docs_images_helper upstream/pr190
# ...finish the work...
# Push the branch
git push -f
Yep that was happening with
save_svg
so I will make sure to future proof that
Thanks! I like the save_docs_svg
name because it makes it clear what this is for and we know that no name like that will ever be used in this crate.
For
save_svg
changing how the opacity is saved to the svg will be required in order to create a png. The alternative would be to try and patchusvg
(and mayberesvg
might need to be patched as well) or just implement asave_png
method that we can use instead (although dependant on the implementation of save_png it might require the same change still).
Let's push this work to a future PR. I want to land this as soon as possible with just a basic amount of functionality. The nice thing about this is that it's hidden behind a cfg
flag, so we don't have to worry about getting it 100% perfect right away. My main concern is setting up a good foundation so this integrates well into the rest of the code and so that we can make future changes without having to re-do all of the work.
For the images generated are they good? as if they are all okay they can be added to the docs by having them reference the master branch. One future issue/difficulty might be versioning the docs although I imagine there won't be any major changes or too many people stuck on a version.
Yes, this is definitely extra tricky because docs.rs doesn't support image hosting at all. You'll notice that all of the images are referenced by their commit ID in the docs. We can work on solving that problem later. The hardest part is regenerating the images, so let's get that right first.
For the images generated are they good?
The images are good, though please do make sure you only add this for examples that don't need the turtle shown. I also noticed while reviewing that some of the images seem to be incorrect. It's worth checking if you accidentally forgot to add a call before a wait_for_click
or something.
GitHub has a rich diff mode that should make the incorrect images very clear:
Hey @JosephLing, anything I can do to help you make progress with this? No worries if you won't have time to work on it anymore. We can take the awesome amount of progress you've made so far and try to finish it some other way if it comes to that. From the last comment I left it looks like there's just a few more steps to go! Appreciate all the time that you've put into this!
Thanks for reaching out @sunjay, life and job hunting have consumed my time/energy recently. I might be able to work on it in a few weeks. But more realisticly if someone wants to take it over I can be around to help on zulip chat 👍
Thanks for all the help! and yep it's close to be finished 😃
No worries @JosephLing. I also went through a bunch of life and job hunting things recently, it can be really rough. 😰 Hope everything works out well for you!
If I get a chance to work on this or if someone else volunteers, we'll pass on the work, but otherwise you are welcome to come back and finish this off anytime. Thanks again for all the time/effort you've put in so far! 🎉
For https://github.com/sunjay/turtle/issues/179 this implements generating the svgs from the doc tests using
docs_image
feature flag.There was one main slight complication in that a few examples required the user to click on the window. For those tests which where originally
no_run
I split them up into separate doc tests but hopefully they should all be one coherent code example...is_visible
, for the title bar I am not sure the best way to handle it but it is easy to get the data out.