Adding a pen trace

[humbug99]

I'd like to have a pen so that my team members can see what the robot did (and maybe be able to measure the length of pen segments). I'm thinking something like pen.down(), pen.up(), and pen.setColor()...

I see that this is in the TODO list already.

Can you give me some guidance as to where to start with this?

Thanks!

[QuirkyCort]

Unfortunately, I don't know of any good ways to do it. Some possible approaches are...

1) Use lines (https://doc.babylonjs.com/babylon101/parametric_shapes) You can use a ray to detect where the pen hits a surface, and use that to draw a line. You would probably want to use some kind of adaptive algorithm (see Ramer–Douglas–Peucker algorithm) to minimize the number of line segments. Problem is that a Babylon.js line is always one pixel wide. Ok if you just want a visual record of the robot's movement, but it's not pretty.

2) Spray decals (https://doc.babylonjs.com/how_to/decals) The robot can spray decals as it moves. These are "spots" not "lines", but it'll look the same if close enough. This will require a lot of spots, and it's probably a terrible idea performance wise.

3) Draw meshes. You can draw a sequence of rectangular meshes, joined end to end. If we are only concerned with drawing on a single flat ground, this is only slightly harder than the previous two methods. If we want the pen to work in any orientation, on any surfaces, and be able to handle cases where the part of the line crosses the edge of an object, then this is very difficult to do.

[humbug99]

Thanks for the ideas!

I have been playing with some ways to do this, with the pen always on. So far, the best that I have come up with is to keep a list of robot positions, and in Robot.render, to draw a vertically oriented Ribbon using a path constructed from the list of robot positions. The Ribbon is only one line wide, so hard to see from the top, but quite nice when seen from above at an angle. It works very nicely with your measurement ruler. (I think this is equivalent to your method #3.)

I was using the body position, but it turns out better to use the position of the center of the wheel axis so that there isn't a strange arc in the path in spin turns.

I'm mostly interested in FLL, so I'd be content with having this work on a flat surface.

Where should I put the definitions of the pen methods down/up/setColor in order to make them available to python, and how do I connect to javascript? (does this need to go in simPython.js?)

[QuirkyCort]

Sounds like you got it! To make it available in Python, edit simPython.js. Basically, you'll need to create a Python class for your pen, but the class is actually written in js. See the Motor class in simPython.js for an example.

For me, the user's program are not expected to access the classes in simPython.js directly, instead, they'll use the ev3dev2 wrapper (eg. ev3dev2/motor.py). This makes it easier to add new wrappers (eg. pybricks). There's no need for you to do the same, and you may want to access the pen class in simPython.js directly.

A more general approach would be to create a new "pen" component, fire a ray from that, and draw using the hit point as a reference. This will let the user play around with different pen position.

I would also draw the mesh flat on the surface (...or to be exact, just slightly above the surface to prevent z-fighting) rather than vertically. If doing it this way, the path length won't necessarily be the same, so a ribbon may not be appropriate. If we add the mesh to the RTT render list, we can then create a scenario with multiple robots detecting each other's lines (Tron?).

[humbug99]

Edit: solved It turns out that my problem was due to browser caching: code loaded by Sk.read() is cached by my browser (Firefox) in such a way that reloading the page does not reload the cached code. You can ignore the following. I'm leaving it here in case knowing about the caching is useful to someone else...

I'm trying to put in the python wrapper and glue, but am having some problems and could use some advice:

I added a Pen class in simPython.js, but I'm getting

AttributeError: 'module' object has no attribute Pen on line 6 (in the simulator console)

where line 6 is the line in the (python) Pen class init method that calls into simPython:

    self.pen = simPython.Pen()

If I replace this line with 'pass', then from python I can import pen.py and do pen = Pen(), and the simulation works, but obviously I can't call any of the Pen methods. I assume that something went wrong in the declaration of the mod.Pen in simPython, but not so badly wrong as to break the other classes in there.

1) Is there somewhere else I need to declare what gets exported from simPython? 2) Any thoughts on how to debug this? 3) In the mod.ColorSensor init, there is a reference to robot. How does simPython have a (js) robot object to use? (And what happens if there are multiple robots?)

I'll attach the current version of simPython.js in case you want to look at it...

Thanks!

[QuirkyCort]

The query string (eg. "?v=1601017657") at the end of each filename in skulpt.js is for dealing with the caching issue. Change the number whenever the file is modified. I'm using unix time, but you can use any numbers; it's only for breaking cache and has no other purposes.

[humbug99]

That's good to know. (I don't do much javascript!) Do you have some script tool for updating all the numbers with the current time when you modify one of the files?

[QuirkyCort]

Yes, but it's not in the repository. It's a bash script, and probably won't work on Windows or Mac.

TIME=$(date +%s)
FILES='index.html arena.html arenaFrame.html configurator.html js/blockly.js js/skulpt.js'

pushd $(pwd)/../public/

for a in $FILES; do
  echo -n Updating $a...
  sed -r "s/\?v=[0-9]+/\?v=$TIME/g" < $a > versioning.Temp
  cat versioning.Temp > $a
  echo Done
done

echo Removing temp files
rm -f versioning.Temp

popd

[humbug99]

I opened a pull request for my initial implementation. Let me know what you think.

With respect to the version numbers in the html / skulpt.js:

• I added a debug flag to skulpt.js that puts the current time in the request rather than the version string, for debug convenience. This means that I can just ignore the caching issue during development without needing to change the version numbers.

• I'm using Ubuntu, so I bumped the version numbers of everything in the html files etc. using your script before my commit/push, but this may not be the right answer, because it causes merge conflicts if the version numbers have also changed in the upstream repo. I think maybe it would be better if I don't bump the version numbers from now on, but you tell me what you want.

[QuirkyCort]

During development, you should just disable cache in your browser's developer's console. No need for a special flag.

No preferences for the version numbering. I'm ok with either way. Eventually, I should update the script to use a hash rather than the time. That should minimize conflicts.

I've taken a quick look through your PR, and it's good work. I'll leave some comments for you in the PR. I'll rather have the pen as a component that the user can add, so I'll reject your PR for now.

[humbug99]

Thanks for the help and comments. A few questions:

• Would you be willing to accept at PR that includes the pen as a component but doesn't improve on pen drawing efficiency, with the understanding that we'd both like to improve the efficiency going forward? (As a babylon novice, I think that it could require quite a bit of experimentation to test possible improvements.)

• With regard to rebuilding the ribbon: It seems to me that the following options are possible:

1. Fully rebuild the ribbon at each time step (current implementation). This could be improved by caching the Material, and also by reducing the number of facets in the ribbon by increasing the distance between pathArray x,y points used for creating the mesh path. The current implementation with animate='penUp' isn't as good of a visualization, but only draws the ribbon once per path segment.

2. Use mesh merge to merge each new position/facet into the mesh. Might save on computation of normals, but it seems to me that this would be problematic because it might cause the Babylon library to reallocate memory for positions and indicies every time and then copy them over to the new memory.

3. Use a custom mesh and vertexData.applyToMesh(customMesh) at each timestep. I suspect that this will also have the reallocation problem, and likely will recompute the normals at each timestep, so I'm not really convinced that this will be an improvement (assuming a relatively efficient implementation of createMesh). (The doc for vertexData.applyToMesh does not say that the arrays in the VertexData object will be used internally by the mesh rather than copied into internal objects in the mesh, so I don't want to assume anything...)

4. Dynamically change the ribbon as points are added. This requires constant length of ribbon in pathArray positions. I tried repeating positions as a way of artificially padding the array to a given length, but that caused an error. (It might be possible to insert extra intermediate x,y steps in a path in order to make it longer in positions, I haven't tried this yet). Once the ribbon reaches a certain length (in x, y points), we could make the ribbon seem to stretch while remaining the same length in positions by skipping some x,y points when filling the pathArray (more at each step). This doesn't solve what to do for the first few timesteps, but maybe some kind of exponential rebuild could happen: rebuild at lengths 1, 2, 4, 8, 16, and so on, so that as the path length increases from 16 thru 31, the ribbon stays at length 16, but we skip x,y points in building the pathArray that will be used for the ribbon rebuild. This avoids most of the reallocations, but still has the problem of the copy and recomputation of normals.

I don't know enough about Babylon to know what's expensive. It seems like you think option 3 would be better than option 1, can you explain why?

[humbug99]

I opened a new pull request that implements the pen as a robot component. This doesn't do much to change the drawing efficiency, aside from reusing the Material where possible.

I did some experimenting in a BJS playground with dynamically changing the ribbon points while keeping the length constant (related to my option #4 described above). I think this could help the frame rate. https://playground.babylonjs.com/#2IWT8Q#5

[QuirkyCort]

Merged

[QuirkyCort]

@humbug99 FYI. I've made some changes to the pen code. Shortened it a bit, and improved efficiency by avoiding recalculation of ribbon path. The pen was also made into an independent component, so we can now have multiple pens. The pen will now need to be added via the robot configurator like the other components, and a port needs to be specified when using it.

The animate and orientation options were dropped. Not sure if there's much use to animate, and orientation my be better served by making the pen rotatable.

I'm gonna leave this in github only for a while before moving it to the main site.