Closed TimUntersberger closed 4 years ago
What you want to do seems to be: you want to be able to call a Rhai function (or closure) inside a registered Rust function.
Check out:
Beware this is low-level API intended for advanced users only.
Alternatively, if what you want is simply to store a closure into some state that you return into Rust, where you'd want to execute that Rhai closure later on in Rust...
In that case you'll want to be able to do two things:
Store a reference to the closure into your Rust struct - you can store the function pointer as FnPtr
type, or you can call the name
method and only store the name of the function, because all closures are actually anonymous functions.
Create a Rust closure from that function name. See https://schungx.github.io/rhai/engine/func.html
And yes, when creating that Rust closure you'll be wrapping the AST and Rhai Engine inside.
Thank you!
I am now trying to save a closure that just calls the rhai function, but I am now hitting problems with the lifetime. My first thought was to just have an engine that lives for the whole program, but because the engine doesn't implement Sync
I can't really send it between threads safely.
I am not really sure how I should solve this.
let fp = val.cast::<FnPtr>();
let name = fp.fn_name().to_string();
action.cb = Some(Box::new(move || {
let engine = ENGINE.lock().unwrap();
engine.consume(&format!("Fn(\"{}\").call()", name));
}));
You can use the sync
feature to make Engine
Send + Sync
.
I am not really sure how I should solve this.
See the example in https://schungx.github.io/rhai/engine/func.html
It creates a Rust closure that you can store. The closure wraps up the Engine
and AST
.
The idea is that you don't create one Engine
for the whole program. You create many Engine
instances. Each of your closures will be wrapping a completely stand-alone copy of the Engine
, plus the AST
.
Creating an Engine
instance is relatively cheap. The only problem is: it is not extremely cheap, so if you do it in a tight loop, you may want to further optimize. For example, you should wrap up all your functions in a custom Package
. Packages are shared so cloning is extremely cheap. Then you can simply Engine::load_package
your custom package and the StandardPackage
with a raw Engine
. This way, engine instantiation is extremely cheap.
If you really want to keep one execution Engine
for the entire program, you can build your own closure. Use Engine::call_fn
to call the Rhai function.
See the examples in https://schungx.github.io/rhai/engine/call_fn.html
I can't really use the sync
feature, because then the recommended way here doesn't work anymore.
let cfg = config.clone(); //Rc<RefCell<Config>>
engine.register_custom_syntax(....);
See the example in https://schungx.github.io/rhai/engine/func.html
The problem is that the function is a property of a map.
Example
#{
actions: [#{
text: "Update",
cb: || {
print("Updating")
}
}]
}
How can I extract the function using create_from_script
if it is contained in a map?
Edit:
I fixed the problem by using Arc<Mutex<Config>>
instead of Rc<RefCell<Config>>
.
I am now calling the function using
engine.consume(&format!("Fn(\"{}\").call()", name));
function body
print("hello world");
but I don't see the hello world
in the output. Is this expected or did I mess something else up?
Edit:
Using
engine.call_fn::<(), ()>(&mut *scope, &*ast, &name, ());
works, but this requires a lot more work. Is it possible for me to use consume
?
works, but this requires a lot more work. Is it possible for me to use
consume
?
I don't see how this can be more work than engine.consume(&ast, ...)
where you'd need to dynamically generate the script in the first place.
Also don't forget your error handling.
but I don't see the
hello world
in the output. Is this expected or did I mess something else up?
Do you get an error evaluating the script? If not, then it should have run successfully.
If you don't see the output, maybe something has eaten it?
A good way is to change some value in your shared object and then test whether the value has changed.
I don't see how this can be more work than engine.consume(&ast, ...) where you'd need to dynamically generate the script in the first place.
Yeah you are right.
I probably had an error, but I didn't see it because I didn't handle the error. Everything is working now.
Thank you
Hello, I'd like to do a very similar thing to this:
#{
actions: [#{
text: "Update",
cb: || {
print("Updating")
}
}]
}
But I'd also like to pass variables from Rust into the Rhai callback and mutate them so the changes are visible in Rust-land.
I've been trying various solutions (more like variations of the same idea) using RefCell<Rc<StructIWantToMutate>>
, but I haven't managed to get anything to work.
What's an (the?) idiomatic way to achieve such a result?
Do I need to push the values I want to mutate into a scope? If so, will the changes to the values in the scope be reflected in my Rust-owned values? Usually I'd assume the answer is no since the values have to be cloned before being pushed into the scope, but what if the values are shared pointers? From my understanding cloning those simply creates another reference to the value stored inside them, so in theory they should be accessible and mutable from Rhai.
Here's a small example of an attempt at implementing my design:
let code = r##"
#{
name: "A",
description: "B",
cost: 1,
health_added: 0,
action: |p1, p2| { p1 += 1 }
}
"##;
let engine = Engine::new();
let mut scope = Scope::new();
let ast = engine.compile_expression_with_scope(&mut scope, code).unwrap();
let res = engine.eval_ast_with_scope::<Map>(&mut scope, &ast).unwrap();
scope.push("", res.clone());
let p1 = Rc::new(RefCell::new(0));
let p2 = Rc::new(RefCell::new(0));
let mut f = move |p1: Rc<RefCell<i32>>, p2: Rc<RefCell<i32>>| {
let action_dyn = res.clone().get("action").unwrap().clone();
let action_ptr = action_dyn.cast::<FnPtr>();
let name = action_ptr.fn_name();
engine.call_fn::<(Rc<RefCell<i32>>, Rc<RefCell<i32>>), ()>(&mut scope, &ast, name, (p1, p2));
};
println!("{},{}", *p1.borrow(), *p2.borrow()); // 0,0
let _ = f(p1.clone(), p2.clone());
println!("{},{}", *p1.borrow(), *p2.borrow()); // 0,0
I asume borrow_mut
should come into play somewhere here to enable mutation of p1
and p2
, but RefMut
doesn't implement Clone
so it's impossible to pass RefMut
arguments to a Rhai function, which in turn means Rhai can't modify the arguments passed into the function.
Sorry if it's a beginner issue, I can understand if you tell me to rtfm but I'd appreciate any help that I can get.
Well, unfortunately I'm going to tell you to RTFM... The Rhai Book has an entire section on advanced patterns showing how to implement exactly what you need.
Essentially you're on the right track. Use Rc<RefCell<T>>
to pass clones of these into a function. borrow_mut
it in order to mutate it.
Also check out the side_effects.rs
test.
Sorry for bumping this again, but are you sure this is doable with a Rhai closure?
All the examples use named functions that are registered from Rust, have their logic written in it and are then used from Rhai to control the Rust program.
For my use case I don't register the functions myself so I can't have, say, anon$cfb82eba22e3a439
, borrow its arguments mutably and operate on them.
Another issue that arises is what exactly is this operation that's performed on the borrowed arguments - I want the arguments passed into the Rhai closure to be dynamically modifiable from said closure, and no matter how hard I try to think about it I can't find a way of doing so.
Again, in the book and the test you referred to the behavior is hard-coded in Rust and only its timing and logic (when/whether to call each Rust function) is determined by the user writing Rhai, whereas I want the behavior to be dynamically determined using a Rhai closure.
I could use return values instead of mutation (I'll probably do it if what I'm trying to do right now doesn't work out) but I'd rather have my API use mutation as it's more concise for my use case.
Thanks in advance for your time.
To Rhai, an Rc<RefCell<T>>
is just another data type. It implements Clone
and that's all Rhai cares. To Rhai, it is no different from i64
.
You can use it everywhere just as you use any other data type.
But if you want to mutate it inside a script, you'll need to register an API for your data type. Rhai doesn't understand borrow_mut
. Nor does it understand that it is an Rc
-wrapped type. It doesn't know anything except that it is a type that implements Clone
.
Therefore, you have to provide your own API abstraction to this type for Rhai. Methods and properties etc.
Some of the examples use Engine::register_get
to register a property getter for a type, for instance. After this you can use my_type.prop
syntax to manipulate that type.
The best way to do it is to create your data type (you can type-def an Rc<RefCell<T>>
, or you can make it struct NewType<T>(Rc<RefCell<T>>)
. Then you call Engine:;register_type_with_name
to register the new type. Then you can register your API on top of it.
Check out the Book section on custom types.
Let's say in your example:
// Define MyType
pub type MyType = Rc<RefCell<i32>;
let mut engine = Engine::new();
// Register API on MyType
engine
.register_type_with_name::<MyType>("MyType")
.register_get_set("data",
|p: &mut MyType| *p.borrow(),
|p: &mut MyType, value: i32| *p.borrow_mut() = value)
.register_fn("+=", |p1: &mut MyType, p2: MyType| *p1.borrow_mut() += *p2.borrow())
.register_fn("-=", |p1: &mut MyType, p2: MyType| *p1.borrow_mut() -= *p2.borrow());
let engine = engine; // Make engine immutable
let code = r##"
#{
name: "A",
description: "B",
cost: 1,
health_added: 0,
action: |p1, p2| { p1 += p2 }
}
"##;
let ast = engine.compile_expression(code)?;
let res = engine.eval_ast::<Map>(&ast)?;
// Make closure
let f = move |p1: MyType, p2: MyType| {
let action_ptr = res["action"].clone().cast::<FnPtr>();
let name = action_ptr.fn_name();
engine.call_fn::<_, ()>(&mut Scope::new(), &ast, name, (p1, p2))?;
};
// Test closure
let p1 = Rc::new(RefCell::new(0));
let p2 = Rc::new(RefCell::new(0));
let _ = f(p1.clone(), p2.clone());
I haven't checked that this works, but you get the general idea.
Thanks for the answer, the type and method registration seem to be the missing pieces in my idea.
However, upon getting your example to compile it seems like f
doesn't modify p1
(tested by printing p1.clone().borrow()
), and I'm stumped as to why.
I've also integrated the general idea of registering a shared pointer to a struct and the struct's methods into a more complex code base and there the engine didn't manage to find the Rhai closure when I called it by name from the Rust closure, but that error is probably on my end so I'll focus on getting interior mutability to work across the language border first.
However, upon getting your example to compile it seems like
f
doesn't modifyp1
(tested by printingp1.clone().borrow()
), and I'm stumped as to why.
Don't forget your error handling. Did you check if call_fn
returns an error?
You're right, call_fn
returns a ErrorFunctionNotFound("anon$...")
in both your example and my library code, I simply forgot to use the question mark operator on the call to f
so the program exited with status 0.
Ah ma bad. Engine::compile_expression
does not support closures nor functions. You need to use Engine::compile
.
Probably should be a parse error.
EDIT: In the next version, this will be a parse error instead of generating a phantom function that doesn't exist.
I see, thanks for making it clear. It's great to hear the error will be clearer in the future.
~Will closure support ever be added to the _expression
functions?~ Thinking about it this can't work due to how closures are implemented.
Apart from the pesky #
that's required in map literals Rhai seems to be a perfect fit for the json-with-closures type configuration format I'm looking for, like Lua but with better Rust integration and it has prettier closure literals.
I think for my use case simply compiling/evaluating the code regularly (not in expression mode) would work fine, but if I have to add a semicolon to the end of every map it'd be a bit annoying as I plan on having one map per file so the end of the configuration is implicitly the EOF. If it's an issue, perhaps this can be solved using custom syntax?
Leaving out the semicolon at the end should be OK.
If you are sure your text strings will never have {
in them, then you can simply global-replace {
with #{
.
I have a few new questions regarding this topic.
Example
let test = "aa";
save_callback(|| {
print(test); // doesn't work with this line. If I remove this line it works
});
save_callback
engine.register_fn("save_callback", |fp: FnPtr| {
let name = fp.fn_name().to_string();
// ...
});
If I call this FnPtr
when it uses automatic currying it doesn't find the function.
engine.call_fn::<(), ()>(&mut scope, &ast, fn_name, ()).unwrap();
I thought that the curry keyword returns a new function pointer? Shouldn't I then be able to call the new function without any arguments?
I actually noticed that you can't import a module inside a closure. Would it be possible to add this?
Example
let fp = || {
import "nog/http" as HTTP;
HTTP::get("http://worldclockapi.com/api/json/utc/now");
};
Note: this is not important for me if Automatic currying modules
would work.
It would be really cool if modules would also get automatically curried inside anonymous functions.
Example
import "nog/http" as HTTP;
let fp = || {
HTTP::get("http://worldclockapi.com/api/json/utc/now");
};
I don't know whether this is related to Automatic currying with callback
. If this is not supported yet, I can open a seperate Issue regarding this.
I thought that the curry keyword returns a new function pointer? Shouldn't I then be able to call the new function without any arguments?
No, you can't call the function directly. Currying works by silently providing arguments to functions. The functions takes a single argument test
which the curried function pointer carries.
EDIT: You need to use the FnPtr::call_dynamic
method to call the function pointer. This takes care of currying.
I actually noticed that you can't import a module inside a closure. Would it be possible to add this?
Hhhmmm... never tried it like this. I think it should work. What is the error you're getting?
EDIT: Yes this is a bug that prevents it from working. It'll be fixed.
UPDATE: The latest version on master fixes this. Will show up in 0.18.3
.
Automatic currying modules
It may be possible... but a bit clumsy. I'll look into that.
Apart from the pesky
#
that's required in map literals Rhai seems to be a perfect fit for the json-with-closures type configuration format I'm looking for, like Lua but with better Rust integration and it has prettier closure literals.
If you intend to make many of these closures, do not simply instantiate an engine with Engine::new()
and then register methods each time. That would be very inefficient if done enough times.
There are a few options that you can use in this case:
1) Define a custom package containing all your functions. Create this package and the StandardPackage
once. Then just Engine::new_raw()
to create a raw Engine
and Engine::load_package
them. Packages are shared so they can be cloned very cheaply.
2) Wrap a single Engine
inside a Arc<Mutex>
or Rc<RefCell>
. If you're evaluating these closures in parallel, this Engine
instance may be a bottleneck.
If I use FnPtr::call_dynamic
instead I get the following error:
Function not found: 'callback (Fn)'
Usage
let test = "asfas";
callback(||{
print(test);
});
I guess you did not provide the correct context when you call call_dynamic
. This is advanced usage.
Check out the section in the Book on passing a callback to a Rust function: https://schungx.github.io/rhai/rust/register-raw.html
let mut engine = Engine::new();
#[allow(deprecated)]
engine.register_raw_fn(
"save_callback",
&[TypeId::of::<FnPtr>()],
|engine: &Engine, lib: &Module, args: &mut [&mut Dynamic]| {
let fn_ptr = std::mem::take(args[0]).cast::<FnPtr>();
fn_ptr.call_dynamic(engine, lib, None, [])
},
);
engine.eval(
r#"
let test = "aa";
save_callback(|| {
print(test);
});
"#,
)?;
OMG haha. I guess I really needed some sleep. I called engine.call_fn_dynamic
instead ... Sorry.
My exact use case is that I save the name of the function in a struct that gets called later called in the program whenever the user wants.
Flow
I don't think I can construct the FnPtr
later on based on the name right?
For more content I save the engine, scope and AST reference globally.
I don't think I can construct the
FnPtr
later on based on the name right?
Yes, you can. Fn("my_func")
constructs a function pointer based on function name. You can also create a FnPtr
from a String
.
A closure is nothing but syntactic sugar. It desugars to Fn("fn_name").curry(argument)...
It still doesn't find the function
This is how I am calling the function
let engine = ENGINE.lock().unwrap();
let mut scope = SCOPE.lock().unwrap();
let ast = AST.lock().unwrap()
let _ = engine
.eval_expression::<FnPtr>(&format!("Fn(\"{}\")", fn_name))
.unwrap()
.call_dynamic(&*engine, &*ast, None, []);
.map_err(|e| error!("{}", e.to_string()));
This is how I get the function name
engine.register_fn("callback", |fp: FnPtr| {
let name = fp.fn_name().to_string();
...
})
Config
let test = "asfas";
callback(||{
print(test);
});
Error
Function not found: 'anon$bfb1131363253114'
As I mentioned, when you curry a function pointer (which is what closures are doing when they capture external variables), you create more parameters on the function. These extra parameters are hidden to you.
Therefore, you'd think anon$bfb1131363253114
has no parameters (because of the ||
), but it actually is like this:
fn anon$bfb1131363253114(test) {
print(test);
}
Therefore you're not finding it because anon$bfb1131363253114
expects one argument and you passed it none. If fact, it expects the captured test
variable which resides in the curried part of the original function pointer. Although the test
variable went out of scope after the end of the script, it stays around if it is captured by the closure, and lives as long as the function pointer is alive.
What you'll need is to keep the original function pointer. That's the only way you can use a closure from Rust with captured variables. Because, otherwise, you'll lose the values of the captured variables.
let cb: Arc<RwLock<Option<FnPtr>>> = Arc::new(RwLock::new(None));
let cb_clone = cb.clone();
engine.register_fn("callback", move |fp: FnPtr| {
// Store the function pointer outside
*cb_clone.lock().unwrap() = Some(fp);
});
let fp_ptr = cb.lock().unwrap().as_ref().unwrap();
let _ = fn_ptr.call_dynamic(&*engine, &*ast, None, [])?;
I thought about saving the FnPtr
, but then I can't derive PartialEq
. Would it be possible to make the FnPtr
derive PartialEq
? If not I'll hack around this problem.
You can't because each FnPtr
is different with different curried variables.
What is the need to compare FnPtr
instances? I don't think any programming language allows you to compare two closures with each other...
I have a massive enum that now needs to have a FnPtr
. I already know how to hack around this problem, so don't worry.
#[derive(Display, Clone, PartialEq, Debug)]
pub enum KeybindingType {
CloseTile,
MinimizeTile,
ResetColumn,
ResetRow,
Quit,
ChangeWorkspace(i32),
ToggleFloatingMode,
ToggleMode(String),
ToggleWorkMode,
IncrementConfig(String, i32),
DecrementConfig(String, i32),
ToggleConfig(String),
MoveWorkspaceToMonitor(i32),
ToggleFullscreen,
Launch(Command),
Focus(Direction),
Resize(Direction, i32),
Swap(Direction),
Callback(String), // <--- here
MoveToWorkspace(i32),
Split(SplitDirection),
}
I don't think any programming language allows you to compare two closures with each other...
Well, since a pointer is just a memory address, if two FnPtr
point to the same function they are equal. I don't really need this, it would have been nice to have.
For reference a stackoverflow answer
C 2011 (N1570 Committee Draft) 6.5.9 6: “Two pointers compare equal if and only if … both are pointers to the same … function …. So, yes, two pointers to the same function compare equal.
Well, since a pointer is just a memory address, if two
FnPtr
point to the same function they are equal. I don't really need this, it would have been nice to have.
True, but a closure is more than a memory address. A closure is a function pointer plus captured context. The captured context can still be compared, but only if their content is comparable.
Your point would be correct for a pure FnPtr
with no captures.
If you want to make Callback(FnPtr)
comparable, you can create a wrapper newtype:
#[derive(Debug, Clone)]
struct FnPtrWrapper(FnPtr);
impl PartialEq<FnPtrWrapper> for FnPtrWrapper {
fn eq(&self, other: &Self) -> bool {
// Two FnPtr are equal if their function names are equal
self.0.fn_name().eq(other.0.fn_name())
}
}
Automatic currying modules
It may be possible... but a bit clumsy. I'll look into that.
Any updates regarding this? Can I help somehow?
If you want to make Callback(FnPtr) comparable, you can create a wrapper newtype:
Damn, that's really clever!
Any updates regarding this? Can I help somehow?
Yes, you're welcome to take a whack at this! Modules are not loaded in the same stack as variables, that's why it may be difficult, as you cannot simply pass it into the function as a parameter.
Originally a module can be the value of a Dynamic
value and that makes this simple. However, it has since been separated out of normal values (and variables) in order to avoid borrow-checker issues.
Damn, that's really clever!
Not quite... :-D In Rust, when you have a problem, the first solution everybody suggests is a "newtype".
I uses this trick to make a FloatWrapper
type which implements Hash
so Dynamic
can be Hash
.
EDIT: Also, implement Deref
on your wrapper derferencing to the original type and Bob's your uncle!
The following works. It may be close to what you need:
let engine = Engine::new();
let mut ast = engine.compile(
r#"
let test = "hello";
|x| test + x
"#,
)?;
// Save the function pointer together with captured variables
let fn_ptr = engine.eval_ast::<FnPtr>(&ast)?;
// Get rid of the script, retaining only functions
ast.retain_functions(|_, _, _| true);
// Closure 'f' captures: the engine, the AST, and the curried function pointer
let f = move |x: INT| fn_ptr.call_dynamic(&engine, ast, None, [x.into()]);
assert_eq!(f(42)?.as_str(), Ok("hello42"));
Yes, you're welcome to take a whack at this! Modules are not loaded in the same stack as variables, that's why it may be difficult, as you cannot simply pass it into the function as a parameter.
Is there a reason why the import
statement isn't just syntactic sugar for something like this:
// import "test" as test;
let test = import("test");
Could we maybe introduce an import
function that does this and rewrite the import statement so that it uses the function?
Another thing I thought about is having a global list of imported modules.
Example
import "test" as TEST; //is global
fn f() {
import "test2" as TEST2; //is not global
print(TEST::message); //works
print(TEST2::message); //works
}
print(TEST2::message); //doesn't work
Another thing I thought about is having a global list of imported modules.
This will go against Rhai's fundamental design - there is ever only one scope. To avoid searching through a scope chain.
Therefore, nothing in functions can access any scope outside.
Is there a reason why the import statement isn't just syntactic sugar for something like this:
It used to. The old implementation has modules being a possible value in a variable.
However, modules are always read-only and immutable, while variables are mutable. Always having to take a mutable reference to a module goes afoul of the borrow checker in a lot of places.
That's because you'll need the following from the variables stack (i.e. the Scope):
1) a mutable reference to the this
object,
2) a reference to the module so functions can search within its scope.
and you need them simultaneously. And the borrow checker doesn't like that. Eventually you end up cloning all modules just to pass them around.
However, modules are always read-only and immutable, while variables are mutable. Always having to take a mutable reference to a module goes afoul of the borrow checker in a lot of places.
Could we use constants instead? I know that they can't be expression, but what if we cheat a bit?
Could we use constants instead? I know that they can't be expression, but what if we cheat a bit?
Not easily. You need a mutable reference to the current scope, that means it is not easy to have another reference to something else within that scope. That's the limit of Rust.
A solution is to make everything within the Scope reference-counted with interior mutability. That'll make performance suck because most variables access are not going to be modules.
Another solution is to simply clone all the modules required (so you don't need a reference to the scope). However, that'll make performance suck for modules.
After giving this some more thought, I think I will actually disable automatic currying, because this makes anonymous functions inconsistent with normal functions. I no longer need the "curry module" stuff, but I will think about having modules as variables/constants a bit more and if I find a good solution I will open a seperate issue.
Thank you for everything!
Great. I suppose the same thing. For config purposes, it is rarely necessary to capture variables.
You can pull in Rhai with no_closure
to disable capturing. Then you can simply store the function name.
Are you OK with this issue? Can we close this?
If you are asking me, then yes I am okay with closing this. I didn't want to close this as @omrisim210 could still have some problems.
@omrisim210 are you OK with closing this?
I never managed to do what I wanted to do but I'm okay with closing this.
I was originally planning on having a Rust library that can read Rhai files to populate its data structures that include callbacks, but seeing as this is rather complex I ended up removing Rhai from the library.
I'll use Rhai in an application that uses the library which I believe should make things somewhat simpler.
Hey, I am trying to register a function that takes a rhai function as parameter and saves it in a struct, so that I can later call it. I am currently very confused and I don't know what I should do to achieve my goal.
Should I save the AST of the function and later evaluate it?
Setup
Usage
Thank you!