search

godot-rust / gdext

Rust bindings for Godot 4
https://mastodon.gamedev.place/@GodotRust
Mozilla Public License 2.0
2.99k stars 187 forks source link

Tracking issue for builtin feature-completeness #310

Open lilizoey opened 1 year ago

lilizoey commented 1 year ago

More detailed breakdown of the builtin types than present in #24.

Similar to #209 and #143, there are more builtins missing functions, here's a list of our progress when it comes to implementing all functionality.

The missing functionality here is largely going to be easy to reimplement, unless otherwise specified. We should try to reimplement what we can in rust, but for many types (like for instance the stringy types) it's likely not going to be very easy to do so. In those cases we can simply make a wrapper around a call to the InnerX type, (for instance InnerGString for GString). Which simply makes a call to godot to call that function.

For anything labelled :grey_question: it would be good to check if all the godot methods are implemented yet. Feel free to leave a comment on this post if you do. Keep in mind that not everything is directly implemented as a method in rust, some things for instance are trait implementations, for instance Projection(Transform3D from) was reimplemented as a impl From<Transform3D> for Projection { .. }.

Legend

:red_square: bare-bones, missing almost everything :yellow_circle: most functions re-implemented :green_heart: all planned functions re-implemented, (could still mean there are QOL things we could add) :construction: work already in progress :grey_question: to be confirmed if everything is re-implemented

Stringy Types

Bounding Boxes

Matrices

Vectors

Misc Math

Arrays

Standard Collections

T4rmyn commented 1 year ago

I've been checking the vectors and these is what I've found so far:

lilizoey commented 1 year ago

im working on GodotString and StringName.

lilizoey commented 1 year ago

im working on GodotString and StringName.

I've gotten a bit side-tracked. Feel free to have a look here if someone else wants to continue, https://github.com/lilizoey/gdextension/tree/feature/stringy-types. Though there are some things i'm not sure if should go into a PR as is (like the indexing operators).

0awful commented 8 months ago

Dictionary

Godot Method Rust Method
clear() -> void (mutates) clear(&mut self)
duplicate(deep: bool) -> Dictionary duplicate_deep() -> Dictionary duplicate_shallow() -> Dictionary
erase(key: Variant) -> bool (mutates) remove<K>(&mut self, key: K) -> Option<Variant>
find_key(value: Variant) -> Variant find_key_by_value<V>(value: V) -> Option<Variant>
get(key: Variant, default: Optional<Variant>) -> Variant get<K>(key: K) -> Option<Variant> get_or_nil<K>(k: K) -> Variant
has(key: Variant) -> bool contains_key<K>(key: K) -> bool
has_all(keys: Array<Variant>) -> bool contains_all_keys(key: Array<Variant>) -> bool
hash() -> int hash() -> u32
is_empty() -> bool is_empty() -> bool
is_read_only() -> bool
keys() -> Array<Variant> keys_array() -> Array<Variant>
make_read_only() -> void (mutates)
merge(dictionary: Dictionary, overwrites: Optional<bool>) -> void (mutates) extend_dictionary(&mut self, other: Dictionary, overwrite: bool)
"set" dict[k] = v set<K, V>(&mut self, key: K, value: V)
size() -> int len() -> usize
values() -> Array values_array() -> Array<Variant>
No GDScript Equivalent insert(&mut self) iter_shared keys_shared

Status

The only functions that aren't present are is_read_only()and make_read_only(). All rust functions have the same mutability rules. Rust has three functions that aren't present in gdscript. Information was compiled from gdscript docs and gdext docs

The only different function is erase(). It has a different return type (Option<Variant>) which conveys all the same information and some. It doesn't make sense to reduce it to mirror gdscript exactly.

Remaining Work

If we believe it is worthwhile to copy the read_only semantics of gdscript, then that is all that remains. If such functionality is not something gdext is interested in then dictionaries are complete.

0awful commented 8 months ago

Array

Godot Method Rust Method
array[i] -> ? get(index:usize) -> T
array[i] = V mutates set(index:usize, value:T) mutates
all(Callable) -> bool
any(Callable) -> bool
append(Variant) -> void mutates This is an alias of push_back()
append_array(Array) -> void mutates extend_array(other:Array<T>) mutates
assign(Array) -> void mutates
back() -> Variant last() -> Option<T>
bsearch(Variant, before=true) -> int binary_search(v: &T) -> usize
bsearch_custom(Variant, Callable, before=true) -> int
clear() -> void mutates clear() mutates
count(Variant) -> int count(V:&T) -> usize
duplicate(deep=false) -> Array duplicate_shallow() duplicate_deep()
erase(Variant) -> void mutates erase(v:&T)
fill(Variant) -> void mutates fill(v:&T)
filter(Callable) -> Array Iter
find(Variant, from=0) -> int find(v:&T, from:Option<usize>) -> Option<usize>
front() -> Variant first() -> Option<T>
get_typed_builtin() -> int
get_typed_class_name() -> StringName
get_typed_script() -> Variant
has(Variant) -> bool contains(v:&T) -> bool
hash() -> int hash() -> u32
insert(int, Variant) -> int mutates insert(index:usize, v:T)
is_empty() -> bool is_empty() -> bool
is_read_only() -> bool
is_same_typed(Array) ->bool`
is_typed() -> bool
make_read_only() -> void mutates
map(Callable) -> Array Iter
max() -> Variant max() -> Option<T>
min() -> Variant min() -> Option<T>
pick_random() -> Variant pick_random() -> Option<T>
pop_at(int) -> Variant mutates remove(index:usize) -> T
pop_back() -> Variant mutates pop() -> Option<T>
pop_front() -> Variant mutates pop_front() -> Option<T>
push_back(Variant) -> void mutates push(v:T)
push_front(Variant) -> void mutates push_front(v:T)
reduce(Callable, Variant accum=null) -> Variant Iter
remove_at(int) -> void mutates remove(index:usize) -> T
resize(int) -> int mutates resize(size:usize) -> () mutates
reverse() -> void mutates reverse() -> () mutates
rfind(Variant, int from=-1) -> int rfind(value:&T, from:Option<usize>) -> Option<usize>
shuffle() -> void mutates shuffle() mutates
size() -> int len() -> usize
slice(begin:int, end:int=2147483647, step:int=1, deep:bool=false) -> Array subarray_shallow(begin: usize, end:usize, step:Option<isize>) subarray_deep(begin: usize, end:usize, step:Option<isize>)
sort() -> void sort_unstable() mutates
sort_custom(Callable) -> void
No GDScript Equivalent try_get(index:usize)

Status

Like dictionary there is no is_read_only() and make_read_only().

The "type" methods don't make sense in the context of gdext. Though there are similar things present on variants, so it may be worth bringing it to this level.

all() and any() don't have mirrors within gdext, but can be achieved relatively easily due to the Iter trait. I'm not finding slice() in gdext, though perhaps it is present via a trait. _custom() variants for sort and bsearch don't exist. assign() doesn't have an analogue, but also perhaps shouldn't. assign() doesn't strike me as a very useful method, but maybe my personal belief is irrelevant here.

Remaining Work

slice() bsearch_custom() sort_custom() seem like good candidates.

all() and any() strike me as an example of "won't do". Iter exposes all you need to implement that yourself.

I personally don't see value in assign() but we may want it purely for mirroring gdscript.

I don't know about the typed methods or read_only and would appreciate someone else making a call in that area.

edit: slice() is subarray_shallow() or subarray_deep()

0awful commented 8 months ago

PackedByteArray

Godot Method Rust Method
array[index] -> int get(index:usize) -> u8
append(int) -> bool push(value:u8) "docs call this out explicitly
append_array(PackedByteArray) extend_array(&packedByteArray)
bsearch(value:int, before=true) -> int binary_search(value:u8) -> usize
clear() clear()
compress(compression_mode:int=0) -> PackedByteArray
count(int) -> int count(value:u8) -> usize
decode_double(int) -> float
decode_float(int) -> float
decode_half(int) -> float
decode_s8(int) -> int
decode_s16(int) -> int
decode_s32(int) -> int
decode_s64(int) -> int
decode_u8(int) -> int
decode_u16(int) -> int
decode_u32(int) -> int
decode_u64(int) -> int
decode_var(byte_offset:int, allow_objects:bool=false) -> Variant
decode_var_size(byte_offset:int, allow_objects:bool=false) -> int
decompress(buffer_size: int, compression_mode:int = 0) -> PackedByteArray
decompress_dynamic(max_output_size:int, compression_mode:int=0) -> PackedByteArray
duplicate() -> PackedByteArray probably clone
encode_double(byte_offset:int, value:float)
encode_float(byte_offset:int, value:float)
encode_half(int, float)
encode_s8(int, int)
encode_s16(int, int)
encode_s32(int, int)
encode_s64(int, int)
encode_u8(int, int)
encode_u16(int, int)
encode_u32(int, int)
encode_u64(int, int)
encode_var(int, Variant, bool=false) -> int
fill(int) fill(value:u8)
find(int, int=0) -> int find(value:u8, from:Option<usize>) -> Option<usize>
get_string_from_ascii() -> string
get_string_from_utf8() -> string
get_string_from_utf16() -> string
get_string_from_utf32() -> string
get_string_from_wchar() -> string
has(int) -> bool contains(value:u8) -> bool
has_encoded_var(int, bool=false) -> bool
hex_encode() -> String
insert(int, int) -> int insert(index:usize, value:u8)
is_empty() -> bool is_empty() -> bool
push_back(int) -> bool push(value:u8) "docs call out this explicitly"
remove_at(int) remove(index:usize) -> u8
resize(int) -> int resize(size:usize) -> ()
reverse() reverse()
rfind(int, from:int=-1) -> int rfind(value:u8, from:Option<usize>) -> Option<usize>
set(int, int) set(index: usize, value:u8)
size() -> int len() -> usize
slice(begin:int, end:int=2147483647) -> PackedByteArray subarray(begin:usize, end:usize) ->PackedByteArrayandas_slice()->&[u8]andas_mut_slice()->&mut [u8]`
sort() sort()
to_float32_array() -> PackedFloat32Array
to_float64_array() -> PackedFloat64Array
to_int32_array() -> PackedInt32Array
to_int64_array() -> PackedInt64Array
No Godot Equivalent to_vec() -> Vec<u8>

Status

This is a tough one. It could be far more complete than described here. For example the to_<T>_array() is likely covered by the trait into. Does that also cover encode_<T>()? What then of compress and decompress? I don't know enough to say.

There is also minor strangeness. push() being push_back() and append(). Typically there is a semantic difference there. Begging the question of accuracy of implementation or if naming is appropriate.

Remaining work

I don't know for sure, but it looks like lots.

0awful commented 8 months ago

PackedArray

Outside of packed byte array every godot PackedArray has the same functions and every rust PackedArray has the same methods. Because of that we can generalize here: Godot Method Rust Method
array[index] get(usize) -> <T>
append(<T>) -> bool push<T>
append_array(Packed<T>Array) extend_array(Packed<T>Array)
bsearch(<T>, bool=true) -> int binary_search(<T>) -> usize
clear() clear()
count(<T>) -> int count(<T>) -> usize
duplicate() -> Packed<T>Array clone()
fill(<T>) fill(<T>)
find(<T>, int=0) -> int find(<T>, Option<usize>) -> Option<usize>
has(<T>) -> bool contains(<T>) -> bool
insert(int, <T>) -> int insert(usize,<T>)
is_empty() -> bool is_empty() -> bool
push_back(<T>) -> bool push(<T>)
remove_at(int) remove(usize) -> <T>
resize(int) -> int resize(usize)
reverse() reverse()
rfind(<T>, int) -> int rfind(<T>, Option<usize>) -> Option<usize>
set(int, <T>) set(usize, <T>)
size() -> int len() -> usize
slice(int, end=int_max) -> Packed<T>Array subarray(usize,usize) as_slice() as_mut_slice()
sort() sort()
to_byte_array() -> PackedByteArray
no godot method to_vec()

Status

This is mostly if not entirely complete. to_byte_array() is probably covered by from/into. duplicate() and clone() might be slightly different. push_back() and append() may be slightly different than push()

Remaining work

validate append and push_back() are the same as push(). Validate duplicate() and clone() are the same. Validate to_byte_array() exists as a result of from/into.

0awful commented 8 months ago

Status

Godot Method Rust Method
angle_to(Quaternion) -> float angle_to(Quaternion) -> f32
dot(Quaternion) -> float dot(Quaternion) -> f32
exp() -> Quaternion to_exp() -> Quaternion
from_euler(Vector3) -> Quaternion from_euler(Vector3) -> Quaternion
get_angle() -> float get_angle() -> f32
get_axis() -> Vector3 get_axis() -> Vector3
get_euler(int=2) -> Vector3 to_euler(EulerOrder) -> Vector3
inverse() -> Quaternion inverse() -> Quaternion
is_equal_approx(Quaternion) -> bool
is_finite() -> bool is_finite() -> bool
is_normalize() -> bool is_normalized() -> f32
length() -> float length() -> f32
length_squared() -> float length_squared() -> f32
log() -> Quaternion log() -> Quaternion
normalized() -> Quaternion normalized() -> Quaternion
slerp(Quaternion, float) -> Quaternion slerp(Quaternion, f32) -> Quaternion
slerpni(Quaternion, float) -> Quaternion slerpni(Quaternion, f32) -> Quaternion
spherical_cubic_interpolate(Quaternion, Quaternion, Quaternion, float) -> Quaternion
spherical_cubic_interpolate_in_time(Quaternion, Quaternion, Quaternion, float, float, float) -> Quaternion

Status

Spherical_cubic functions need to be implemented. is_equal_approx should also be implemented, unless another trait accomplishes this. (PartialEq)

Remaining work

Called out in status

lilizoey commented 8 months ago

... is_equal_approx should also be implemented, unless another trait accomplishes this. (PartialEq)

That would be ApproxEq.

0awful commented 8 months ago

Working on the Quaternions. The code calls out an interest in implementing it in rust. So I dug into the engine code for the implementation. I can re-implement that in rust, but it doesn't look like the best algorithm. (I might be wrong about that, but ogre has a simpler implementation) If I opted for a different algorithm we wouldn't necessarily have parity (though we should). Is the preference here identical outcomes or best performance?

It also relies on these hand rolled cubic interpolation functions. Is there an appetite for using a crate to supply cubic interpolation functions, or is the preference maintaining minimal dependencies?

Bromeon commented 8 months ago

In the past, we may have been a bit overzealous at porting Godot functionality to Rust, in an attempt to improve performance. There was also a (only partially correct) understanding that FFI calls would be inherently slow and to be avoided at all costs. However, RIIR (rewrite it in Rust) doesn't come for free: effort for testing and maintenance, risk of discrepancies in behavior, added compile-time overhead. If Godot fixes something, we are either left with different behavior or need to follow suit.

Since these re-implementations were mostly done without measuring, it remains to be seen whether they're actually worth it. For simpler things like vector operations that are delegated to glam, using Rust is more justified, also because this domain is so central to any gamedev code that it's hard to deny savings. But for more involved operations, we would need more evidence, ideally backed by a real application.

Is the preference here identical outcomes or best performance?

Ideally identical, or at least very similar outcomes. That may already not be the case for some of the geometric types we built -- which can lead to subtle bugs in larger programs 🤔 but it's hard to say in general. If an algorithm is 10x faster than another while retaining 99.7% accuracy, then that's likely the better choice.

It also relies on these hand rolled cubic interpolation functions. Is there an appetite for using a crate to supply cubic interpolation functions, or is the preference maintaining minimal dependencies?

I'm generally quite hesitant to add new dependencies, because they have an impact not only on users, but also our CI workflows, slowing down contributions. For something where the added value is unclear, even more so. And massive crates with type shenanigans (nalgebra & Co.) are an absolute non-starter 😀

Yet, as mentioned above, I also don't want to make people do silly code-porting exercises and significantly increase the maintenance surface of the library, when there's no clear need. Using Godot's impl until it causes actual problems is a reasonable approach in my view.

TLDR: I think the first step would be to prove that RIIR is bringing real-world benefits, or else focus on other tasks. We already have a massive library at hand that has parity with Godot and needs zero maintenance and zero compile time on our side, it's called Godot 😉

0awful commented 8 months ago

Using Godot's impl until it causes actual problems is a reasonable approach in my view.

Sounds good. Given that it probably makes most sense to get 100% builtin parity before considering optimizations. Per Quaternions I'll see if I can use the godot builtins. The file is 'special' as it appears to be mostly RIIR. Where other builtins are heavily reliant on godot engine code. That may prove a problem for avoiding RIIR. But I'll give it a go and report back.

Bromeon commented 8 months ago

The file is 'special' as it appears to be mostly RIIR. Where other builtins are heavily reliant on godot engine code.

As mentioned, the RIIR may also have been a mistake in some cases. If we run into problems due to it, or are faced with increased maintenance burden, I'm happy to yeet that code and revert back to Godot's implementation.

0awful commented 8 months ago

570 and #565 complete signals and quaternions respectively. #561 handles array builtin completeness.

570 does expose a few places we need doc aliases and depending on if from_opaque is a reasonable substitute for from_signal may need a from_signal

Each can be updated to green

0awful commented 7 months ago

Vector3 should be green because #316 implemented slerp

Bromeon commented 7 months ago

Thanks, updated according to your last two comments.

There are a few entries that mention documentation. In cases where the Rust API is the same as Godot's, we can reuse the provided docs from Godot. I created https://github.com/godot-rust/gdext/issues/584 for it.

vortexofdoom commented 7 months ago

Is there an argument for implementing std::ops::Index<usize> (and IndexMut) on PackedArray types? get semantics usually return an Option in rust, and don't panic, and Index would keep syntax and expectations wrt out of bounds consistent to both rust and godot.

I guess there's the difference between returning T vs &T but with the smart pointer types those semantics are blurred, and the Output type can reflect that in any case.

Bromeon commented 7 months ago

Is there an argument for implementing std::ops::Index<usize> (and IndexMut) on PackedArray types?

We'd need to double-check that the current implementation of as_slice and as_mut_slice is truly sound -- packed arrays have CoW semantics but we need to ensure that there's no scenario where aliasing can happen.

If that is the case, Index[Mut] traits should also be possible.