objectbox / objectbox-dart

Flutter database for super-fast Dart object persistence
https://docs.objectbox.io/getting-started
Apache License 2.0
927 stars 115 forks source link

Treat int as 8 bytes #529

Closed semin-park closed 11 months ago

semin-park commented 1 year ago

Describe the solution you'd like

Current implementation of PropertyType.int seems to be 32 bit integer. However, dart integers are usually 64 bit integers although it depends on the platform. Thus, it can happen that objectbox truncates the integer to 32 bits only and lose some information.

Describe alternatives you've considered

I considered using byteVector which I believe would work, but the data that I'm dealing with is 64 bit integers, so I would much prefer if objectbox can handle 64 bit integers in the first place.

Thanks!

greenrobot-team commented 1 year ago

Just use the Dart type int without any annotation, e.g. int number;.

Use the @Property annotation only to use a smaller integer, see https://docs.objectbox.io/advanced/custom-types for details.

semin-park commented 1 year ago

Since Dart has varying data type sizes depending on the platform, it would be much more preferable if I could explicitly specify the data type (u64) instead of having to assume that it's probably stored as 64 bits.

Would it make sense to add support for u64 types explicitly?

greenrobot commented 1 year ago

Dart's int precision is not equivalent to a true 64 integer type (although it's much wider than 32 bit ints; it seems to use 64 bits floats for web platform). That is your concern right? In that case we should think about BigInt support, I guess?

semin-park commented 1 year ago

I should have also mentioned that I'm also using the fixnum package's Int64 to store the integers.

Maybe it might be easier if I explain where I'm coming from. The underlying package that produces these 64 bit data type is written in Rust, and I've created a wrapper around it with flutter_rust_bridge, exposing the data type as Int64. Now I'd like to save this as 64 bit data type in objectbox and noticed that objectbox doesn't let me explicitly state this (even though from #comment2, this seems to be already the default behavior).

Maybe I'm doing it the unnatural way, since dart itself doesn't support specifying the integer width, and it's me who's trying to workaround this issue by using Int64.

But I'm still wondering if supporting int as 8 byte integer is still worth it, for the following reasons:

  1. From the second comment, it looks like integers are already saved as 8 bytes by default.
  2. Objectbox lets users specify PropertyType.int, which will then be stored as exactly 4 bytes. This is a deviation from dart int already, so supporting int as 8 bytes is still consistent with the current PropertyType.int support.
  3. 64 bit integer being truncated to 53 bits in some platforms is a platform limitation, rather than objectbox's limitation, and I hope that objectbox could still support explicitly saving as 64 bit integers for the common case where integers are 64 bits.

Thanks!

greenrobot commented 1 year ago

Just to clarify: ObjectBox has true 64 bit ints (aka PropertyType.long). The question is what Dart offers. Int64 seems worthy of supporting it directly; seems favorable over BigInt too...

semin-park commented 1 year ago

I was editing my previous comment, but wow you're fast :) Thank you for the very quick reply!

Just to clarify: ObjectBox has true 64 bit ints (aka PropertyType.long). The question is what Dart offers. Int64 seems worthy of supporting it directly; seems favorable over BigInt too...

I may be mistaken, but I thought that although ObjectBox has OBXPropertyType.Long, it doesn't expose it in PropertyType. The intention of #530 was actually to add this exposure to PropertyType. Please let me know if I'm missing something!

Regarding Int64 support, I would indeed be very interested in seeing this support coming through 😄

greenrobot commented 1 year ago

I hope that objectbox could still support explicitly saving as 64 bit integers for the common case where integers are 64 bits.

Yes, this is the case (always has been); maybe we should be more explicit in the docs? @greenrobot-team

semin-park commented 1 year ago

Just to make sure that we're in sync, I am referring to this when I say that I think objectbox currently doesn't support explicit saving integers as 64 bits.

I do see that OBXPropertyType has Long, but from the user's POV, I don't think we can annotate it with PropertyType yet.

greenrobot commented 1 year ago

I do see that OBXPropertyType has Long, but from the user's POV, I don't think we can annotate it with PropertyType yet.

I guess it's because it's the default; but I see where the confusion comes from... :+1:

greenrobot-team commented 1 year ago

I guess it's because it's the default

Maybe to clear up some misconceptions about Dart: Dart Native only supports 64-bit integers, nothing else (AFAIK). Only Dart Web uses floating point to represent numbers. However, ObjectBox for Dart does not support web (at least right now).

Hence it was decided to not require any annotation when just using the Dart type int and assume users understand it is 64-bit wide (as documented in the Dart int docs).

See also https://dart.dev/guides/language/numbers#dart-number-representation.

As said, using e.g. @Property(type: PropertyType.int) is only there to tell ObjectBox to use a smaller integer on the database level.

But I agree with many people coming from other languages it might make sense to more clearly point this out in the docs.

Edit: updated the supported types page with some notes about this.

github-actions[bot] commented 11 months ago

Without additional information, we are unfortunately not sure how to resolve this issue. Therefore this issue has been automatically closed. Feel free to comment with additional details and we can re-open this issue.