Question: issues with returning C-string

[avetiso]

Hi everyone,

I'm sure I'm making an error in understanding somewhere, so I would appreciate if someone could show me what it is.

I'm trying to use from C functions from our C++ banking library to mask some strings when the user is typing sensitive information.

Here is the signature of the function I am calling:

DART_EXPORT
const char* mask_format(void* formatter, const char* input);

(formatter is a pointer to my string formatting class, it seems to construct just fine)

ffigen runs on the whole header (there are 4 functions) with no errors and generates a bunch of signatures.

1. Most examples I see tell you that ffigen will convert C char to Int8, and yet my run on ffigen this is what I get:

  ffi.Pointer<ffi.Char> mask_format(
    ffi.Pointer<ffi.Void> formatter,
    ffi.Pointer<ffi.Char> input,
  ) {
    return _mask_format(
      formatter,
      input,
    );
  }

Why do I get ffi.Char when almost all examples tell me that dart:ffi will give you Int8, and that pkg_ffi gives you Utf8?

Assuming that generation is correct, I can't call the function (I know the toString() should actually be toDartString() from Utf8, I was just trying things to try and understand what I'm doing wrong)

lib/main.dart:43:53: Error: The argument type 'Pointer<Utf8>' can't be assigned to the parameter type 'Pointer<Char>'.
 - 'Pointer' is from 'dart:ffi'.
 - 'Utf8' is from 'package:ffi/src/utf8.dart'
 ('../../../../../frameworks/flutter/.pub-cache/hosted/pub.dartlang.org/ffi-2.0.1/lib/src/utf8.dart').
 - 'Char' is from 'dart:ffi'.
    _masked_str = mxffi.mask_format(formatter, _str.toNativeUtf8()).toString();

But everywhere in the examples that I see it tells me to call _str.toNativeUtf8(), seemingly with the implication that it'll map to both Utf8 or Int8 fine, but it doesn't seem to work with Char?

I've tried type-mapping using pkg_ffi so that Char maps to Utf8, and I can generate that just fine, too.

However, then when I try to call this line:

_masked_str = mxffi.mask_format(formatter, _str.toNativeUtf8()).toDartString();

I get an error of FormatException: unexpected extension byte (the offset changes)

What am I doing incorrectly? I'm trying to make this as out of the box as possible, so I'd prefer to just use what dart:ffi/ffigen generates without having to specify my own type mapping with pkg_ffi, but there seems to be type mismatches between what ffigen creates and all the examples of passing Dart strings to C and back.

How do I convert my Dart string into Pointer, and how do I convert the returning Pointer back into a Dart String? And if that's not possible because I have to use toNativeUtf8, why does ffigen generate it that way in the first place?

Full C header:

#pragma once

#define DART_EXPORT __attribute__((visibility("default"))) __attribute__((used))

#ifdef __cplusplus
#define DART_EXPORT extern "C" __attribute__((visibility("default"))) __attribute__((used))
#endif

DART_EXPORT 
void* mask_create_formatter(int unmasked_count);

DART_EXPORT
int mask_delete_formatter(void* formatter);

DART_EXPORT
const char* mask_format(void* formatter, const char* input);

DART_EXPORT
const char* mask_clear_formatting(void* formatter, const char* input);

Default generated bindings from ffigen with no type mapping:

// AUTO GENERATED FILE, DO NOT EDIT.
//
// Generated by `package:ffigen`.
import 'dart:ffi' as ffi;

class NativeLibrary {
  /// Holds the symbol lookup function.
  final ffi.Pointer<T> Function<T extends ffi.NativeType>(String symbolName)
      _lookup;

  /// The symbols are looked up in [dynamicLibrary].
  NativeLibrary(ffi.DynamicLibrary dynamicLibrary)
      : _lookup = dynamicLibrary.lookup;

  /// The symbols are looked up with [lookup].
  NativeLibrary.fromLookup(
      ffi.Pointer<T> Function<T extends ffi.NativeType>(String symbolName)
          lookup)
      : _lookup = lookup;

  ffi.Pointer<ffi.Void> mask_create_formatter(
    int unmasked_count,
  ) {
    return _mask_create_formatter(
      unmasked_count,
    );
  }

  late final _mask_create_formatterPtr =
      _lookup<ffi.NativeFunction<ffi.Pointer<ffi.Void> Function(ffi.Int)>>(
          'mask_create_formatter');
  late final _mask_create_formatter = _mask_create_formatterPtr
      .asFunction<ffi.Pointer<ffi.Void> Function(int)>();

  int mask_delete_formatter(
    ffi.Pointer<ffi.Void> formatter,
  ) {
    return _mask_delete_formatter(
      formatter,
    );
  }

  late final _mask_delete_formatterPtr =
      _lookup<ffi.NativeFunction<ffi.Int Function(ffi.Pointer<ffi.Void>)>>(
          'mask_delete_formatter');
  late final _mask_delete_formatter = _mask_delete_formatterPtr
      .asFunction<int Function(ffi.Pointer<ffi.Void>)>();

  ffi.Pointer<ffi.Char> mask_format(
    ffi.Pointer<ffi.Void> formatter,
    ffi.Pointer<ffi.Char> input,
  ) {
    return _mask_format(
      formatter,
      input,
    );
  }

  late final _mask_formatPtr = _lookup<
      ffi.NativeFunction<
          ffi.Pointer<ffi.Char> Function(
              ffi.Pointer<ffi.Void>, ffi.Pointer<ffi.Char>)>>('mask_format');
  late final _mask_format = _mask_formatPtr.asFunction<
      ffi.Pointer<ffi.Char> Function(
          ffi.Pointer<ffi.Void>, ffi.Pointer<ffi.Char>)>();

  ffi.Pointer<ffi.Char> mask_clear_formatting(
    ffi.Pointer<ffi.Void> formatter,
    ffi.Pointer<ffi.Char> input,
  ) {
    return _mask_clear_formatting(
      formatter,
      input,
    );
  }

  late final _mask_clear_formattingPtr = _lookup<
      ffi.NativeFunction<
          ffi.Pointer<ffi.Char> Function(ffi.Pointer<ffi.Void>,
              ffi.Pointer<ffi.Char>)>>('mask_clear_formatting');
  late final _mask_clear_formatting = _mask_clear_formattingPtr.asFunction<
      ffi.Pointer<ffi.Char> Function(
          ffi.Pointer<ffi.Void>, ffi.Pointer<ffi.Char>)>();
}

[avetiso]

If I use a type mapping like this:

  type-map:
    'native-types': # Targets native types.
      'char':
        'lib': 'pkg_ffi' # predefined import.
        'c-type': 'Utf8'
        'dart-type': 'Utf8'

Then I am able to call _masked_str = mxffi.mask_format(formatter, _str.toNativeUtf8()).toDartString(); just fine, but I still run into FormatException: Missing extension byte (the offset varies here, as well).

[avetiso]

This is how I am getting the library:

    var mxffi = mx.NativeLibrary(ffi.DynamicLibrary.executable());
    var formatter = mxffi.mask_create_formatter(2);
     _masked_str = mxffi.mask_format(formatter, _str.toNativeUtf8()).toDartString();

[avetiso]

In this function in my C++ implementation, returning a string literal works as expected.

When I print the string before returning, it is also masked as expected.

const char* mask_format(void* formatter, const char* input) {
    std::string str = input;
    std::string masked = reinterpret_cast<MX::Utils::String::MaskFormatter*>(formatter)->format(str);
    std::cout << masked << "              <-here";
    std::cout << masked.c_str() << "      cstr";
    //return masked.c_str();
    //return reinterpret_cast<MX::Utils::String::MaskFormatter*>(formatter)->format(input).c_str();
    return "string literal that always works";
}

Trying to return the string with .c_str() or copying it and then returning with .c_str() seems to cause issues/malformation.

[avetiso]

Will open another issue, I think I've figured this one out.