dart-lang / native

Dart packages related to FFI and native assets bundling.
BSD 3-Clause "New" or "Revised" License
93 stars 33 forks source link

[DartError] RangeError: Invalid value: Only valid value is 0: 1; #0 Array._checkIndex (dart:ffi-patch/ffi_patch.dart:349:7)#1 main (package:swiftsoft/main.dart)<asynchronous suspension> #1200

Open kirill-21 opened 2 weeks ago

kirill-21 commented 2 weeks ago

After updating flutter i started having errors everywhere where i use FFI Array in Structs when i try to access index greater then 0, why?

/// The TOKEN_PRIVILEGES structure contains information about a set
/// of privileges for an access token.
/// The LUID_AND_ATTRIBUTES structure represents a locally unique identifier
/// (LUID) and its attributes.
sealed class LuidAndAttributes extends Struct {
  external LUID luid;
  @Uint32()
  external int attributes;
}

base class LUID extends Struct {
  @Uint32()
  external int LowPart;

  @Int32()
  external int HighPart;
}

sealed class TokenPriviliges extends Struct {
  @Uint32()
  external int privilegeCount;
  @Array(1)
  external Array<LuidAndAttributes> privileges;
}

  final Pointer<TokenPriviliges> token = malloc<TokenPriviliges>();
  token.ref.privileges[0].luid.HighPart = 1;
  token.ref.privileges[0].luid.LowPart = 2;

  token.ref.privileges[1].luid.HighPart = 1;
  token.ref.privileges[1].luid.LowPart = 2;

  print([token.ref.privileges[0].luid.HighPart, token.ref.privileges[1].luid.HighPart]);
[DartError] RangeError: Invalid value: Only valid value is 0: 1; #0      Array._checkIndex (dart:ffi-patch/ffi_patch.dart:349:7)#1      main (package:swiftsoft/main.dart)<asynchronous suspension>
kirill-21 commented 2 weeks ago

But if you specify

@Array(3)

, then you'll be able to do that. But what if i need to create Any size array like in C++? There you have to mention 1 to do that. I do not see anything related to this on Breaking changes page

image

dcharkes commented 2 weeks ago

Any size array like in C++

Variable length arrays are not supported yet:

The issue with variable size length arrays at the end of structs is that sizeOf<TokenPriviliges>() cannot report a length anymore. And as such malloc<TokenPriviliges>() would return the wrong size.

I did fix some missing offsets recently (https://dart-review.googlesource.com/c/sdk/+/345060, https://dart-review.googlesource.com/c/sdk/+/345061).

Without those checks you might have used sizeOf or malloc and get wrong behavior.

Let me think of a workaround. Do you have a Pointer<TokenPriviliges>? Or a TokenPriviliges.

If you have Pointer<TokenPriviliges>, you could try:

Pointer<TokenPriviliges>.fromAddress(pointer.address + sizeOf<Uint32>())

The offset jumps over the first field. A Pointer denotes a variable length array, an Array a fixed size array. If we addressed https://github.com/dart-lang/sdk/issues/41237, you could have used that instead of the manual pointer address calculation. If we addressed https://github.com/dart-lang/sdk/issues/54526, you could make it an array again based on privilegeCount.

(If you have TokenPriviliges, I don't really know a workaround, because such object could either live in Dart or in C memory. You can't return a TokenPriviliges by value, precisely because the compiler doesn't know how many bytes it is, because it's variable length.)

kirill-21 commented 2 weeks ago

The problem is that it's a regression, it was working fine on previous stable release.

No, it's not a pointer :c

kirill-21 commented 2 weeks ago

Example i provided is only an example, there are a lot of places like process threads handling where there can be hundreds of threads inside an array and you have no idea how many there would be on compile time to specify appropriate number

typedef struct _SYSTEM_PROCESS     // common members
{
    ULONG          NextThreadOffset;
    ULONG          TotalThreadCount;
    LARGE_INTEGER  WorkingSetPrivateSize;
    ULONG          HardFaultCount;
    ULONG          NumberOfThreadsHighWatermark;
    ULONGLONG      CycleTime;
    LARGE_INTEGER  CreateTime;
    LARGE_INTEGER  UserTime;
    LARGE_INTEGER  KernelTime;
    UNICODE_STRING ImageName;
    KPRIORITY      BasePriority;
    HANDLE         UniqueProcessId;
    HANDLE         InheritedFromUniqueProcessId;
    ULONG          HandleCount;
    ULONG          SessionId;
    ULONG_PTR      UniqueProcessKey;
    ULONG_PTR      PeakVirtualSize;
    ULONG_PTR      VirtualSize;
    ULONG          PageFaultCount;
    ULONG_PTR      PeakWorkingSetSize;
    ULONG_PTR      WorkingSetSize;
    ULONG_PTR      QuotaPeakPagedPoolUsage;
    ULONG_PTR      QuotaPagedPoolUsage;
    ULONG_PTR      QuotaPeakNonPagedPoolUsage;
    ULONG_PTR      QuotaNonPagedPoolUsage;
    ULONG_PTR      PagefileUsage;
    ULONG_PTR      PeakPagefileUsage;
    ULONG_PTR      PrivatePageCount;
    LARGE_INTEGER  ReadOperationCount;
    LARGE_INTEGER  WriteOperationCount;
    LARGE_INTEGER  OtherOperationCount;
    LARGE_INTEGER  ReadTransferCount;
    LARGE_INTEGER  WriteTransferCount;
    LARGE_INTEGER  OtherTransferCount;
    SYSTEM_THREAD  aThreads [1];
} SYSTEM_PROCESS_INFORMATION, *PSYSTEM_PROCESS_INFORMATION;
#define SYSTEM_PROCESS_ sizeof (SYSTEM_PROCESS_INFORMATION)
HosseinYousefi commented 2 weeks ago

Is there an option to generate a pointer instead of the array?

sealed class TokenPriviliges extends Struct {
  @Uint32()
  external int privilegeCount;

  external Pointer<LuidAndAttributes> privileges;
}

Maybe we can automatically turn any array[1] to a pointer in ffigen.

dcharkes commented 2 weeks ago

Do you have the code somewhere available for me to look at?

Returning a struct with a variable length last member by value in a C function is undefined behavior. It is not specified in the ABI (application binary interface). So I'm thinking your could returns a pointer to such struct from a dart:ffi call, which is then followed by a .ref.

dcharkes commented 2 weeks ago

Is there an option to generate a pointer instead of the array?

sealed class TokenPriviliges extends Struct {
  @Uint32()
  external int privilegeCount;

  external Pointer<LuidAndAttributes> privileges;
}

Maybe we can automatically turn any array[1] to a pointer in ffigen.

That doesn't work. Structs can be returned by value. Then they are copied into the Dart heap. And then the GC can move them. Array is special in that it can be backed either by Dart memory or by native memory.

kirill-21 commented 2 weeks ago

Here is real usage example with such Array. Upon launch it prints you all system processes and then loops through their threads to check the state. This code was working fine on previous Flutter version and works bad on new one

dependencies:
  flutter:
    sdk: flutter
  ffi: ^2.1.2
  win32: ^5.5.1
import 'dart:io';

import 'package:ffi/ffi.dart';
import 'package:flutter/material.dart';
import 'dart:ffi';

import 'package:win32/win32.dart';

void main() {
  /// Loops through all system processes with [NtQuerySystemInformation].
  Future<bool> loopThroughSystemProcessesWithNtApi(Function callback) async {
    final Pointer<Uint32> dwSize = malloc<Uint32>();
    Pointer<SystemProcessInformation> buffer =
        malloc<SystemProcessInformation>();

    // SystemProcessInformation = 5;
    int info = dNtQuerySystemInformation(
      SystemInformationClass.systemProcessInformation,
      nullptr,
      0,
      dwSize,
    );

    // [STATUS_INFO_LENGTH_MISMATCH] means that we've got the size and now need
    // to call this method once again but with actual pointer to the buffer
    if (!ntSuccess(info) && info == statusInfoLengthMismatch) {
      // Limit attempts to 5
      for (int i = 0; i < 5; i++) {
        malloc.free(buffer);
        buffer = malloc<SystemProcessInformation>(dwSize.value);
        info = dNtQuerySystemInformation(
          SystemInformationClass.systemProcessInformation,
          buffer,
          dwSize.value,
          dwSize,
        );
        if (ntSuccess(info)) break;
      }
    }

    // Failed to retrieve processes list
    if (!ntSuccess(info)) {
      print('Failed to loop trough system processes: $info');
      malloc.free(dwSize);
      malloc.free(buffer);
      return false;
    }

    // Save original buffer address for future no-crash release
    final int realBufferAddress = buffer.address;

    // Loop through all found entities
    while (buffer.ref.nextEntryOffset != 0) {
      try {
        // Stop looping if we found what we've been looking for
        if ((await callback.call(buffer)) == true) break;

        // Stop updating the loop to avoid crashes
        if (buffer.ref.nextEntryOffset == 0) break;

        // Update buffer to the next process
        buffer = (buffer.cast<Uint8>() + buffer.ref.nextEntryOffset)
            .cast<SystemProcessInformation>();
      } catch (_) {
        break;
      }
    }

    // Restore original buffer address to avoid crashes
    buffer = Pointer.fromAddress(realBufferAddress);
    if (realBufferAddress != 0) malloc.free(buffer);

    malloc.free(dwSize);
    return true;
  }

  loopThroughSystemProcessesWithNtApi(
    (
      Pointer<SystemProcessInformation>? image, {
      String? retrievedName,
      int? retrievedPId,
    }) {
      // Skip invalid images
      if (image != null && image.ref.imageName.buffer == nullptr) return false;
      if (image?.ref.sessionId == 0 || image == null) return false;

      String? toDartString = retrievedName ??
          image.ref.imageName.buffer
              .toDartString()
              .split(Platform.pathSeparator)
              .last;

      print(
        'Found process $toDartString, number of threads: ${image.ref.numberOfThreads}',
      );

      try {
        for (int i = 0; i < image.ref.numberOfThreads; i++) {
          print(
            'thread $i: ${image.ref.threads[i].threadState}, ${image.ref.threads[i].waitReason}',
          );
        }
      } catch (_) {
        print(_);
      }
    },
  );

  runApp(const MyApp());
}

/// Macros for NTSTATUS.
const int statusInfoLengthMismatch = -1073741820;

/// Checks if Nt... function has succeded.
bool ntSuccess(int result) => result >= 0;

// Allocalte ntdll API
final Pointer<Utf16> _ntdllTitle = 'ntdll.dll'.toNativeUtf16();
final int ntdllHndle = GetModuleHandle(_ntdllTitle);

/// Dart representation of native [NtQuerySystemInformation] function.
/// ```c
/// [in]            SYSTEM_INFORMATION_CLASS SystemInformationClass,
/// [in, out]       PVOID                    SystemInformation,
/// [in]            ULONG                    SystemInformationLength,
/// [out, optional] PULONG                   ReturnLength
/// ```
int Function(int, Pointer<NativeType>, int, PULONG)?
    _pNtQuerySystemInformationFunc;

/// [NtQuerySystemInformation may be altered or unavailable in future versions
/// of Windows. Applications should use the alternate functions
/// listed in this topic.] Retrieves the specified system information.
int dNtQuerySystemInformation(
  SystemInformationClass dwFlag,
  Pointer<NativeType> buffer,
  int dwSize,
  PULONG out,
) {
  if (_pNtQuerySystemInformationFunc == null) {
    // Find function address with
    final Pointer<Utf8> ansi = 'NtQuerySystemInformation'.toANSI();
    final Pointer pNtQuerySystemInformation = GetProcAddress(ntdllHndle, ansi);
    malloc.free(ansi);

    // Return 0 if function was not found
    if (pNtQuerySystemInformation == nullptr) return 0;

    _pNtQuerySystemInformationFunc = pNtQuerySystemInformation
        .cast<NativeFunction<NTSTATUS Function(ULONG, PVOID, ULONG, PULONG)>>()
        .asFunction<int Function(int, PVOID, int, PULONG)>();
  }

  return _pNtQuerySystemInformationFunc?.call(
        dwFlag.index,
        buffer,
        dwSize,
        out,
      ) ??
      -1;
}

// Shortcut for Pointer<Uint32>
typedef PULONG = Pointer<Uint32>;

/// Basic info about the process
sealed class SystemProcessInformation extends Struct {
  @Uint32()
  external int nextEntryOffset;
  @Uint32()
  external int numberOfThreads;
  external LargeInteger workingSetPrivateSize;
  @Uint32()
  external int hardFaultCount;
  @Uint32()
  external int numberOfThreadsHighWatermark;
  @Uint64()
  external int cycleTime;
  external LargeInteger createTime;
  external LargeInteger userTime;
  external LargeInteger kernelTime;
  external UnicodeString imageName;
  @Int32()
  external int basePriority;
  @IntPtr()
  external int uniqueProcessId;
  @IntPtr()
  external int inheritedFromUniqueProcessId;
  @Uint32()
  external int handleCount;
  @Uint32()
  external int sessionId;
  @IntPtr()
  external int uniqueProcessKey;
  @IntPtr()
  external int peakVirtualSize;
  @IntPtr()
  external int virtualSize;
  @Uint32()
  external int pageFaultCount;
  @IntPtr()
  external int peakWorkingSetSize;
  @IntPtr()
  external int workingSetSize;
  @IntPtr()
  external int quotaPeakPagedPoolUsage;
  @IntPtr()
  external int quotaPagedPoolUsage;
  @IntPtr()
  external int quotaPeakNonPagedPoolUsage;
  @IntPtr()
  external int quotaNonPagedPoolUsage;
  @IntPtr()
  external int pagefileUsage;
  @IntPtr()
  external int peakPagefileUsage;
  @IntPtr()
  external int ppivatePageCount;
  external LargeInteger readOperationCount;
  external LargeInteger writeOperationCount;
  external LargeInteger otherOperationCount;
  external LargeInteger readTransferCount;
  external LargeInteger writeTransferCount;
  external LargeInteger otherTransferCount;
  @Array(1)
  external Array<SystemThreadInformation> threads;
}

/// The SYSTEM_THREAD_INFORMATION structure contains information about
/// a thread running on a system.
sealed class SystemThreadInformation extends Struct {
  external LargeInteger kernelTime;
  external LargeInteger userTime;
  external LargeInteger createTime;
  @Uint32()
  external int waitTime;
  external Pointer<Void> startAddress;
  external ClientID clientId;
  @Int32()
  external int priority;
  @Int32()
  external int basePriority;
  @Uint32()
  external int contextSwitches;
  @Uint32()
  external int threadState;
  @Uint32()
  external int waitReason;
}

/// The UNICODE_STRING structure is used to define Unicode strings.
sealed class ClientID extends Struct {
  @IntPtr()
  external int uniqueProcess;
  @IntPtr()
  external int uniqueThread;
}

/// The UNICODE_STRING structure is used to define Unicode strings.
sealed class UnicodeString extends Struct {
  @Uint16()
  external int length;
  @Uint16()
  external int maximumLength;
  external Pointer<Utf16> buffer;
}

/// Represents a 64-bit signed integer value.
sealed class LargeInteger extends Union {
  external _LargeIntegerFirst parts;
  external _LargeIntegerSecond parts2;
  @Int64()
  external int quadPart;
}

/// Structure for LargeInteger union.
sealed class _LargeIntegerFirst extends Struct {
  @Uint32()
  external int lowPart;
  @Int32()
  external int highPart;
}

/// Structure for LargeInteger union.
sealed class _LargeIntegerSecond extends Struct {
  @Uint32()
  external int lowPart;
  @Int32()
  external int highPart;
}

/// Indicate the kind of system information to be retrieved.
enum SystemInformationClass {
  systemBasicInformation,
  systemProcessorInformation,
  systemPerformanceInformation,
  systemTimeOfDayInformation,
  systemPathInformation,
  systemProcessInformation,
  systemCallCountInformation,
  systemDeviceInformation,
  systemProcessorPerformanceInformation,
  systemFlagsInformation,
  systemCallTimeInformation,
  systemModuleInformation,
  systemLocksInformation,
  systemStackTraceInformation,
  systemPagedPoolInformation,
  systemNonPagedPoolInformation,
  systemHandleInformation,
  systemObjectInformation,
  systemPageFileInformation,
  systemVdmInstemulInformation,
  systemVdmBopInformation,
  systemFileCacheInformation,
  systemPoolTagInformation,
  systemInterruptInformation,
  systemDpcBehaviorInformation,
  systemFullMemoryInformation,
  systemLoadGdiDriverInformation,
  systemUnloadGdiDriverInformation,
  systemTimeAdjustmentInformation,
  systemSummaryMemoryInformation,
  systemMirrorMemoryInformation,
  systemPerformanceTraceInformation,
  systemObsolete0,
  systemExceptionInformation,
  systemCrashDumpStateInformation,
  systemKernelDebuggerInformation,
  systemContextSwitchInformation,
  systemRegistryQuotaInformation,
  systemExtendServiceTableInformation,
  systemPrioritySeperation,
  systemVerifierAddDriverInformation,
  systemVerifierRemoveDriverInformation,
  systemProcessorIdleInformation,
  systemLegacyDriverInformation,
  systemCurrentTimeZoneInformation,
  systemLookasideInformation,
  systemTimeSlipNotification,
  systemSessionCreate,
  systemSessionDetach,
  systemSessionInformation,
  systemRangeStartInformation,
  systemVerifierInformation,
  systemVerifierThunkExtend,
  systemSessionProcessInformation,
  systemLoadGdiDriverInsystemSpace,
  systemNumaProcessorMap,
  systemPrefetcherInformation,
  systemExtendedProcessInformation,
  systemRecommendedSharedDataAlignment,
  systemComPlusPackage,
  systemNumaAvailableMemory,
  systemProcessorPowerInformation,
  systemEmulationBasicInformation,
  systemEmulationProcessorInformation,
  systemExtendedHandleInformation,
  systemLostDelayedWriteInformation,
  systemBigPoolInformation,
  systemSessionPoolTagInformation,
  systemSessionMappedViewInformation,
  systemHotpatchInformation,
  systemObjectSecurityMode,
  systemWatchdogTimerHandler,
  systemWatchdogTimerInformation,
  systemLogicalProcessorInformation,
  systemWow64SharedInformationObsolete,
  systemRegisterFirmwareTableInformationHandler,
  systemFirmwareTableInformation,
  systemModuleInformationEx,
  systemVerifierTriageInformation,
  systemSuperfetchInformation,
  systemMemoryListInformation,
  systemFileCacheInformationEx,
  systemThreadPriorityClientIdInformation,
  systemProcessorIdleCycleTimeInformation,
  systemVerifierCancellationInformation,
  systemProcessorPowerInformationEx,
  systemRefTraceInformation,
  systemSpecialPoolInformation,
  systemProcessIdInformation,
  systemErrorPortInformation,
  systemBootEnvironmentInformation,
  systemHypervisorInformation,
  systemVerifierInformationEx,
  systemTimeZoneInformation,
  systemImageFileExecutionOptionsInformation,
  systemCoverageInformation,
  systemPrefetchPatchInformation,
  systemVerifierFaultsInformation,
  systemsystemPartitionInformation,
  systemsystemDiskInformation,
  systemProcessorPerformanceDistribution,
  systemNumaProximityNodeInformation,
  systemDynamicTimeZoneInformation,
  systemCodeIntegrityInformation,
  systemProcessorMicrocodeUpdateInformation,
  systemProcessorBrandString,
  systemVirtualAddressInformation,
  systemLogicalProcessorAndGroupInformation,
  systemProcessorCycleTimeInformation,
  systemStoreInformation,
  systemRegistryAppendString,
  systemAitSamplingValue,
  systemVhdBootInformation,
  systemCpuQuotaInformation,
  systemNativeBasicInformation,
  systemErrorPortTimeouts,
  systemLowPriorityIoInformation,
  systemTpmBootEntropyInformation,
  systemVerifierCountersInformation,
  systemPagedPoolInformationEx,
  systemsystemPtesInformationEx,
  systemNodeDistanceInformation,
  systemAcpiAuditInformation,
  systemBasicPerformanceInformation,
  systemQueryPerformanceCounterInformation,
  systemSessionBigPoolInformation,
  systemBootGraphicsInformation,
  systemScrubPhysicalMemoryInformation,
  systemBadPageInformation,
  systemProcessorProfileControlArea,
  systemCombinePhysicalMemoryInformation,
  systemEntropyInterruptTimingInformation,
  systemConsoleInformation,
  systemPlatformBinaryInformation,
  systemPolicyInformation,
  systemHypervisorProcessorCountInformation,
  systemDeviceDataInformation,
  systemDeviceDataEnumerationInformation,
  systemMemoryTopologyInformation,
  systemMemoryChannelInformation,
  systemBootLogoInformation,
  systemProcessorPerformanceInformationEx,
  systemCriticalProcessErrorLogInformation,
  systemSecureBootPolicyInformation,
  systemPageFileInformationEx,
  systemSecureBootInformation,
  systemEntropyInterruptTimingRawInformation,
  systemPortableWorkspaceEfiLauncherInformation,
  systemFullProcessInformation,
  systemKernelDebuggerInformationEx,
  systemBootMetadataInformation,
  systemSoftRebootInformation,
  systemElamCertificateInformation,
  systemOfflineDumpConfigInformation,
  systemProcessorFeaturesInformation,
  systemRegistryReconciliationInformation,
  systemEdidInformation,
  systemManufacturingInformation,
  systemEnergyEstimationConfigInformation,
  systemHypervisorDetailInformation,
  systemProcessorCycleStatsInformation,
  systemVmGenerationCountInformation,
  systemTrustedPlatformModuleInformation,
  systemKernelDebuggerFlags,
  systemCodeIntegrityPolicyInformation,
  systemIsolatedUserModeInformation,
  systemHardwareSecurityTestInterfaceResultsInformation,
  systemSingleModuleInformation,
  systemAllowedCpuSetsInformation,
  systemVsmProtectionInformation,
  systemInterruptCpuSetsInformation,
  systemSecureBootPolicyFullInformation,
  systemCodeIntegrityPolicyFullInformation,
  systemAffinitizedInterruptProcessorInformation,
  systemRootSiloInformation,
  systemCpuSetInformation,
  systemCpuSetTagInformation,
  systemWin32WerStartCallout,
  systemSecureKernelProfileInformation,
  systemCodeIntegrityPlatformManifestInformation,
  systemInterruptSteeringInformation,
  systemSupportedProcessorArchitectures,
  systemMemoryUsageInformation,
  systemCodeIntegrityCertificateInformation,
  systemPhysicalMemoryInformation,
  systemControlFlowTransition,
  systemKernelDebuggingAllowed,
  systemActivityModerationExeState,
  systemActivityModerationUserSettings,
  systemCodeIntegrityPoliciesFullInformation,
  systemCodeIntegrityUnlockInformation,
  systemIntegrityQuotaInformation,
  systemFlushInformation,
  systemProcessorIdleMaskInformation,
  systemSecureDumpEncryptionInformation,
  systemWriteConstraintInformation,
  systemKernelVaShadowInformation,
  systemHypervisorSharedPageInformation,
  systemFirmwareBootPerformanceInformation,
  systemCodeIntegrityVerificationInformation,
  systemFirmwarePartitionInformation,
  systemSpeculationControlInformation,
  systemDmaGuardPolicyInformation,
  systemEnclaveLaunchControlInformation,
  systemWorkloadAllowedCpuSetsInformation,
  systemCodeIntegrityUnlockModeInformation,
  systemLeapSecondInformation,
  systemFlags2Information,
  systemSecurityModelInformation,
  systemCodeIntegritySyntheticCacheInformation,
  systemFeatureConfigurationInformation,
  systemFeatureConfigurationSectionInformation,
  systemFeatureUsageSubscriptionInformation,
  systemSecureSpeculationControlInformation,
  systemSpacesBootInformation,
  systemFwRamdiskInformation,
  systemWheaIpmiHardwareInformation,
  systemDifSetRuleClassInformation,
  systemDifClearRuleClassInformation,
  systemDifApplyPluginVerificationOnDriver,
  systemDifRemovePluginVerificationOnDriver,
  systemShadowStackInformation,
  systemBuildVersionInformation,
  systemPoolLimitInformation,
  systemCodeIntegrityAddDynamicStore,
  systemCodeIntegrityClearDynamicStores,
  systemDifPoolTrackingInformation,
  systemPoolZeroingInformation,
  systemDpcWatchdogInformation,
  systemDpcWatchdogInformation2,
  systemSupportedProcessorArchitectures2,
  systemSingleProcessorRelationshipInformation,
  systemXfgCheckFailureInformation,
  systemIommuStateInformation,
  systemHypervisorMinrootInformation,
  systemHypervisorBootPagesInformation,
  systemPointerAuthInformation,
  systemSecureKernelDebuggerInformation,
  systemOriginalImageFeatureInformation,
  maxsystemInfoClass,
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // TRY THIS: Try running your application with "flutter run". You'll see
        // the application has a purple toolbar. Then, without quitting the app,
        // try changing the seedColor in the colorScheme below to Colors.green
        // and then invoke "hot reload" (save your changes or press the "hot
        // reload" button in a Flutter-supported IDE, or press "r" if you used
        // the command line to start the app).
        //
        // Notice that the counter didn't reset back to zero; the application
        // state is not lost during the reload. To reset the state, use hot
        // restart instead.
        //
        // This works for code too, not just values: Most code changes can be
        // tested with just a hot reload.
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // TRY THIS: Try changing the color here to a specific color (to
        // Colors.amber, perhaps?) and trigger a hot reload to see the AppBar
        // change color while the other colors stay the same.
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          //
          // TRY THIS: Invoke "debug painting" (choose the "Toggle Debug Paint"
          // action in the IDE, or press "p" in the console), to see the
          // wireframe for each widget.
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            const Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headlineMedium,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}
dcharkes commented 2 weeks ago

Reading a bit more on https://devblogs.microsoft.com/oldnewthing/20040826-00/?p=38043, interpreting an array of length exactly size one as something else is what you do in C/C++ manually by using some macros to figure out how many bytes to malloc.

And then in C you rely on having no bounds checks on array access when accessing elements.

(Offtopic: As a high level comment, it's kind of weird that Windows uses length 1 arrays, but the GNU standard is 0 length arrays for variable length inline arrays at the end of structs.)

When using dart:ffi your structs can be either in the Dart heap or on the native memory. Once you have a struct class object it's invisible if it's backed by the Dart heap or native memory. Having a pointer to a struct shows you that it's backed by native memory.

In your code, you know that it's backed by native memory:

Pointer<SystemProcessInformation>? image,

But once one does .ref that information is lost.

We might want to allow unbounded array reads for native memory, but not for the Dart heap. Since Array can be backed by both, I'm not yet entirely sure how we can allow for C behavior if it's C backed memory.

To use the fact that it's C backed memory you could do the loop as follows:

      try {
        for (int i = 0; i < image.ref.numberOfThreads; i++) {
          final threads = Pointer<SystemThreadInformation>.fromAddress(
              image.address +
                  sizeOf<SystemProcessInformation>() -
                  sizeOf<SystemThreadInformation>());
          print(
            'thread $i: ${threads[i].threadState}, ${threads[i].waitReason}',
          );
        }
      } catch (_) {
        print(_);
      }

We could simplify the calculation of threads if we implemented an offsetOfField for struct fields (https://github.com/dart-lang/sdk/issues/41237).

kirill-21 commented 2 weeks ago

Yeah, this workaround seems to be working, but re-writing all the places where it's used seems to be hard, couldn't this be done on Dart side automatically when you try accessing the Array?

dcharkes commented 2 weeks ago

couldn't this be done on Dart side automatically when you try accessing the Array?

I've penned down some ideas at https://github.com/dart-lang/sdk/issues/55964.

Yeah, this workaround seems to be working, but re-writing all the places where it's used seems to be hard

If you're targetting Flutter stable you might have to, because any fix we'll come up with will take a while to be in the stable release. I'm sorry for the churn!

Thanks for the report!