OSX arm returns "" from ffi_str(entry.d_name)

[stuta]

I tried this with a new MacBook Pro.

Otherwise, it seems to work, but after lib.readdir(dir._dentry) -call entry.d_name[0] and other bytes are 0. Any idea how to solve this?

[sonoro1234]

What happens with standard lfs?

[stuta]

I'm using https://github.com/keplerproject/luafilesystem as a backup now - it works well. This is the only ffi code this far that does not work in arm.

[sonoro1234]

Be sure to use last LuaJIT as there was one arm issue recently fixed

[stuta]

I did build it yesterday, it's the latest.

[sonoro1234]

@spacewander any thoughts?

[sonoro1234]

@stuta May be struct dirent has a different definition on your system. Could you provide struct dirent definition for your system (searching included files in standard lfs compilation)?

[xixihahahei]

$ man 2 stat

 struct stat { /* when _DARWIN_FEATURE_64_BIT_INODE is NOT defined */
     dev_t    st_dev;    /* device inode resides on */
     ino_t    st_ino;    /* inode's number */
     mode_t   st_mode;   /* inode protection mode */
     nlink_t  st_nlink;  /* number of hard links to the file */
     uid_t    st_uid;    /* user-id of owner */
     gid_t    st_gid;    /* group-id of owner */
     dev_t    st_rdev;   /* device type, for special file inode */
     struct timespec st_atimespec;  /* time of last access */
     struct timespec st_mtimespec;  /* time of last data modification */
     struct timespec st_ctimespec;  /* time of last file status change */
     off_t    st_size;   /* file size, in bytes */
     quad_t   st_blocks; /* blocks allocated for file */
     u_long   st_blksize;/* optimal file sys I/O ops blocksize */
     u_long   st_flags;  /* user defined flags for file */
     u_long   st_gen;    /* file generation number */
 };

 However, when the macro _DARWIN_FEATURE_64_BIT_INODE is defined, the stat structure will
 now be defined as:

 struct stat { /* when _DARWIN_FEATURE_64_BIT_INODE is defined */
     dev_t           st_dev;           /* ID of device containing file */
     mode_t          st_mode;          /* Mode of file (see below) */
     nlink_t         st_nlink;         /* Number of hard links */
     ino_t           st_ino;           /* File serial number */
     uid_t           st_uid;           /* User ID of the file */
     gid_t           st_gid;           /* Group ID of the file */
     dev_t           st_rdev;          /* Device ID */
     struct timespec st_atimespec;     /* time of last access */
     struct timespec st_mtimespec;     /* time of last data modification */
     struct timespec st_ctimespec;     /* time of last status change */
     struct timespec st_birthtimespec; /* time of file creation(birth) */
     off_t           st_size;          /* file size, in bytes */
     blkcnt_t        st_blocks;        /* blocks allocated for file */
     blksize_t       st_blksize;       /* optimal blocksize for I/O */
     uint32_t        st_flags;         /* user defined flags for file */
     uint32_t        st_gen;           /* file generation number */
     int32_t         st_lspare;        /* RESERVED: DO NOT USE! */
     int64_t         st_qspare[2];     /* RESERVED: DO NOT USE! */
 };

[xixihahahei]

                       _DARWIN_FEATURE_ONLY_64_BIT_INODE not defined
                  -------------------------+-------------------------------
                                           |       Deployment Target
                       user defines:       |   < 10.5       10.5    > 10.5
                  -------------------------+-------------------------------
                           (none)          |   32-bit      32-bit   64-bit
                  _DARWIN_NO_64_BIT_INODE  |   32-bit      32-bit   32-bit
                  _DARWIN_USE_64_BIT_INODE |   32-bit      64-bit   64-bit
                  -------------------------+-------------------------------

                         _DARWIN_FEATURE_ONLY_64_BIT_INODE defined
                  -------------------------+-------------------------------
                       user defines:       | Any Deployment Target
                  -------------------------+-------------------------------
                           (none)          | 64-bit-only
                  _DARWIN_NO_64_BIT_INODE  |   (error)
                  _DARWIN_USE_64_BIT_INODE | 64-bit-only
                  -------------------------+-------------------------------

       32-bit       32-bit inode values are enabled, and the legacy structures involving the
                    ino_t type are in use.  The macro _DARWIN_FEATURE_64_BIT_INODE is not
                    defined.

       64-bit       64-bit inode values are enabled, and the expanded structures involving
                    the ino_t type are in use.  The macro _DARWIN_FEATURE_64_BIT_INODE is
                    defined, and loader symbols will contain the $INODE64 suffix.

       64-bit-only  Like 64-bit, except loader symbols do not have the $INODE64 suffix.

       (error)      A compile time error is generated.

[xixihahahei]

@stuta May be struct dirent has a different definition on your system. Could you provide struct dirent definition for your system (searching included files in standard lfs compilation)?

I think it depends on system version

[xixihahahei]

I tried this with a new MacBook Pro.

Otherwise, it seems to work, but after lib.readdir(dir._dentry) -call entry.d_name[0] and other bytes are 0. Any idea how to solve this?

you can use this in line number 675, only for MacOS >= 10.5

            dirent_def = [[
                /* if _DARWIN_FEATURE_64_BIT_INODE was defined */
                struct dirent {
                    uint32_t d_ino;        /* file number of entry */
                    uint64_t d_seekoff;    /* seek offset (optional, used by servers) */
                    uint16_t d_reclen;     /* length of this record */
                    uint16_t d_namlen;     /* length of string in d_name */
                    uint8_t  d_type;       /* file type, see below */
                    char     d_name[1024];  /* name must be no longer than this */
                };
            ]]

[xixihahahei]

I tried this with a new MacBook Pro.

Otherwise, it seems to work, but after lib.readdir(dir._dentry) -call entry.d_name[0] and other bytes are 0. Any idea how to solve this?

try this https://github.com/sonoro1234/luafilesystem/pull/8

[sonoro1234]

@xixihahahei as stat structure for OSX in ffi_lfs equals the second stat structure posted, it seems that it would be failing in OSX but when system version < 10.5?

[xixihahahei]

@xixihahahei as stat structure for OSX in ffi_lfs equals the second stat structure posted, it seems that it would be failing in OSX but when system version < 10.5?

the stats structure in lfs_ffi.lua assumes that _DARWIN_FEATURE_64_BIT_INODE is defined, has some one try it before system version < 10.5?

     The buf argument is a pointer to a stat structure as defined by <sys/stat.h> and into which information is
     placed concerning the file.  When the macro _DARWIN_FEATURE_64_BIT_INODE is not defined (see below for
     more information about this macro), the stat structure is defined as:

     struct stat { /* when _DARWIN_FEATURE_64_BIT_INODE is NOT defined */
         dev_t    st_dev;    /* device inode resides on */
         ino_t    st_ino;    /* inode's number */
         mode_t   st_mode;   /* inode protection mode */
         nlink_t  st_nlink;  /* number of hard links to the file */
         uid_t    st_uid;    /* user-id of owner */
         gid_t    st_gid;    /* group-id of owner */
         dev_t    st_rdev;   /* device type, for special file inode */
         struct timespec st_atimespec;  /* time of last access */
         struct timespec st_mtimespec;  /* time of last data modification */
         struct timespec st_ctimespec;  /* time of last file status change */
         off_t    st_size;   /* file size, in bytes */
         quad_t   st_blocks; /* blocks allocated for file */
         u_long   st_blksize;/* optimal file sys I/O ops blocksize */
         u_long   st_flags;  /* user defined flags for file */
         u_long   st_gen;    /* file generation number */
     };

     However, when the macro _DARWIN_FEATURE_64_BIT_INODE is defined, the stat structure will now be defined
     as:

     struct stat { /* when _DARWIN_FEATURE_64_BIT_INODE is defined */
         dev_t           st_dev;           /* ID of device containing file */
         mode_t          st_mode;          /* Mode of file (see below) */
         nlink_t         st_nlink;         /* Number of hard links */
         ino_t           st_ino;           /* File serial number */
         uid_t           st_uid;           /* User ID of the file */
         gid_t           st_gid;           /* Group ID of the file */
         dev_t           st_rdev;          /* Device ID */
         struct timespec st_atimespec;     /* time of last access */
         struct timespec st_mtimespec;     /* time of last data modification */
         struct timespec st_ctimespec;     /* time of last status change */
         struct timespec st_birthtimespec; /* time of file creation(birth) */
         off_t           st_size;          /* file size, in bytes */
         blkcnt_t        st_blocks;        /* blocks allocated for file */
         blksize_t       st_blksize;       /* optimal blocksize for I/O */
         uint32_t        st_flags;         /* user defined flags for file */
         uint32_t        st_gen;           /* file generation number */
         int32_t         st_lspare;        /* RESERVED: DO NOT USE! */
         int64_t         st_qspare[2];     /* RESERVED: DO NOT USE! */
     };

[sonoro1234]

Just pushed testINO64.lua to ino64 branch Could you please run it in your several osx and post result?

[stuta]

Arm64 M1 works, but rosetta x64 is still missing the first character. Same problem every time.

Wine x64 works. My wine does not support x86, but I can test it with a real machine when others do work.

[sonoro1234]

runing testINO64.lua on a MacBook Pro which works correctly with lfs_ffi.lua posts: OSX isINO64 = true ffi.arch x64 ffi.abi('64bit') true version 10.15.6 readdir_cast dirent32 *

So it seem that here isINO64 does not work but readdir_cast works

[stuta]

I have a very simple solution. Change this in Ino64 -branch and then these systems work.

Tested and works:

• OSX arm64
• OSX x64
• OSX x86
• Windows x64
• Windows x86
• Linux x64

In Linux ARCH == 'arm64' is not in code so it prints: "TODO support other Linux architectures".

Fix:

--detect INO64 on osx
local isINO64
if OS=='OSX' then
    if ffi.arch == "arm64" then
        isINO64 = true
    else
        isINO64 = false
    end
end

[stuta]

Now with Windows 11, we need also Windows arm64 solution in the future.

[stuta]

Code lib.strprint fails in Linux x64, fixed code:

local function errno()
  return tostring(ffi.errno()) -- ffi_str(lib.strprint(ffi.errno()))
end

[stuta]

I need support for Linux arm64 because we want to install to AWS Lambda function using arm processors. I installed AWS Linux using 'docker pull --platform linux/arm64 amazonlinux'.

I tried Linux arm64 by fixing elseif ARCH == 'arm' -> elseif ARCH == 'arm' or ARCH == 'arm64'.

The program starts but I get cannot obtain information from file -error when trying to use lfs.attributes() on directory.

Is there anything I can do to fix this error? Maybe running gcc -E or something like that on some file to get correct headers?

[stuta]

I created a simple C program and created a new cdef with it using GCC -E, but I still got the same errors. Then I found this: https://github.com/torvalds/linux/blob/v4.17/include/uapi/asm-generic/unistd.h#L150. Syscall number stat_syscall_num = 43 and lstat_syscall_num = 44 do work.

There are some errors: mode_to_ftype(mode) returns 'other', in my tests mode is 120699413 and ftype is 45056. When I call _M.dir I get print cannot open .../common/audit.log.json : 20 and then segmentation fault.

I tested both in AWS Linux and in OSX lima (https://github.com/lima-vm/lima).

Here is attrs after lfs.attributes() -call to a directory.

{
  "mode": "other",
  "ino": 1048576,
  "blksize": 0,
  "size": 270972520,
  "blocks": 255,
  "permissions": "----w-r-x",
  "rdev": 6774313,
  "nlink": 0,
  "uid": 6774313,
  "dev": 1702057286,
  "gid": 0
}

This is the generated cdef, but it seems to work like the previous one.

ffi.cdef([[ // generated in arm64 aws linux2
    typedef unsigned long int __dev_t;
    typedef unsigned int __uid_t;
    typedef unsigned int __gid_t;
    typedef unsigned long int __ino_t;
    typedef unsigned int __mode_t;
    typedef unsigned int __nlink_t;
    typedef long int __off_t;
    typedef int __blksize_t;
    typedef long int __blkcnt_t;
    typedef struct {
        __dev_t st_dev;
        __ino_t st_ino;
        __mode_t st_mode;
        __nlink_t st_nlink;
        __uid_t st_uid;
        __gid_t st_gid;
        __dev_t st_rdev;
        __dev_t __pad1;
        __off_t st_size;
        __blksize_t st_blksize;
        int __pad2;
        __blkcnt_t st_blocks;
        struct timespec st_atim;
        struct timespec st_mtim;
        struct timespec st_ctim;
        int __glibc_reserved[2];
    } lfs_stat;
]])