apps: Add uploadfiles to simplify file loading to NuttX

[acassis]

Summary

Add uploadfiles to simplify file loading to NuttX

Impact

It will allow user to include files to use on NuttX without using SDCard, Flash, etc

Testing

STM32F4Discovery

[acassis]

@xiaoxiang781216 @patacongo Is there some way to use the ROMFS directly in the flash without using the RAMDISK? @JorgeGzm asked me: since it is already in the flash, should it be mounted automatically? Then I realized: since it is in flash, it could be accessed directly without mounting a RAMDISK. Something like a MTD that access directly the internal flash, like the progmem, but simpler.

[acassis]

I just realized the after mounting to ROMFS, the amount of RAM used is not exactly the size of the file in the ROMFS, actually it is way less then the size of the file:

NuttShell (NSH) NuttX-12.5.1
nsh> free
                   total       used       free    maxused    maxfree  nused  nfree
        Umem:     516896      11448     505448      12952     367216     44      4
nsh> upload
Registering romdisk at /dev/ram0
Mounting ROMFS filesystem at target=/data with source=/dev/ram0
nsh> free
                   total       used       free    maxused    maxfree  nused  nfree
        Umem:     516896      11688     505208      13912     367216     51      5
nsh> ls -l /data
/data:
 -r--r--r--        4300 yes.wav
nsh>

11688−11448 = 240

[patacongo]

@xiaoxiang781216 @patacongo Is there some way to use the ROMFS directly in the flash without using the RAMDISK?

In order to mount a file system, you need a block (or MTD) driver. A block driver is a better choice for RAM or FLASH-based since erasure is not needed. So no, it cannot be used "directly". I am not even sure what that would mean.

[patacongo]

I just realized the after mounting to ROMFS, the amount of RAM used is not exactly the size of the file in the ROMFS, actually it is way less then the size of the file:

That is expected. Nothing needs to be allocated. The ROMFS volume is in FLASH; nothing nothing additional needs to be instantiated. It simply needs to be associated with a block driver. That is when the ROMDISK does; it makes the volume in FLASH accessible as any other block device.

[patacongo]

Add uploadfiles to simplify file loading to NuttX

I don't understand the purpose of this. A more standard way of doing this is to mount create a RAMDISK at /data, then load the files into the RAM disk. Does this really need to be re-invented?

[acassis]

Add uploadfiles to simplify file loading to NuttX

I don't understand the purpose of this. A more standard way of doing this is to mount create a RAMDISK at /data, then load the files into the RAM disk. Does this really need to be re-invented?

The idea is copy the files in the computer directly to some uniform/default place to be integrated into ROMFS, without extra effort. Imagine you want some samples audio file and you don't have Flash or SDCard in the board, then just copy it to apps/uploadfiles and it will be in the firmware to be used later

[patacongo]

The idea is copy the files in the computer directly to some uniform/default place to be integrated into ROMFS, without extra effort. Imagine you want some samples audio file and you don't have Flash or SDCard in the board, then just copy it to apps/uploadfiles and it will be in the firmware to be used later

That memory should not be in the heap (unless it is preallocated). The memory management logic does modify metadata throughout the heap. Positioning it in some reserved, separate memory location would safe. Or preallocating the memory by creating a ROM disk. You can get the address of the allocated memory using the BIOC_XIPBASE ioctl command, then copy into the disk image.

NOTE: the copied file must be a file system image. You can't copy files into it.

[acassis]

Yes, it is a ROM Disk, the genromfs just create a romfs.image with the files that user copied to there. It is like the bastest example, but instead generating the romfs.img with fixed test files, it will create with any file the user copy to there.

[patacongo]

This kind of logic will not work in the KERNEL build. It should work in a FLAT build. I think the PROTECTED build as well although the volume would probably need to be in user space which is a little unusual. Perhaps the option should not be selectable in the KERNEL build.

[xiaoxiang781216]

@acassis the official way to convert a file to a block device and mount a file system is using the loop device: https://nuttx.apache.org/docs/latest/components/drivers/character/loop.html which should be worked even in protected and kernel mode.

[patacongo]

which should be worked even in protected and kernel mode.

I think Alan wants to load the file into physical memory using JTAG. This would work in KERNEL mode only if the file is loaded into contiguous, physical memory underlying a region in the kernel address space.

User space addresses are invalid after a context switch, for example.

This would have to be handled by kernel logic, not by logic in apps/. Using a loop device within the OS is awkward. A ROM disk would accomplish the same thing.

[acassis]

This kind of logic will not work in the KERNEL build. It should work in a FLAT build. I think the PROTECTED build as well although the volume would probably need to be in user space which is a little unusual. Perhaps the option should not be selectable in the KERNEL build.

You are right, it should has "depends on BUILD_FLAT

[acassis]

which should be worked even in protected and kernel mode.

I think Alan wants to load the file into physical memory using JTAG. This would work in KERNEL mode only if the file is loaded into contiguous, physical memory underlying a region in the kernel address space.

User space addresses are invalid after a context switch, for example.

This would have to be handled by kernel logic, not by logic in apps/. Using a loop device within the OS is awkward. A ROM disk would accomplish the same thing.

My idea is just using this ROM Disk solution to include files inside NuttX. Maybe this ROM Disk should be included in the kernel side instead of apps. What do you think?

[patacongo]

Maybe this ROM Disk should be included in the kernel side instead of apps. What do you think?

That might be the cleaner solution and would work in all build modes.