Using 'import' in MicroPython in the browser

[jmdevy]

Hi again,

I am looking into how one might get 'import' to work in MicroPython in the browser, also without fetching it from a server.

For example, say I have the following python files to be used in MicroPython

main.py

import test
print(test.value)

test.py

value = 10

the expected output is '10'.

On a filesystem, this project would look like

• ProjectDir
  • main.py
  • test.py

Now, say a user makes both of these files on some kind of text editor on a webpage and there is access to the contents of each file through javascript. How could 'main.py' know about 'test.py' in the browser?

I know of BrowserFS (https://github.com/jvilk/BrowserFS) but am not entirely sure how this would work with rp2040js.

I have seen ways of overriding the MicroPython importer to use provided modules to fetch the contents of files from a URL (https://github.com/micropython/micropython/issues/4972#issuecomment-626111144). I suppose that may be used in some way by using a Blob (https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL) to provide a client-side URL for the file contents to be fetched from in the MicroPython script (?)

In any case, I am looking for insight on how to get 'import' working in python scripts with MicroPythin using this library in browser

[jmdevy]

Just tested 'file = open("data.txt", "w")' to try and create a file in MicroPython, it returns Erno 19 which means filesystem not mounted, as expected.

Maybe there is a way to write files to flash using rp2040js like when writing the uf2?

EDIT: Is there an example of the code that writes project files to flash? Like explained here: https://docs.wokwi.com/guides/micropython

[jmdevy]

I think I'm starting to figure it out, at least where to maybe start. Looks like the simulator uses littlefs.wasm: https://www.npmjs.com/package/littlefs for flash management

EDIT: Should I try to use rp2040.flash.set(data, address) or rp2040.writeUint8(address, value) and in what way does data need to be formatted for MicroPython to see it as a file? Where does it need to be located in flash? What address range?

[urish]

Good observation - MicroPython indeed uses LittleFS.

On the Raspberry Pi Pico, I found out the it works this way:

1. The file system starts at offset 0xa0000 of the flash
2. The block size is 4096
3. The amount of blocks is 352

You can also deduce these values from the MicroPython source code:

1. Filesystem size
2. Filesystem offset
3. Block size (and also here)

[jmdevy]

OK, using that information I've tried to write a text file to flash after the uf2 in this way

text.txt

This is text

Trying to load text file to flash

  // Try transferring text file to flash
  const textRes = await fetch("test.txt");
  const textBuffer = await textRes.arrayBuffer();
  const textData = new Uint8Array(textBuffer);

  var flashIndex = 0;

  while (flashIndex< textData.length) {
    const dataBlock = textData.slice(flashIndex, flashIndex + 256);
    console.log(flashIndex + 0xa0000);
    rp2040.flash.set(dataBlock, flashIndex + 0xa0000);
    flashIndex = flashIndex + 256;
  }

That is just writing the contents of the file to flash, I don't expect MicroPython will be able to tell this is a file without some extra (unknown to me) metadata/formatting.

Running

import os
print(os.listdir('/'))

outputs

[]

My main point of confusion is what data should accompany the file contents for it to show up in MicroPython?

I am aware there is extra complexity that I may be missing when it comes to ensuring data is written to flash in a correct manner (blocks, fragmentation, etc)

I'm curious how littlefs.wasm could be used

[urish]

That is just writing the contents of the file to flash, I don't expect MicroPython will be able to tell this is a file without some extra (unknown to me) metadata/formatting.

That's exactly where littlefs comes into play. I haven't had a change to properly release the code for littlefs-wasm, but here's a quick code example showing how to use it to create a simple filesystem:

const createLittleFS = require('littefs');

const BLOCK_COUNT = 352;
const BLOCK_SIZE = 4096;

const flash = new Uint8Array(BLOCK_COUNT * BLOCK_SIZE);

(async function () {
  const littlefs = await createLittleFS();
  function flashRead(cfg, block, off, buffer, size) {
    const start = block * BLOCK_SIZE + off;
    littlefs.HEAPU8.set(flash.subarray(start, start + size), buffer);
    return 0;
  }
  function flashProg(cfg, block, off, buffer, size) {
    const start = block * BLOCK_SIZE + off;
    flash.set(littlefs.HEAPU8.subarray(buffer, buffer + size), start);
    return 0;
  }
  function flashErase(cfg, block) {
    const start = block * BLOCK_SIZE;
    flash.fill(0xff, start, start + BLOCK_SIZE);
    return 0;
  }
  const read = littlefs.addFunction(flashRead, 'iiiiii');
  const prog = littlefs.addFunction(flashProg, 'iiiiii');
  const erase = littlefs.addFunction(flashErase, 'iii');
  const sync = littlefs.addFunction(() => 0, 'ii');

  const writeFile = littlefs.cwrap(
    'lfs_write_file',
    ['number'],
    ['number', 'string', 'string', 'number']
  );

  const config = littlefs._new_lfs_config(read, prog, erase, sync, BLOCK_COUNT, BLOCK_SIZE);
  const lfs = littlefs._new_lfs();
  littlefs._lfs_format(lfs, config);
  littlefs._lfs_mount(lfs, config);
  const fileData = 'This is text\n';
  writeFile(lfs, 'test.txt', fileData, fileData.length);
  littlefs._lfs_unmount(lfs);
  littlefs._free(lfs);
  littlefs._free(config);
})();

when this code finishes running, the flash array will contain the raw content of the filesystem, which you can then copy to location 0xa0000 of the simulated Pi Pico flash.

[urish]

(You can get the littlefs package from npm)

[jmdevy]

That all worked! I had to use browserify to get it to work in the browser