WebREPL improvements - Githubissues

robert-hh commented 5 months ago

I tried the IDE using WebREPL and generally it works fine. Even If I do not use IDEs at all for MicroPython, I can admit that is is usable. I tried it with an ESP8266 and W600. For ESP8266 it works well, with W600 there were problem when saving files. But maybe I have to repeat the test or there is a problem with the W600 port.

A few minor hiccups came up:

I get confused by the fact, that the WebREPL icon shows "green" like connected even if it is not. Like after Soft Reset or Hard Reset. So when a user pushes these buttons, the WebREPL state should change to "Disconnected", and the WebREPL button should then allow a reconnect, even if the "Do not allow reconnects" was selected on login.
Generally WebREPL could/should have a "Disconnect" button or mode.
Since WebREPL connections have to be made quite often in a session, remembering the last IP Address would also be helpful.

robert-hh commented 5 months ago

One other point now that I restarted it again: On start it shows a random file, starting with "# ViperIDE - MicroPython Web IDE". For me it is more confusing that helpful. If at all, it could be a short help text written as Python comments or help string. The Python comment as grey color on black is hard to read.

vshymanskyy commented 5 months ago

Yes, thanks for the valuable comments. This can definitely be improved. Are you running ViperIDE locally or directly from viper-ide.org ?

robert-hh commented 5 months ago

Locally. On the first attempt running from viper-ide.org did not work properly. But I did not go into an analysis but switched to the local copy.

robert-hh commented 5 months ago

Oh yes: F5 for "run file" disconnects WebREPL. Browser: Firefox.

robert-hh commented 5 months ago

It is a good approach to show non-text files like .mpy as hex-dump. But it cannot be edited of in case of a .mpy not executed. I'm not suggesting that. It's surely not suitable for an IDE aiming at ease of use.

Version 0.3.3. works better with a W600. My first attempt was with version 0.3.2.

robert-hh commented 5 months ago

What I miss or did not see yet was an option to create a new file. It is possible to first create an empty file in the REPL terminal and then refresh the file pane (it took me a while to find the refresh button). But that's tedious.

robert-hh commented 5 months ago

Just noticed: With today's version you skipped the "do not allow reconnects" selection.

robert-hh commented 5 months ago

You asked about the versions: With the online version of ViperIDE an attempt to use webrepl just switches over to the MicroPython WebREPL app. I do not know if that is intentional.

vshymanskyy commented 5 months ago

To create a file, hit the "+" button next to the folder. Then enter the relative path you want to create (it can include "/"). I.e. "aaa/bbb/ccc/" will create 3 folders. "aaa/test.py" will create a folder and a file

vshymanskyy commented 5 months ago

I do not know if that is intentional.

It's not. You need to use the viper-tools package which provides a tuned version on WebREPL. Check out the docs. This will not navigate to the MPY webrepl app.

vshymanskyy commented 5 months ago

Just noticed: With today's version you skipped the "do not allow reconnects" selection.

Could you elaborate plz?

robert-hh commented 5 months ago

I did a "pull" from the repository again (second time today) and it's back. Edit: The create new file option works. I did not notice the icon, liek it took a while for me to identify the "refresh" icon.

vshymanskyy commented 5 months ago

Ok, take your time playing with the latest version. It would be helpful if you could provide a summary/recap based on that ;)

vshymanskyy commented 5 months ago

Please also check out the new Remote P2P connection: https://github.com/orgs/micropython/discussions/15329

robert-hh commented 5 months ago

Please also check out the new Remote P2P connection:

I have no clue how that should be used.

vshymanskyy commented 5 months ago

Open the bridge page in one tab, ViperIDE in another tab.

Connect Bridge to the device.
Connect ViperIDE to the Bridge using P2P ID.

But it can actually be a different device in a completely different location and network.

robert-hh commented 5 months ago

Please also check out the new Remote P2P connection:

I tried it with both the target board connected by USB and by REPL, using an Android Tablet for the access. It kind of works. The remote can connect and with the USB connection I get a file pane content & can switch it. But as soon as I do a little bit more, the connections breaks and cannot be restored. So it's still a long way to go and maybe too early to include it in the Viper IDE's main line. What I do not understand is the use case. If the intention is to have the board sitting somewhere remote and being able to access it and operate it, then the random P2P ID looks like a show stopper. That would have to be communicated for each (re-)connection. Besides that, the the Bridge App seems to act as a kind of router. But routers can do the same thing with VPN tunnels and are usually pretty stable.

vshymanskyy commented 5 months ago

Yes, i'm just exploring it at the moment. Currently the only use case for this is remote debugging of the IDE itself, i.e. with some specific device/firmware configuration. I can try to make the random ID stable (i.e. not changing unless you manually re-generate it)

vshymanskyy commented 5 months ago

I'm leaning towards running a WebSocket streaming server, so that remote devices can be connected to ViperIDE without the Bridge.

p.s. Ideally, running a WebRTC client on the device directly.. But this is much harder to achieve

robert-hh commented 5 months ago

The viper IDE itself looks generally good. Especially the file transfer seems to work, which seems broken at the standard webrepl app. So it's just minor work flow topics that may be useful. I mentioned a few above, like the disconnect switch for webrepl and remembering the most recent IP address for Webrepl. Some more little things:

When one shrinks the terminal window, the line with the cursor disappears at the bottom and does not scroll up, even if the device generates output. The bottom line of the terminal must not drop below the lower border.
Save-as may be helpful.
Having a bright color scheme. At a bright place, the dark scheme is hardly readable, especially the grey-on-black areas.
Search (Ctrl-F) is limited to the actual visible content. That's kind of confusing.
Having a file pane for local files may be useful.

vshymanskyy commented 5 months ago

Thanks!

vshymanskyy commented 5 months ago

It is a good approach to show non-text files like .mpy as hex-dump. But it cannot be edited of in case of a .mpy not executed. I'm not suggesting that. It's surely not suitable for an IDE aiming at ease of use.

Could you please also elaborate on this? What would you expect? Please provide an example use case, maybe

robert-hh commented 5 months ago

The Vider IDE allows to show, edit, save and run Python code as text files. Binary Files as displayed as hex dump. This includes compiled python code, the .mpy files. An extension could be further actions with binary files.

.mpy files could be saved and executed. Maybe you could show the MPY version number of the file.
generic binary files could be modified & saved.

That's maybe only useful if they can be loaded from the PC's file system. But in the interest of a slim IDE which supports the core functions reliably and fast without much "under-the-hood" activity you could as well decide to keep things as they are. This "under-the-hood" activity makes Thonny a constant source of trouble.

vshymanskyy commented 5 months ago

Comments are now much more readable
WebREPL and P2P now remembers the previous user input
For search, the CodeMirror built-in search bar is now enabled. It is cumbersome to use, but eat least consistent. Read more here: https://codemirror.net/5/demo/search.html

Regarding P2P, there's some binary encoding issue, once that is resolved it should work much more stable.

vshymanskyy commented 5 months ago

.mpy files could be saved and executed.

Editing HEX is cumbersome and particularly mind-blowing for .mpy format which is not even documented AFAIK ;) So yes, no editing for binary files, just a viewer. BUT, maybe it's beneficial to allow running .mpy files. This is easy to add, but adds little value from my understanding.

vshymanskyy commented 5 months ago

@robert-hh could you please run https://viper-ide.org/benchmark.html with your W600 ? It seems to work stable via USB. Didn't try with WebREPL yet.

{"machine":"Generic W600 device with W600-x8","release":"1.23.0-preview","sysname":"w600","version":"MicroPython 17abadc-dirty on 2024-05-26","mpy_ver":6,"sys_path":["/lib",".frozen",""],"connection":"usb"}
Writing binary file 10240 bytes... 3546.6ms
Reading... 1927.0ms
Writing ascii file 10240 bytes... 2782.8ms
Reading... 1927.1ms
Listing FS... 296.6ms
Cleanup... 297.4ms

 === OK ===

robert-hh commented 5 months ago

Using WebRPL:

{"machine":"Wemos W600 Pico with W600-B8","release":"1.24.0-preview","sysname":"w600","version":"MicroPython v1.24.0-preview.79.g4d2a2fe48.dirty on 2024-06-20","mpy_ver":6,"sys_path":["",".frozen","/lib"],"connection":"ws"}
Writing binary file 10240 bytes... 30547.0ms
Reading... 2386.0ms
Writing ascii file 10240 bytes... 18494.0ms
Reading... 2387.0ms
Listing FS... 357.0ms
Cleanup... 506.0ms

Using USB

{"machine":"Wemos W600 Pico with W600-B8","release":"1.24.0-preview","sysname":"w600","version":"MicroPython v1.24.0-preview.79.g4d2a2fe48.dirty on 2024-06-20","mpy_ver":6,"sys_path":["",".frozen","/lib"],"connection":"usb"}
Writing binary file 10240 bytes... 3142.3ms
Reading... 1961.3ms
Writing ascii file 10240 bytes... 2430.4ms
Reading... 1938.7ms
Listing FS... 153.4ms
Cleanup... 213.9ms

vshymanskyy commented 5 months ago

Yes I also was able to test it via WebREPL:

{"machine":"Generic W600 device with W600-x8","release":"1.24.0-preview","sysname":"w600","version":"MicroPython v1.24.0-preview.79.g4d2a2fe48 on 2024-06-19","mpy_ver":6,"sys_path":["",".frozen","/lib"],"connection":"ws"}
Writing binary file 10240 bytes... 38211.5ms
Reading... 2404.5ms
Writing ascii file 10240 bytes... 22677.4ms
Reading... 2342.9ms
Listing FS... 277.9ms
Cleanup... 620.2ms

 === OK ===

Observations:

this is very slow and fails with timeout sometimes.
it also clobbered the w600 file system once...

robert-hh commented 5 months ago

Editing HEX is cumbersome and particularly mind-blowing for .mpy format which is not even documented AFAIK ;) So yes, no editing for binary files, just a viewer. BUT, maybe it's beneficial to allow running .mpy files. This is easy to add, but adds little value from my understanding.

I did not suggest HEX editing for .mpy. If at all, it would only be useful for other binary data. Running .mpy files would only be consistent with the .py file feature to run a program/script. Obviously that can be done as well by import mpy-file in the Terminal window.

I can confirm:

Readability of comment is better.
Search in the text windows works. You could reset the last searched item on file change. I agree that the control codes are a bit strange and may clash with the control codes of the browser. Especially Ctrl-R for as common key for Replace should be avoided.
The IP address is remembered, as well as the password.

The issues with the disappearing command line still persists.

robert-hh commented 5 months ago

The WebREPL performance test with a ESP8266 is not much faster:

{"machine":"ESP module with ESP8266","release":"2.2.0-dev(9422289)","sysname":"esp8266","version":"MicroPython v1.24.0-preview.78.g3a6dcbebd.dirty on 2024-06-19","mpy_ver":6,"sys_path":["",".frozen","/lib","/"],"connection":"ws"}
Writing binary file 10240 bytes... 29440.0ms
Reading... 2017.0ms
Writing ascii file 10240 bytes... 17348.0ms
Reading... 2015.0ms
Listing FS... 439.0ms
Cleanup... 506.0ms

 === OK ===

vshymanskyy commented 5 months ago

On W600

In general, I found WebREPL on W600 rather unstable:

not always connects
sometimes fails with a timeout

There's no specific issue with the Save File function. It works much better with RPi Pico W and ESP32 so I assume this is W600 issue.

Other items:

WebREPL icon shows "green" after soft/hard reset - Can't reproduce. W600 gets immediately disconnected when I hit soft reset
F5 for "run file" disconnects WebREPL (Firefox) - Can't reproduce
When one shrinks the terminal window, the line with the cursor disappears, Moves off screen - IN TODO
Ability to disconnect - IN TODO

vshymanskyy commented 5 months ago

disappearing command line - should be fixed now

robert-hh commented 5 months ago

F5 is the page reload hotkey on Firefox.

vshymanskyy commented 5 months ago

Keyboard shortcuts are fixed. Thanks for reporting.

vshymanskyy commented 5 months ago

F5 and Ctrl+S are fixed now.
added an ability to disconnect (just click any of the connect buttons again)

I think i finished my todo list. Some other requests are premature/hard to implement at the moment.

robert-hh commented 5 months ago

Very good. Works as intended. This is now in most aspects the better WebREPL client.

The only thing missing is a simple ability to transfer files between board and PC. Of course that could be done by cut & paste between the Viper IDE edit window and a PC's editor, but that is tedious.

vshymanskyy commented 5 months ago

I'm planning to add drag'n'drop for files and maybe folders directly from the OS.

robert-hh commented 5 months ago

Sounds good.

vshymanskyy commented 5 months ago

Thanks for all the testing and feedback.

vshymanskyy commented 4 months ago

On start it shows a random file, starting with "# ViperIDE - MicroPython Web IDE". For me it is more confusing that helpful. If at all, it could be a short help text written as Python comments or help string.

@robert-hh your wishes come true

P.S. We have automatic syntax checking and an emulator now: https://viper-ide.org/?emulator=1

robert-hh commented 4 months ago

This new a start page looks fine. But I have a question. I recall that I could open ViperIDE from the local file store to run it offline. That does not work any more. On an attempt to open ViperIDE.html I just get a blank browser screen.

vshymanskyy commented 4 months ago

This can no longer work, because the project migrated to use CodeMirror 6 and this requires a bundler. You can download the html file from viper-ide.org and it should work standalone. However, viper-ide.org itself should no longer require an internet connection. The website should work/load even when you are offline

vshymanskyy / ViperIDE

WebREPL improvements #10

On W600

Other items: