Open bollwyvl opened 4 years ago
My workshop for the next robot conference was accepted, which confirms that I will be attending the sprints there. Apparently there is still no installer available for Robot Framework, so I've been thinking, if I should try to fork this project and try this "multiple size-based releases" for that. For plain robot it would make sense to have minimal version with just robot framework, but also a version for getting started with selenium testing (the real reason for the installer).
workshop for the next robot conference was accepted
Hooray!
fork this project
You know I'm glad to help, and a fork is just more repos to watch: I don't see any reason why we wouldn't start it here, and why we wouldn't want synchronized, automated releases of the whole shooting match. If somebody wants to then fork that, at some point, that's great.
We can even stop doing the kinda gross templating, and just manage the yaml generation more simply in python directly, if we start feeling confident. The CI build would take longer, but i think there's value in managing a single pipeline for all the stuff together.
minimal version with just robot framework
yup, as a CI thing, that would be great. MiniRobotConda
. Even if it doesn't ship conda
(which it probably should, if only for activation and administration chores) it can't get much smaller than about 70mb, but that's not so bad!
version for getting started with selenium testing
so even there i would imagine two flavors:
WebRobotConda
(just rf, rfsl, sl and maybe rfstl, if that plays out, and geckodriver)FireWebRobotConda
(as above, but also firefox)i'm really liking having firefox in our installer, and if we can do our little part to help more people test on firefox, i'm all for it. The only gotcha is on Linux, you do still have to have gtk3
installed, even when running --headless
, but i'll still take it.
Unless we ship all the chromedrivers, we can't really ship any chromedrivers, but we could ship webdrivermanager, and indeed ship all of them pre-cached.
Thanks. I can agree with your reasoning. I also prefer Firefox being bundled because now things just work. It also makes sense that whatever version or browser you use for surfing in the web should not interfere with learning Selenium testing. Finally, promoting Firefox would not hurt the diversity in browser ecosystem...
That said I cannot really focus on this before November, but will then.
That said I cannot really focus on this before November, but will then.
I hear you, my friend. Who knows, maybe i'll be able to throw some stuff at it. What is your priority for getting something out? Maybe hack on the summary of this issue to make it clear. I guess I would lean towards the smallest functioning robot environment, just so we have a size/complexity baseline, but if the web one is more useful we could start there.
I’d vote for the baseline. We already know that we can make a GB sized installer with everything :)
After the baseline, another baseline with SL, gecko and Firefox, and then documented approach to fork and extend those baselines.
Finally, at the conf it makes sense to discuss what libraries should be bundled with the default starter (for learning and getting started purposes):
i did a little playing with this last night:
609M Sep 30 23:58 RobotLab-2019.9.1-Linux-x86_64.sh
84M Oct 1 00:05 MiniRobot-2019.9.1-Linux-x86_64.sh
161M Oct 1 00:35 ExoRobot-2019.9.1-Linux-x86_64.sh
Mini
is rf + conda. not shipping conda only saves about 2mb.
Exo
is Mini
, plus every optional dependency i could find in the rf codebase. The heavy hitter is definitely wx
, which screenshots uses. Might as well throw in RIDE at that point...
Very cool! Which screenshots feature relies on wx? Does dialogs library work? The point for a installer is to give environment that just works, so from that point of view 200M is not much nowadays.
And now that you mentioned RIDE... does it already support Python 3? Sure many would like it to be included and it’s been hard to install on many systems because of wx. (Although that probably was because for long it required very old version of that.)
asn1crypto-0.24.0-py36_1003.tar.bz2 atk-2.32.0-haf93ef1_0.tar.bz2 ca-certificates-2019.9.11-hecc5488_0.tar.bz2 cairo-1.16.0-hfb77d84_1002.tar.bz2 certifi-2019.9.11-py36_0.tar.bz2 cffi-1.12.3-py36h8022711_0.tar.bz2 chardet-3.0.4-py36_1003.tar.bz2 conda-4.7.12-py36_0.tar.bz2 conda-package-handling-1.6.0-py36h516909a_0.tar.bz2 cryptography-2.7-py36h72c5cf5_0.tar.bz2 docutils-0.15.2-py36_0.tar.bz2 expat-2.2.5-he1b5a44_1003.tar.bz2 fontconfig-2.13.1-h86ecdb6_1001.tar.bz2 freetype-2.10.0-he983fc9_1.tar.bz2 fribidi-1.0.5-h516909a_1002.tar.bz2 gdk-pixbuf-2.36.12-h7a26e22_1003.tar.bz2 gettext-0.19.8.1-hc5be6a0_1002.tar.bz2 glib-2.58.3-h6f030ca_1002.tar.bz2 gobject-introspection-1.58.2-py36h5503ade_1002.tar.bz2 graphite2-1.3.13-hf484d3e_1000.tar.bz2 gst-plugins-base-1.14.5-h0935bb2_0.tar.bz2 gstreamer-1.14.5-h36ae1b5_0.tar.bz2 gtk2-2.24.32-h586f36d_1.tar.bz2 harfbuzz-2.4.0-h9f30f68_3.tar.bz2 icu-64.2-he1b5a44_1.tar.bz2 idna-2.8-py36_1000.tar.bz2 jpeg-9c-h14c3975_1001.tar.bz2 libffi-3.2.1-he1b5a44_1006.tar.bz2 libgcc-ng-9.1.0-hdf63c60_0.tar.bz2 libglu-9.0.0-hf484d3e_1000.tar.bz2 libiconv-1.15-h516909a_1005.tar.bz2 libpng-1.6.37-hed695b0_0.tar.bz2 libstdcxx-ng-9.1.0-hdf63c60_0.tar.bz2 libtiff-4.0.10-h57b8799_1003.tar.bz2 libuuid-2.32.1-h14c3975_1000.tar.bz2 libxcb-1.13-h14c3975_1002.tar.bz2 libxml2-2.9.9-hee79883_5.tar.bz2 libxslt-1.1.33-h31b3aaa_0.tar.bz2 lxml-4.4.1-py36h7ec2d77_0.tar.bz2 lz4-c-1.8.3-he1b5a44_1001.tar.bz2 ncurses-6.1-hf484d3e_1002.tar.bz2 openssl-1.1.1c-h516909a_0.tar.bz2 pango-1.42.4-ha030887_1.tar.bz2 pathlib2-2.3.5-py36_0.tar.bz2 pcre-8.41-hf484d3e_1003.tar.bz2 pip-19.2.3-py36_0.tar.bz2 pixman-0.38.0-h516909a_1003.tar.bz2 pthread-stubs-0.4-h14c3975_1001.tar.bz2 pycosat-0.6.3-py36h14c3975_1001.tar.bz2 pycparser-2.19-py36_1.tar.bz2 pyopenssl-19.0.0-py36_0.tar.bz2 pypubsub-4.0.3-py_0.tar.bz2 pysocks-1.7.1-py36_0.tar.bz2 pyte-0.8.0-py_0.tar.bz2 python-3.6.7-h357f687_1005.tar.bz2 pyyaml-5.1.2-py36h516909a_0.tar.bz2 readline-8.0-hf8c457e_0.tar.bz2 requests-2.22.0-py36_1.tar.bz2 robotframework-3.1.2-py_0.tar.bz2 ruamel_yaml-0.15.71-py36h14c3975_1000.tar.bz2 setuptools-41.2.0-py36_0.tar.bz2 six-1.12.0-py36_1000.tar.bz2 sqlite-3.29.0-hcee41ef_1.tar.bz2 tk-8.6.9-hed695b0_1003.tar.bz2 tqdm-4.36.1-py_0.tar.bz2 urllib3-1.25.6-py36_0.tar.bz2 wcwidth-0.1.7-py_1.tar.bz2 wheel-0.33.6-py36_0.tar.bz2 wxpython-4.0.6-py36h8f66242_1.tar.bz2 xorg-kbproto-1.0.7-h14c3975_1002.tar.bz2 xorg-libice-1.0.10-h516909a_0.tar.bz2 xorg-libsm-1.2.3-h84519dc_1000.tar.bz2 xorg-libx11-1.6.8-h516909a_0.tar.bz2 xorg-libxau-1.0.9-h14c3975_0.tar.bz2 xorg-libxdmcp-1.1.3-h516909a_0.tar.bz2 xorg-libxext-1.3.4-h516909a_0.tar.bz2 xorg-libxrender-0.9.10-h516909a_1002.tar.bz2 xorg-renderproto-0.11.1-h14c3975_1002.tar.bz2 xorg-xextproto-7.3.0-h14c3975_1002.tar.bz2 xorg-xproto-7.0.31-h14c3975_1007.tar.bz2 xz-5.2.4-h14c3975_1001.tar.bz2 yaml-0.1.7-h14c3975_1001.tar.bz2 zlib-1.2.11-h516909a_1006.tar.bz2 zstd-1.4.0-h3b9ef0a_0.tar.bz2
Which screenshots feature relies on wx?
Looks like wx
, gtk
or pillow
would work. I don't think gtk
is supported on windows, but pillow
is handy, anyhow...
Does dialogs library work?
if all it takes is tk
! haven't tested.
RIDE... does it already support Python 3?
looks like they test with 3.6. and don't pin wx
. I haven't tried packaging it in a while... couldn't get hidpi to work correctly..
No I didn't put selenium in one of the new ones, but I do think shipping Firefox/geckodriver with it is probably the right thing to do, if it should Just Work after downloading (e.g in CI). Do you think wx/ride are worth the size hit for the web excursion? Or just also add it to Exo? You wouldn't actually use wx in CI.
Also what about SSHLibrary? It's first party, solid as hell, relatively light, super useful, and makes it easy to test iot stuff, as well as vms d cloud stuff... But you kinda have to know what you're doing because passwords. Put in Exo?
WhiteRobot? I've still not made it work on win10, and don't like how it's packages. But sounds so good!
SikuliRobot: I always consider it the Last Argument of Kings of test execution. Hard to maintain, but can probably do the thing. Packaging Java stuff is no problem, though jython probably is or will be soon, if you expect to be able to run anything else after 89 days!
The whole *Robot branding thing is a bit silly, of course, and wouldn't help people. If we were doing this for real, we'd probably need a choose-your-own-adventure page "what are you testing?" "What operating system are you testing on?" That kind of thing. Have a "just show me the robots" escape hatch, and let people pick the one they want from a table of key packages. Do the classic up-sell and point them towards RobotLab because it has more check marks :)
I definitely want to belay the Hub one, for now. I've made a totally self-contained jupyterhub work with ansible/systems spawner (haven't tried a k8s version yet). in certain kinds of training (schools), the participants may only have an older Chromebook, and wouldn't be able to install random software. Also building for ARM is still a touch outside of our capabilities. Anyhow: set up a NUC, run the installer, throw in your course content, turn on WiFi, and you're good to go in seconds per participant rather than tens-of-minutes.
On Thu, Oct 3, 2019, 00:22 Asko Soukka notifications@github.com wrote:
Am I correct that ExoRobot does not yet include Selenium testing dependencies? I recall robot users have a long hoped for an installer with Selenium dependencies. But adding Firefox with geckodriver into mix would probably make ExoRobot fat? So, should there be "SeleniumRobot"?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/robots-from-jupyter/robotlab/issues/42?email_source=notifications&email_token=AAALCRCIQ3CBAGLUSQZUKOTQMVXQZA5CNFSM4IYDQJ6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAG5ZWI#issuecomment-537779417, or mute the thread https://github.com/notifications/unsubscribe-auth/AAALCRC53EF77KHDIGYLF4TQMVXQZANCNFSM4IYDQJ6A .
Ok Let's not guess too far ahead :) I could see two separate installer tracks: one for RobotLab and the other one for more conservative Robot Framework users.
Quoting Nicholas Bollweg (2019-09-18 23:42:00)
MicrobotLab ð¬
- for CI environments
But "how micro" this could be, because "Lab"-branding would probably make this need nbconvert and nbrobot (the latter from robotkernel) to be able to execute notebooks. (I do actually use nbrobot in production RPA already.) No wx required here.
RobotLabLite ð
I'm not quite convinced that this is really needed. This could be used as a base to build custom environment, but why not just install miniconda then?
RobotLab ð
- as it is now!
- even shipping Firefox is great!
I agree everything on this. Big is good here. It's so good that the download is worth waiting for.
RobotHub ð¡
- ship a whole [3]littlest jupyter hub
This is interesting. I have never set up JupyterHub myself yet, but this would have very interesting educational and enterprise use cases :)
Then the more conservative Robot Framework track. Robot Framework is completely missing standalone installers and your work seems to be the best there is so far. It feels a bit weird to design and build it here, but I understood that it would be better than forking this into another repository to follow.
MiniRobot as it is. It's good to have something minimal with at least pip to install some libraries when you know what to do.
ExoRobot was interesting. Like "MiniRobot" was just a proof-of-concept and "ExoRobot" was the actually practical base to build an environment. I feel this makes more sense thatn "RobotLabLite", but I'm not sure if ride really belongs to here (even when it comes with wx).
WebRobot (terrible name). Comparable to RobotLab. Should come at least with Ride, SeleniumLibrary, geckodriver and Firefox. But maybe also with SSHLibrary, RESTinstance and SeleniumTestability. Really hard to draw line where to end here...
Should I ask the other workshops at the next Robocon if they would like to have your installer experience?
A lot to chew on there!
I can rework some of the existing ones with your guidance above.... Adding the web one is a no-brainer, we already have everything.
Maybe we need a further piece of the name on the non-lab ones, e.g WebRobotBox or something, as "*robot" is super overloaded, and we don't need to get into any trouble.
The only bounding condition on us shipping more installers is supporting them, to my mind. What we would not want is upstreams getting complaints from users when things don't work, if the installers are not a suggested install mechanism.
The workshop case is ideal, though, as the instructor is on the hook to provide support, and can't disappear. You should certainly offer it.
On Thu, Oct 3, 2019, 13:36 Asko Soukka notifications@github.com wrote:
Ok Let's not guess too far ahead :) I could see two separate installer tracks: one for RobotLab and the other one for more conservative Robot Framework users.
Quoting Nicholas Bollweg (2019-09-18 23:42:00)
MicrobotLab ð¬
- for CI environments
But "how micro" this could be, because "Lab"-branding would probably make this need nbconvert and nbrobot (the latter from robotkernel) to be able to execute notebooks. (I do actually use nbrobot in production RPA already.) No wx required here.
RobotLabLite ð
I'm not quite convinced that this is really needed. This could be used as a base to build custom environment, but why not just install miniconda then?
RobotLab ð
- as it is now!
- even shipping Firefox is great!
I agree everything on this. Big is good here. It's so good that the download is worth waiting for.
RobotHub ð¡
- ship a whole [3]littlest jupyter hub
This is interesting. I have never set up JupyterHub myself yet, but this would have very interesting educational and enterprise use cases :)
Then the more conservative Robot Framework track. Robot Framework is completely missing standalone installers and your work seems to be the best there is so far. It feels a bit weird to design and build it here, but I understood that it would be better than forking this into another repository to follow.
MiniRobot as it is. It's good to have something minimal with at least pip to install some libraries when you know what to do.
ExoRobot was interesting. Like "MiniRobot" was just a proof-of-concept and "ExoRobot" was the actually practical base to build an environment. I feel this makes more sense thatn "RobotLabLite", but I'm not sure if ride really belongs to here (even when it comes with wx).
WebRobot (terrible name). Comparable to RobotLab. Should come at least with Ride, SeleniumLibrary, geckodriver and Firefox. But maybe also with SSHLibrary, RESTinstance and SeleniumTestability. Really hard to draw line where to end here...
Should I ask the other workshops at the next Robocon if they would like to have your installer experience?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/robots-from-jupyter/robotlab/issues/42?email_source=notifications&email_token=AAALCRDICGDT7ZE7NF53HK3QMYUSRA5CNFSM4IYDQJ6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAI7MKI#issuecomment-538048041, or mute the thread https://github.com/notifications/unsubscribe-auth/AAALCRDJHTFILBLS42GYN33QMYUSRANCNFSM4IYDQJ6A .
@bollwyvl To avoid misunderstandings, don’t waste your time to rework anything yet. As you said, all the pieces are already there and probably even I could put them together if I really need to. Let’s continue to focus on what this should mean for the “Lab” and only return to the generic installers when there is real need (and a possibly a maintainer).
Disclaimer about the CI-version: I don’t have personal use for it, because we have Nix on our CI machines :/
Great, focus on lab good. I like the nbrobot-forward idea. Re: nix: indeed, once nix supports one click installers on windows, I'll happily use it to train people on things other than nix. :P
Back on topic: I guess to the marketing-as-design point, if we sketch up what we'd like a blog post/landing page to contain, that would help resolve some of the ambiguity... Will take a crack at it. Documentation need still remains the same, but maybe there's a story in there... The (somewhat neglected) irobotframework docs built with nbsphinx might be a good prototype.
Sry, have been deep in the language server stuff. I'll try to get back to this after a bit of a holiday this weekend: basically, I intend to keep everything in #52, but not actually build any of the general purpose stuff, land that, and then have a look at some of the other concepts we're developing here. Maybe do some design sketches on the road :)
Back on the LSP stuff: it's taken some really nasty stuff down in python/r/node land, and am only now getting to where I can help out with the browser testing. Been stealing a lot from this repo :P The end game there is to get all of that into JupyterLab itself, but there are a lot of nuts and bolts to figure out before then.
When that's good to go (on pip and npm) some time next month, I think we'll want the UI things it already offers (jump to definition, highlight references, etc) for .robot
, .robot.ipynb
, output.xml
and more possible stuff down the pipe (outline view, tidy). Shipping the existing robot language server implementation in node is no problem for robotlab, but there are things it can't do that kernels can, e.g. you have to libdoc your 3rd party stuff ahead of time. We can at least check it out! It should just be:
entry_point
wrapper in python (or do some jupyter_notebook_config.d
stuff)I can also imagine a future where kernels become implicit language servers and clients: with what we ended up with, it's a many-to-many relationship between documents and language servers, everything goes through the notebook server, what's a few more? Heck, there are even nix language servers (though can't tell which of the rust, go, or haskell implementations will win). Additionally, to that end, here's a cool thread about a guix kernel: https://groups.google.com/forum/#!topic/jupyter/jUfLBPSlbXY
And 'cuz I know you love looking at robot reports...
Whoops, meant to post this here, not there:
I did do some drawing: nothing super solid to show for it. There's not much you can do with a feature table, and there's certainly bounded complexity to how much you can stuff in there. There are a few hierarchies that came to mind, but i think one of the more logical ones was...
man
/libdoc/inspector/lsp/...)rm -rf
vs click-and-uninstall
A funny thing that came to mind, sort of related to the Hub concept, is a "Pro" spin (think a second, full day workshop), which includes the things you would need to extract, build, test, and package new robot libraries (or extensions to kernel) as part/in support of a team. On the backend, you'd have your tooling like git
, while on the front-end, I'd stuff in jupyterlab-git. Ship pytest
, robotframework-statuschecker
, robotframework-lint
, black
, etc. Ship conda-build
, and maybe even constructor
(since we can't ship nix :stuck_out_tongue_winking_eye: ). Anyhow :pizza: for thought.
RobotLab is big and likely to get bigger as more cool features need more (exotic) dependencies.
We could just as easily spin a couple different intstallers a la Miniconda/Anaconda in CI.
MicrobotLab 🔬
RobotLabLite 🔍
opencv,nodejs,pandoc), and even repackage some things (jupyterlab assets)RobotLab 🔭
RobotHub 📡