search

mu-editor / mu

A small, simple editor for beginner Python programmers. Written in Python and Qt5.
http://codewith.mu
GNU General Public License v3.0
1.42k stars 437 forks source link

Windows installer - what python libraries to 'package' #340

Closed martinohanlon closed 6 years ago

martinohanlon commented 6 years ago

A new windows installer has been created for Mu which packages up Python and creates a single installer to allow those in restricted environments (i.e. schools) to "get started with python programming using mu". All good stuff.

A constraint of taking this approach is that you also need to package the Python libraries that a school might want to use into the installer as well as the self contained Mu cant use system installed packages.

Its worth noting that the preferred method of installing mu is to use pip so you get the full power of python and all of the flexibility but it is simply not achievable for many.

This is also not a 1 chance opportunity, if there are libraries missing, re-cutting an install is a relatively simple process.

So... What libraries to package? Please insert your views here.

martinohanlon commented 6 years ago

To kick us off:

tkinter turtle gpiozero guizero pygame pygamezero networkzero

martinohanlon commented 6 years ago

@whaleygeek @bennuttall @waveform80 @davidhoness @MarcScott @rikcross @jrobinson-uk @mrichardson23 @tracygardner @caitlynraspberrypi @topshed @lhyams

tagging you in, in an attempt to get a wider set of views.

DavidWhaleMEF commented 6 years ago 
bitio
mcpi
bottle
py-sonicpy
DavidWhaleMEF commented 6 years ago

A good source of ideas would be to mine the MagPi magazines and grep 'pip install' and see what comes back, then curate a smaller list from that.

The same could be done by a grep 'pip install' through the github repos that back this page: https://projects.raspberrypi.org/en/projects

bennuttall commented 6 years ago

@DavidWhaleMEF What is py-sonicpy? Do you mean python-sonic? https://pypi.python.org/pypi/python-sonic

I think just using python-osc is preferred now the OSC protocol fully supported in Sonic Pi.

bennuttall commented 6 years ago

Suggestions:

mrichardson23 commented 6 years ago

I'm a big fan of and use requests a ton for projects. Would it be helpful here as well?

martinohanlon commented 6 years ago

@bennuttall does the sense_emu run on Windows?

bluedot is Linux only due to the requirement to talk to the bluetooth hardware via dbus.

MarcScott commented 6 years ago

guizero

On 14 Feb 2018 4:19 pm, "Matt Richardson" notifications@github.com wrote:

I'm a big fan of and use requests a ton for projects. Would it be helpful here as well?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mu-editor/mu/issues/340#issuecomment-365659380, or mute the thread https://github.com/notifications/unsubscribe-auth/AETp8oJqK6Um7rIca9o_xDLuM-y1ySKQks5tUwdkgaJpZM4SFEHK .

bennuttall commented 6 years ago

@bennuttall does the sense_emu run on Windows?

I believe so. You're in a better position than me to try it :)

martinohanlon commented 6 years ago

I believe so. You're in a better position than me to try it :)

I am, and it doesn't!

ZanderBrown commented 6 years ago

sense_emu uses pygobject I believe and should run on Windows but as mu is Qt based (rather than Gtk) any installer would be rather large to include the required DLLs

waveform80 commented 6 years ago

sense_emu does run on windows (I tested on linux and windows while writing it) but installing GTK3 is an absolute pig and requires very specific versions of several things (also, I've no idea how easy it would be to integrate its installer with other things). Unfortunately Windows support in GTK has always felt very "oh, if we really have to" (in rather stark contrast to Qt).

On the more general topic, do you want some more "common" libraries included?

P.S. I thought tkinter was included in the base python package (and turtle)? (I know Debian splits out tk for dependency reasons, but I was fairly sure it was included by default on other platforms).

ntoll commented 6 years ago

Hi Folks,

This is great and thank you for all the suggestions.

However, there's a danger of including everything but the kitchen sink. Just so we have some sense of scope -- we're looking for Python modules that can be installed on Windows 32 and 64 bit via pip (i.e. there are existing wheels in PyPI).

When I saw the mention of GTK on Windows I snorted my tea over my laptop. There's definitely a cost/benefit analysis to do here ;-) Basically, let's go for the easy low-hanging fruit and anything that requires more onerous dependencies (such as a whole new GUI framework) we can debate once we've a "first cut". ;-)

I'm very enthusiastic to use Mu's new Windows installer as a way to "liberally" piggy-back other useful Python based modules, especially if they're the basis of Raspberry Pi learning resources, but I also want to get it right for the easy stuff before we tackle more complicated nice-to-have packages.

Does this make sense? Comments welcome! :-)

N.

waveform80 commented 6 years ago

Yup, perfect sense, and I'd second that GTK on Windows is likely more trouble than it's worth. I'd suggest numpy and Pillow are likely used in several learning resources, not sure about requests or matplotlib (and the latter can get horribly large with its GUI options) so they could be chopped

Dave.

On Thu, 15 Feb 2018, 12:08 Nicholas Tollervey, notifications@github.com wrote:

Hi Folks,

This is great and thank you for all the suggestions.

However, there's a danger of including everything but the kitchen sink. Just so we have some sense of scope -- we're looking for Python modules that can be installed on Windows 32 and 64 bit via pip (i.e. there are existing wheels in PyPI).

When I saw the mention of GTK on Windows I snorted my tea over my laptop. There's definitely a cost/benefit analysis to do here ;-) Basically, let's go for the easy low-hanging fruit and anything that requires more onerous dependencies (such as a whole new GUI framework) we can debate once we've a "first cut". ;-)

I'm very enthusiastic to use Mu's new Windows installer as a way to "liberally" piggy-back other useful Python based modules, especially if they're the basis of Raspberry Pi learning resources, but I also want to get it right for the easy stuff before we tackle more complicated nice-to-have packages.

Does this make sense? Comments welcome! :-)

N.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mu-editor/mu/issues/340#issuecomment-365908509, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBr3nvjKYFcxXPiPgWh3eS8NhnKcUkaks5tVB40gaJpZM4SFEHK .

ZanderBrown commented 6 years ago

It'd be great if Gtk3 was included but it would take a reasonable amount of work, perhaps somebody from the Gtk team could advise

ntoll commented 6 years ago

@ZanderBrown so you inadvertently touch on an important point.... which version..? Raspbian uses GTK2 yet the latest is GTK3. I've never used them above and beyond "hello world" type essays in the art just to get a feel for how it works, but I assume versions 2 and 3 have different APIs (otherwise, why the difference in version number?).

Right now, I'm erring on not touching GTK with a barge-pole simply because it's too complicated with very little in return IYSWIM. ;-)

ntoll commented 6 years ago

After discussions with @martinohanlon, these will be the initial Python packages included in the Mu installer:

martinohanlon commented 6 years ago

For clarity, we should note that:

Are already included as there are requirements for Mu itself.

martinohanlon commented 6 years ago

Those which are not included are done so for a number of reasons, mainly due to:

ZanderBrown commented 6 years ago

Unfortunately from the python perspective 2 Vs 3 is very different but as far as they are native 2&2 and 3&3 have inadvertently become locked so to speak (at the C level they are far more API compatible but different ABI and other internals)

It's true the Raspbian LXDE DE is largely Gtk2 but things like the Add / Remove Software, sense_emu & Calculator are Gtk3 as are most other DE for the Pi

From the wider Linux perspective Gtk3 is Wayland native and the binding is achieved through PyGObject giving support for various other libs 'for free' so in that sense is the obvious candidate

Is there any possibility of having an 'extras' pack for these GObject based things? Things like sense_emu would be great to have available but the extra deps would make the install quite large when it's not needed by many. This would also allow us to make it available at a later date

carlosperate commented 6 years ago

Out of curiosity, what's the plan for these dependencies with the deb installer for Raspbian? Are they all going to be installed from apt packages in the global python3 environment?

ntoll commented 6 years ago

@carlosperate that would be the most sensible thing to do AFAICT.

jrobinson-uk commented 6 years ago

Ditto, python Osc should be used

On 14 Feb 2018 15:10, "Ben Nuttall" notifications@github.com wrote:

@DavidWhaleMEF https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_davidwhalemef&d=DwMCaQ&c=DpyQ_ftY536pf7wCBQXXU58xADDRY77THQzJu1OmzOo&r=2wvnhXO3TbNcY7ANItBwBeGwJTsEqSzIgM7_LIWP7HM&m=dIi4bKwAZwKhoeYjZh1P_6dfegThM--fHCOQq79Y15U&s=lPo2XdvVmlw1oBi9KZb42koFEOrhse4prOJBlmL-AZA&e= What is py-sonicpy? Do you mean python-sonic? https://pypi.python.org/pypi/python-sonic https://urldefense.proofpoint.com/v2/url?u=https-3A__pypi.python.org_pypi_python-2Dsonic&d=DwMCaQ&c=DpyQ_ftY536pf7wCBQXXU58xADDRY77THQzJu1OmzOo&r=2wvnhXO3TbNcY7ANItBwBeGwJTsEqSzIgM7_LIWP7HM&m=dIi4bKwAZwKhoeYjZh1P_6dfegThM--fHCOQq79Y15U&s=0MBCGeGGaGrFvcD2Soe867NQxNZV7xv1K5U_ugy_zMY&e=

I think just using python-osc is preferred now it's fully supported in Sonic Pi.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mu-2Deditor_mu_issues_340-23issuecomment-2D365636696&d=DwMCaQ&c=DpyQ_ftY536pf7wCBQXXU58xADDRY77THQzJu1OmzOo&r=2wvnhXO3TbNcY7ANItBwBeGwJTsEqSzIgM7_LIWP7HM&m=dIi4bKwAZwKhoeYjZh1P_6dfegThM--fHCOQq79Y15U&s=bCYGZ1VvvXxCWLdQvZpb32e_j76vsqTZZG5iN1XKZQ8&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AEe9QN72D5syOFdLsGmqC-2DWZkxSmyZA1ks5tUvdugaJpZM4SFEHK&d=DwMCaQ&c=DpyQ_ftY536pf7wCBQXXU58xADDRY77THQzJu1OmzOo&r=2wvnhXO3TbNcY7ANItBwBeGwJTsEqSzIgM7_LIWP7HM&m=dIi4bKwAZwKhoeYjZh1P_6dfegThM--fHCOQq79Y15U&s=U6kawGPjcHUYp8dl6oZ0S9ez44bZmeFf9zNpB0U-Axc&e= .

carlosperate commented 6 years ago

@ntoll as this issue is for the Windows packages, and my previous question is likely to divert into a parallel conversation, I've opened https://github.com/mu-editor/mu/issues/348.

ntoll commented 6 years ago

@carlosperate +1 ;-)

yonghuming commented 6 years ago

@ntoll i have read all the souce of python turtle and in it it say you can use any GUI framework replace tkinter, thus mu is writed using pyqt, why not relace the GUI of python turtle use pyqt?

waveform80 commented 6 years ago

@ntoll i have read all the souce of python turtle and in it it say you can use any GUI framework replace tkinter, thus mu is writed using pyqt, why not relace the GUI of python turtle use pyqt?

Because, whilst it is possible, it's a lot more work than it looks. The GUI interface stuff doesn't look terribly long in turtle.py but it's relying on a fair bit of inherited functionality (and some wonderfully dodgy hacks: take a look at __forwardmethods). Porting it to another framework basically means writing a compatibility layer for Tk's Canvas and Frame classes (and the main loop) in Qt. It can be done, but this isn't a quick "do it in a day" job.

ntoll commented 6 years ago

Heh... anything is possible (given enough time and money). :-)

So, while @yonghuming's suggestion is actually VERY interesting, for the purposes of Mu at this moment in time the cost of time / effort isn't viable. However, I'd love to be able to embed turtle into a Qt application. Perhaps this is a new module in the waiting (so others can re-use the effort).

tjguk commented 6 years ago

It occurred to me that someone must have done this before:

https://github.com/Transpyler/qturtle

(Haven't tried it or anything...)

waveform80 commented 6 years ago

Afraid I already had a grep through that earlier (same thought occurred :) - it's a re-implementation of turtle (doesn't use the built-in library at all, so not quite the same thing as is suggested above). It might be reasonably complete, but I haven't dug that far into it.

ntoll commented 6 years ago

Folks, this has all just landed in master (getting tkinter automatically into the build was a royal PIA).

The following two builds have been tested on both Windows 7 32bit and Windows 10 64bit (they install, and I can get GUIZero working, turtle drawing things, requests grabbing URLs, Pillow importing images and a simple tkinter application with no errors). GPIOZero and piGPIO are also bundled but I need to add a feature where envars can be set within Mu to make these work (as discussed with @bennuttall and coming early next week - the important thing is that they're already packaged pending this update).

https://s3-eu-west-2.amazonaws.com/mu-builds/windows/mu_2018-03-16_18_52_master_9b119d5_64bit.exe

https://s3-eu-west-2.amazonaws.com/mu-builds/windows/mu_2018-03-16_18_56_master_9b119d5_32bit.exe

mu-guizero

martinohanlon commented 6 years ago

Legend.... I will have a go later.

MarcScott commented 6 years ago

That's awesome @ntoll Quick question - do you need to set the env variables to use remote pins. Thought you could state the IP directly in the file.

from gpiozero.pins.pigpio import PiGPIOFactory
factory = PiGPIOFactory(host="192.168.1.79")
ntoll commented 6 years ago

@MarcScott I'm working on a way to configure envars in Python 3 mode and it should land later today or tomorrow.

waveform80 commented 6 years ago

Quick question - do you need to set the env variables to use remote pins. Thought you could state the IP directly in the file.

No, you don't have to set the env variables. As you note it's possible to specify hosts manually (which also means one script can control multiple Pis by constructing multiple factories); the env vars simply set the default. If the goal is to keep things as simple as possible for the user's script then it's wise to go the env vars route - but they can always be overridden within the script. For full details see environment variables and onwards in the GPIO Zero docs.

bennuttall commented 6 years ago

Exactly. The end goal is that someone on a PC (with Pi Zero attached) can type code that would work on a Pi, without having to change it, e.g.

from gpiozero import LED

led = LED(2)

led.blink()

rather than:

from gpiozero import LED
from gpiozero.pins.pigpio import PiGPIOFactory

pizero = PiGPIOFactory('fe80::1%usb0')

led = LED(2, pin_factory=pizero)

led.blink()

Also they wouldn't have to know what IP address to use - this would be the default.