RFT: croagh framework - Githubissues

dnschneid commented 11 years ago

I've created a branch called "croagh" that adds a framework and abstractions that should make it reasonably straightforward to add new distributions with minimal to no duplicated code.

But first, we have to make sure the new installer framework doesn't screw up anything, so please test croagh with everything you can. If it installs and upgrades chroots without any errors, it should be fine (since nothing after that has fundamentally changed).

Once that's confirmed OK and merged, we can look into bringing @drinkcat's Arch bootstrap code and compatibility into master. I'll have to start thinking of a new explanation for the project's name...

drinkcat commented 11 years ago

Nice, that's good news! Constantly rebasing my master branch is not the most fun thing ever (I accidentally broke it today...), and your architecture looks simpler and more scalable than my brute-force copy-paste ,-)

For the project name, maybe U for universal, and, well, the "T" is there to make it sound yummy ,-)

I'll have a look at the code, and let you know if I see potential issues with Arch integration.

tedm commented 11 years ago

Thanks. Am testing croagh now, and it's working just like crouton. Installed cli-extra -r raring, used it, deleted it, entered existing chroots, used, exited. All is the same. Anything specific to test?

tedm commented 11 years ago

question - would it be possible to reconsider the naming conventions of targets and releases? I understand that you can only have one DE per release, and the DE name is the default target name, and you can always check the real chroot name outside the chroot in /usr/local/chroots for enter-chroot, delete-crhoot, etc., but if we can now have more than one chroot with the same DE, the startupscript name should possibly reflect the chroot or target that it will bring up. Currently it can change on you, and require you to remember what target name you used during install or -n parameter passed, and not be able to look up easily.

For example, startxfce4 may have started your xfce target for a long time, then you create another chroot higher in the alphabet, and now your startxfce4 needs you to remember parameters to load, otherwise the first in the alphabet comes up.

Perhaps it is as simple as keeping a list of targets installed somewhere, but preferably a unique startup script name for each target.

Also, after an install, the 'enter-chroot' should possibly say 'enter-chroot-[chroot filename] otherwise one would always enter an arbitrary chroot.

While in a chroot, uname -a doesn't say much, just arm v7, kernel 3.4, etc. Is there a way or command within the chroot to know which chroot target, and release you are in?

dnschneid commented 11 years ago

Pretty much just need to test that the install completes with various targets and options; the rest should be just as (non-)functional as before.

We can address naming conventions in another bug if you'd like to file one.

tedm commented 11 years ago

OK, installing more targets, now e17 / raring. This error message, I've seen in crouton, so I don't think it's new, but just fyi:

... Fetched 22.5 MB in 1min 17s (290 kB/s)
Extracting templates from packages: 100% Can not write log, openpty() failed (/dev/pts not mounted? ...

dnschneid commented 11 years ago

Yeah, that's nothing to worry about. The debootstrap second stage temporarily clobbers a lot of the bindmounts inside of the chroot.

tedm commented 11 years ago

opened a new crosh / shell window and installed sudo sh -e ~/Downloads/croagh -t e17 -r raring Install went OK, entered username / pw. tried to start with sudo starte17

It's possible I had a window that had enter-chroot into the alarm chroot, but I'm sure that xfce (alarm) wasn't running.

After the usual unmount-chroots didn't work, I power cycled, and alarm still won't unmount:

nor will the sigkill loop abort with ctrl-c.:

chronos@localhost / $ sudo starte17

Initializing built-in extension Generic Event Extension Initializing built-in extension SHAPE Initializing built-in extension MIT-SHM Initializing built-in extension XInputExtension Initializing built-in extension XTEST Initializing built-in extension BIG-REQUESTS Initializing built-in extension SYNC Initializing built-in extension XKEYBOARD Initializing built-in extension XC-MISC Initializing built-in extension SECURITY Initializing built-in extension XINERAMA Initializing built-in extension XFIXES Initializing built-in extension RENDER Initializing built-in extension RANDR Initializing built-in extension COMPOSITE Initializing built-in extension DAMAGE Initializing built-in extension MIT-SCREEN-SAVER Initializing built-in extension DOUBLE-BUFFER Initializing built-in extension RECORD Initializing built-in extension DPMS Initializing built-in extension X-Resource Initializing built-in extension XVideo Initializing built-in extension XVideo-MotionCompensation [dix] Could not init font path element /usr/share/fonts/OTF/, removing from list! [dix] Could not init font path element /usr/share/fonts/Type1/, removing from list! ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed /usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1 /usr/bin/xinit: Unable to run program "/usr/bin/enlightenment_start": No such file or directory Specify a program on the command line or make sure that /usr/bin is in your path.

/usr/bin/xinit: connection to X server lost

waiting for X server to shut down ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed crouton-clip not listening on port 30001. ratpoison:manage.c:236: error: XGetWMName failed Cannot figure previous or current window (,aura). Cannot parse window type (*#2#Unnamed).

Unmounting /usr/local/chroots/alarm... Sending SIGTERM to processes under /usr/local/chroots/alarm... chronos@localhost / $ sudo unmount-chroot alarm Unmounting /usr/local/chroots/alarm... chronos@localhost / $ sudo starte17

Initializing built-in extension Generic Event Extension Initializing built-in extension SHAPE Initializing built-in extension MIT-SHM Initializing built-in extension XInputExtension Initializing built-in extension XTEST Initializing built-in extension BIG-REQUESTS Initializing built-in extension SYNC Initializing built-in extension XKEYBOARD Initializing built-in extension XC-MISC Initializing built-in extension SECURITY Initializing built-in extension XINERAMA Initializing built-in extension XFIXES Initializing built-in extension RENDER Initializing built-in extension RANDR Initializing built-in extension COMPOSITE Initializing built-in extension DAMAGE Initializing built-in extension MIT-SCREEN-SAVER Initializing built-in extension DOUBLE-BUFFER Initializing built-in extension RECORD Initializing built-in extension DPMS Initializing built-in extension X-Resource Initializing built-in extension XVideo Initializing built-in extension XVideo-MotionCompensation [dix] Could not init font path element /usr/share/fonts/OTF/, removing from list! [dix] Could not init font path element /usr/share/fonts/Type1/, removing from list! ratpoison:manage.c:236: error: XGetWMName failed /usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1 /usr/bin/xinit: Unable to run program "/usr/bin/enlightenment_start": No such file or directory Specify a program on the command line or make sure that /usr/bin is in your path.

/usr/bin/xinit: connection to X server lost

waiting for X server to shut down ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed Cannot parse window type (#1#Unnamed). Cannot parse window type (#1#Unnamed). ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed Cannot figure previous or current window (,aura). ratpoison:manage.c:236: error: XGetWMName failed

Unmounting /usr/local/chroots/alarm... Sending SIGTERM to processes under /usr/local/chroots/alarm... chronos@localhost / $ sudo unmount-chroot -n alarm -afkn Illegal option -n unmount-chroot [options] name [...]

Unmounts one or more chroots, optionally killing any processes still running inside them.

By default, it will run in interactive mode where it will ask to kill any remaining processes if unable to unmount the chroot within 5 seconds.

Options: -a Unmount all chroots in the CHROOTS directory. -c CHROOTS Directory the chroots are in. Default: /usr/local/chroots -f Forces a chroot to unmount, potentially breaking or killing other instances of the same chroot. -k KILL Send the processes SIGKILL instead of SIGTERM. -p Print to STDOUT the processes stopping a chroot from unmounting. -t TRIES Number of seconds to try before signalling the processes. Use -t inf to be exceedingly patient. Default: 5 -y Signal any remaining processes without confirmation. Automatically escalates from SIGTERM to SIGKILL. chronos@localhost / $ sudo unmount-chroot -afk Unmounting /usr/local/chroots/alarm... Unmounting /usr/local/chroots/precise... Unmounting /usr/local/chroots/raring... chronos@localhost / $ sudo starte17

Initializing built-in extension Generic Event Extension Initializing built-in extension SHAPE Initializing built-in extension MIT-SHM Initializing built-in extension XInputExtension Initializing built-in extension XTEST Initializing built-in extension BIG-REQUESTS Initializing built-in extension SYNC Initializing built-in extension XKEYBOARD Initializing built-in extension XC-MISC Initializing built-in extension SECURITY Initializing built-in extension XINERAMA Initializing built-in extension XFIXES Initializing built-in extension RENDER Initializing built-in extension RANDR Initializing built-in extension COMPOSITE Initializing built-in extension DAMAGE Initializing built-in extension MIT-SCREEN-SAVER Initializing built-in extension DOUBLE-BUFFER Initializing built-in extension RECORD Initializing built-in extension DPMS Initializing built-in extension X-Resource Initializing built-in extension XVideo Initializing built-in extension XVideo-MotionCompensation [dix] Could not init font path element /usr/share/fonts/OTF/, removing from list! [dix] Could not init font path element /usr/share/fonts/Type1/, removing from list! ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed /usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1 /usr/bin/xinit: Unable to run program "/usr/bin/enlightenment_start": No such file or directory Specify a program on the command line or make sure that /usr/bin is in your path.

/usr/bin/xinit: connection to X server lost

waiting for X server to shut down ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed Cannot figure previous or current window (,aura). Cannot figure previous or current window (,aura). ratpoison:manage.c:236: error: XGetWMName failed ratpoison:manage.c:236: error: XGetWMName failed crouton-clip not listening on port 30001.

Unmounting /usr/local/chroots/alarm... Sending SIGTERM to processes under /usr/local/chroots/alarm... /usr/local/bin/xinit: line 60: 10530 Terminated sleep 1 /usr/local/bin/xinit: line 60: 10531 Terminated sleep 1 /usr/local/bin/xinit: line 60: 10532 Terminated sleep 1 Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... ^CUnmounting /usr/local/chroots/alarm... ^CUnmounting /usr/local/chroots/alarm... ^C^C^C^C^C^C^C^C^C^C^C^CUnmounting /usr/local/chroots/alarm... ^CUnmounting /usr/local/chroots/alarm... ^C^C^CUnmounting /usr/local/chroots/alarm... ^CUnmounting /usr/local/chroots/alarm... ^CUnmounting /usr/local/chroots/alarm... ^C^C^C^C^C^C^C^C^C^C^C^CUnmounting /usr/local/chroots/alarm... Sending SIGTERM to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm... Sending SIGKILL to processes under /usr/local/chroots/alarm...

tedm commented 11 years ago

from crosh/shell sudo su top, I was able to kill the sh with -9, then as root, apparently the "p" which I thought displayed hanging processes unmounted alarm finally, but starte17 showed alarm still mounted. Will have to delete-chroot alarm and power cycle...

localhost bin # ls delete-chroot edit-chroot enter-chroot mount-chroot starte17 startxfce4 unmount-chroot localhost bin # ./unmount-chroot -afk Unmounting /usr/local/chroots/alarm... Failed to unmount /usr/local/chroots/alarm. Kill processes? [a/k/y/p/N] a Sending SIGKILL to processes under /usr/local/chroots/alarm... Unmounting /usr/local/chroots/precise... Unmounting /usr/local/chroots/raring... localhost bin # ./unmount-chroot -afkp Unmounting /usr/local/chroots/alarm... Unmounting /usr/local/chroots/precise... Unmounting /usr/local/chroots/raring... localhost bin #

tedm commented 11 years ago

After a lot of failed unmounts, I was finally able to delete-chroot alarm, power cycle, then try starte17 again, but now it is having trouble unmounting precise (xfce) which definitely wasn't running or entered since last power cycle.

The dnsmasq, and whoopsie messages indicate that it may be trying to start xfce (precise) as those messages do come up when xfce (precise starts), even though this e17 target was using -r raring and chroot name is raring.

Will try to delete raring next.

crosh> shell chronos@localhost / $ sudo starte17 Unknown username "dnsmasq" in message bus configuration file Unknown username "whoopsie" in message bus configuration file

[dix] Could not init font path element /usr/share/fonts/X11/misc, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list! [dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list! /usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1 /bin/sh: 0: Can't open /usr/bin/enlightenment_start /usr/bin/xinit: connection to X server lost

waiting for X server to shut down X Error of failed request: BadWindow (invalid Window parameter) Major opcode of failed request: 20 (X_GetProperty) Resource id in failed request: 0x1000001 Serial number of failed request: 13 Current serial number in output stream: 13 X Error of failed request: BadWindow (invalid Window parameter) Major opcode of failed request: 20 (X_GetProperty) Resource id in failed request: 0xa00001 Serial number of failed request: 13 Current serial number in output stream: 13

Unmounting /usr/local/chroots/precise... Sending SIGTERM to processes under /usr/local/chroots/precise... Sending SIGKILL to processes under /usr/local/chroots/precise... Sending SIGKILL to processes under /usr/local/chroots/precise... Sending SIGKILL to processes under /usr/local/chroots/precise... Sending SIGKILL to processes under /usr/local/chroots/precise... Sending SIGKILL to processes under /usr/local/chroots/precise...

tedm commented 11 years ago

with alarm and raring (e17) deleted, the old precise xfce starts up, and exits well. Will try kde or another chroot with croagh later, but I think even though precise (xfce) is behaving well, there may be something "sticky" in my setup...

Unmounting /usr/local/chroots/precise... Sending SIGTERM to processes under /usr/local/chroots/precise... chronos@localhost / $

dnschneid commented 11 years ago

Cool. Anyone else get the chance to test this new installer?

tedm commented 11 years ago

after kde / raring finished, entered user / pw, sudo startkde, I got an all white screen, with black square block in upper left. system was frozen. power-cycled, something thinks precise is still mounted. Successfully entered xfce precise, exited, appears to unmount, startkde gave white screen with black rectangle. mouse is active, but can't hot-key back to chromeos.

crosh> shell chronos@localhost / $ sudo startkde Unknown username "dnsmasq" in message bus configuration file Unknown username "whoopsie" in message bus configuration file

[dix] Could not init font path element /usr/share/fonts/X11/misc, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list! [dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list! [dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list! /usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1 /bin/sh: 0: Can't open /usr/bin/startkde /usr/bin/xinit: connection to X server lost

waiting for X server to shut down Unmounting /usr/local/chroots/precise... Sending SIGTERM to processes under /usr/local/chroots/precise... chronos@localhost / $

dnschneid commented 11 years ago

I'm pretty sure you already know that if you don't specify -n, you're going to be entering the first chroot alphabetically, regardless of what is installed in it. Nothing strange is happening here.

tedm commented 11 years ago

argh, croargh, forgot. startkde -n raring does bring up kde / raring, still exits with sigkill loop that has to be terminated from another shell.

Also, noted that double light tap to click wasn't working in kde/raring, had to hard press the touchpad.

When a user deletes a chroot (e.g. delete-chroot raring) associated with a DE target (e.g. e17), the starte17 wrapper stays around. It's confusing when that chroot is gone...

I'm not sure the tap to click, and sigkill loop are croagh related, but I can delete the kde / raring and re-install with crouton if needed. let me know. Thanks for reminding me about the -n

drinkcat commented 11 years ago

Well, quick tests show that Ubuntu still works. Debian does not work, because the mirror is Ubuntu's by default.

In installer/main.sh, the following variables are distribution specific:

MIRROR (MIRROR86 and MIRRORARM should be removed)
ARCH (e.g., Arch linux uses x86_64 instead of amd64, armv7h for ARM, etc.)

Also, I'm not sure if the files listing release names is really a good idea ({debian/ubuntu}/releases), wouldn't it be easier to add a new parameter -d DISTRO (-d is already taken, so, something else), and then have a default release per distribution? The current way also means you would need to update the list everytime there is a new release (not often, but, well...), and could lead to conflicts if 2 distributions happen to choose the same name (not likely, agree). Also, Arch and Gentoo are rolling release so it does not make much sense for them...

Maybe we could have a "pre-install" script, distribution-specific, that we could source before doing the bulk of the work, and that would set those variables: MIRROR, ARCH, RELEASE (if relevant for the distro, and if not set already), NAME (if not set)?

dnschneid commented 11 years ago

Good catch on the variables.

I figured debian wouldn't work as is (it never in crouton); in fact currently there's no code in "core" to set up the software sources for debian.

I feel that specifying the distro separately is redundant; I don't mind updating the release list once every 6 months or so since it makes things way simpler for the user. This also works nicely with the default-name-is-the-release, and provides backwards compatibility with existing chroots. For Arch and Gentoo, a "release" is more like a derivative, so releases named "arch", "alarm", "gentoo", etc would make perfect sense in this context.

Help also needs to be extended to somehow list the supported releases. I'll likely do something akin to pre-install, but it needs to be somewhat better-defined. I'll keep thinking about it.

dnschneid commented 11 years ago

Okay. I've added a "defaults" file to the distro, which needs to set ARCH and MIRROR at a minimum (if not specified).

Also debian kinda works now maybe...

DennisLfromGA commented 11 years ago

... I'm installing a new croagh-quantal chroot now with targets: chrome-beta cli-extra gtk-extra kde unity xfce I'll let y'all know how it goes...

btw - is version (0-20130601151326~croagh:6560aba4) per the above link (https://docs.google.com/uc?export=download&id=0B1QUtoJwc9L3SmxyQmcxRUdNbFk) the latest or should we be checking another link for updates?

dnschneid commented 11 years ago

That link will always be the latest until croagh gets merged in, after which it will be a dead link.

DennisLfromGA commented 11 years ago

Thanx, I'll assume the 'crouton-desktops' link will be treated the same way...

dnschneid commented 11 years ago

Yep. No developments there, unfortunately.

tedm commented 11 years ago

installed croagh with sh -e ~/Downloads/croagh -t kde, lxde, touch -r quantal

took 1-2 hours, but completed. not sure how to test touch on the samsung arm? kde and lxde work as with crouton - e.g. tap to click doesn't work, but the desktops come up when specified with start[kde lxde] -n quantal

On Sat, Jun 1, 2013 at 8:44 PM, David Schneider notifications@github.comwrote:

Yep. No developments there, unfortunately.

— Reply to this email directly or view it on GitHubhttps://github.com/dnschneid/crouton/issues/176#issuecomment-18801484 .

DennisLfromGA commented 11 years ago

...follow-up.... croagh install took a while but went fine and targets kde & xfce did well, unity was kind of bare and, of course didn't log out but all went well, even did a croagh update and no problems. I'm sure there's differences underneath but it behaved just like crouton to me so I think that's a good thing.

dnschneid commented 11 years ago

Yep, it's really just the installer that has been reworked; once running it should be the exact same.

tedm commented 11 years ago

Dennis, I realize you're on the x86 platform, but does the kde DE have tap to click working OK? Does anyone else on the ARM platform have tap to click not working on kde (but working with xfce) ?

DennisLfromGA commented 11 years ago

@tedm - I just now checked all three of my DE's ( kdm, unity, xfce4) on croach -r quantal and the touchpad seemed to work well including tap-to-click. Granted, I'm not proficient with the touchpad since I use an external mouse (Logitech M325) normally but I managed to kludge along pretty well.

My credentials are: 
Acer C710-2847 w/ 4GB RAM
croagh: version 0-20130601151326~croagh:6560aba4
Version 28.0.1500.20 beta
Platform 4100.17.0 (Official Build) beta-channel parrot
Firmware Google_Parrot.2685.37.0

tedm commented 11 years ago

Thanks Dennis, since your x86 version of quantal with croagh and kde is working, if others incl. ARM users are getting tap to click, then perhaps my system is acting up. I installed croagh and kde quantal last night, along with targets, lxde, and touch. Am running the dev channel, and can look up any specifics if any developers need that.

Exiting the kde / quantal chroot also gives some error messages and a usual sigkill loop from last night's croagh, but might be related to whatever is preventing tap to click, requiring the pressure double clicks on the trackpad:

Application '/usr/bin/nepomukservicestub nepomukbackupsync' exited normally...
[/usr/bin/nepomukservicestub] virtual
Soprano::ODBC::Connection::~Connection() QThread(0x15255f8)
[/usr/bin/nepomukservicestub] Shutting down virtuoso instance 16261
startkde: Shutting down...
klauncher: Exiting on signal 1
NepomukServer(16129)/kdecore (KConfigSkeleton) KCoreConfigSkeleton::writeConfig:
startkde: Running shutdown scripts...
Running Mixer_Backend destructor
Running Mixer_Backend destructor
Running Mixer_Backend destructor
Running Mixer_Backend destructor
startkde: Done.
/usr/bin/xinit: connection to X server lost

waiting for X server to shut down XIO:  fatal IO error 0 (Success) on
X server ":1"
      after 53 requests (31 known processed) with 0 events remaining.
Control process died, committing suicide!

Unmounting /usr/local/chroots/quantal...
Application 'akonadiserver' exited normally...
D-Bus session bus went down - quitting
Sending SIGTERM to processes under /usr/local/chroots/quantal...
Sending SIGKILL to processes under /usr/local/chroots/quantal...
Sending SIGKILL to processes under /usr/local/chroots/quantal...
Sending SIGKILL to processes unde

dnschneid commented 11 years ago

@drinkcat any other integration issues you can foresee?

drinkcat commented 11 years ago

For now, 2 things:

All targets are assumed to be supported on all distributions/releases, and we are explicitely blacklisting things that do not work (like in targets/unity). Could we switch to a whitelist system, where each target states which distributions/releases are supported? This would make: 1. Adding more distributions easier and faster (you could start by porting core, and slowly adding other targets). 2. Adding a new target easier (no need to test it on 3-4 distributions immediately). We could provide a "force" option if the user really wants to try a target.
install_pkg and PKGEXT look very debian/ubuntu specific to me. install_pkg is found in 3 targets. For 2 of them (touch, xephyr) they are enclosed in an if block that is ubuntu-specific. For the 3rd one (chrome-common), I doubt the code is generic enough to work on something else than ubuntu/debian: it won't on arch/gentoo, and the xdg-utils thing may not be necessary on SuSE/Fedora. (and, wouldn't it be better to add Google's repository and install the package as normal?)

Those are fairly minor issues, I guess I'll see the real ones (if any...) when I start porting my modifications.

drinkcat commented 11 years ago

Starting porting my code, hit one more:

grep -q "=$RELEASE\$" "$CHROOT/etc/lsb-release"; in main.sh should be distribution specific (Arch does not have that file, and in that case we should use /etc/os-release). Maybe the defaults script can define a function that performs that test?

drinkcat commented 11 years ago

And also note the commits I pushed to you in #192 (2 so far).

drinkcat commented 11 years ago

A last one for tonight (not really about the framework itself, though):

enter-chroot complains: chown: invalid user: 'messagebus:messagebus'. On Arch, the username is dbus:dbus. There is a way to autodetect (one way is via the pidfile, then /proc/#/status, maybe there is a better way), I'll look at it tomorrow.

dnschneid commented 11 years ago

Thanks for the fixes and notes.

I would like for all distributions to support all targets (within the realm of possibility), and I don't think I'll be merging a new distro into master until it hits that state. There may be targets that are simply not possible to support yet (i.e. Unity on Debian), so it's probably worth adding blacklist functionality (including architecture) to the target. I will not add a whitelist.
install_pkg + PKGEXT are designed to work with manually installed rpm packages as well (and equivalent for other platforms). In fact, with the ARCH set correctly, the chrome targets should just work as written on rpm-based distros. The reason install_pkg used inside of Ubuntu-specific blocks is simply because the version available in the repos is insufficient for whatever reason.
The Chrome package installs and manages its own repo for auto-updates.
If xdg-utils isn't necessary on a distro, you can filter it out by changing the package descriptor to something like xdg-utils,fedora=,.
The lsb-release thing is definitely broken for alternate distros. At this point it would probably be better to just write the release to "$CHROOT/etc/crouton/release" and just check that (falling back on /etc/lsb-release for old chroots).
That autodetection method sounds circular. The most correct way would be to parse the relevant init script for the user name, but that's complicated. The easy way is to try one and then the other if the first fails :)

drinkcat commented 11 years ago

I would like for all distributions to support all targets (within the realm of possibility).

Ok.

install_pkg + PKGEXT are designed to work with manually installed rpm packages as well (and equivalent for other platforms). In fact, with the ARCH set correctly, the chrome targets should just work as written on rpm-based distros. The reason install_pkg used inside of Ubuntu-specific blocks is simply because the version available in the repos is insufficient for whatever reason.

Yes, I see the idea, my point is that I find it odd to have a generic facility that: 1. Will not apply for some distributions (no install_pkg will be called in Arch/Gentoo). 2. Is effectively used only once in its generic form (and it will be put in an if block once I port Arch).

The Chrome package installs and manages its own repo for auto-updates.

Yes, I was just wondering if it would be possible to "bootstrap" it by adding the repository manually (and then let Chrome manage it). That would make the code neater, but I have no idea if that's practical.

The lsb-release thing is definitely broken for alternate distros. At this point it would probably be better to just write the release to "$CHROOT/etc/crouton/release" and just check that (falling back on /etc/lsb-release for old chroots).

Unless the user does something crazy like upgrading the release...

That autodetection method sounds circular.

Oh yeah. Eyes got crossed and I though the daemon was started before the chown (which would make no sense).

The most correct way would be to parse the relevant init script for the user name, but that's complicated. The easy way is to try one and then the other if the first fails :)

/etc/dbus-1/system.conf is not too hard to parse (it would get messy if the user decides to extend it with files in /etc/dbus-1/system.d/, though). I'll give it a go later.

One more issue:

Can we drop the dependency on chromium? I never had it installed on Arch. Most WM/DE install chromium. But chromium takes quite a bit of space, and is more of a personal preference. We could still provide it as a target, and update the "easy" instructions to -t xfce,chromium so users don't get lost.

dnschneid commented 11 years ago

Yes, I see the idea, my point is that I find it odd to have a generic facility that: 1. Will not apply for some distributions (no install_pkg will be called in Arch/Gentoo). 2. Is effectively used only once in its generic form (and it will be put in an if block once I port Arch).

It is indeed a bit odd, although it is nice to shift the implementation elsewhere so that targets are easier to create. Sounds like Arch/Gentoo will simply implement it as an error message. But really, it's no different from having a custom function defined for a distro--this way at least it has a predefined syntax. Maybe change the wording in the comments to say that install_pkg can be implemented as an error message if the distro doesn't support downloadable packages?

Yes, I was just wondering if it would be possible to "bootstrap" it by adding the repository manually (and then let Chrome manage it). That would make the code neater, but I have no idea if that's practical.

I don't really see how that would be any neater, especially since you may end up with two instances of the Chrome repo in your sources once the package is installed, and that install method is really not aligned with the way Chrome is packaged.

Unless the user does something crazy like upgrading the release...

That's what -uu is for.

/etc/dbus-1/system.conf is not too hard to parse (it would get messy if the user decides to extend it with files in /etc/dbus-1/system.d/, though). I'll give it a go later.

is that...XML? Yikes. Hopefully you can come up with some elegant way of parsing it.

Can we drop the dependency on chromium? I never had it installed on Arch. Most WM/DE install chromium. But chromium takes quite a bit of space, and is more of a personal preference. We could still provide it as a target, and update the "easy" instructions to -t xfce,chromium so users don't get lost.

I was indeed really hesitant to add that when I did. I'd be okay removing the dependency once we have a method for opening new tabs on the host side, in which case that can be the default URL handler. Until then, I'd rather not leave the people who blindly install DEs without a built-in web browser (how else would they search for "how to switch back to Chromium OS"?), and Chromium makes the most sense for default browser in the repo since it's available on all architectures and, well, you ARE on a Chromebook (or box). If I recall correctly, with dependencies it's smaller than Firefox (and Chrome, since unlike Chrome, Chromium builds against shared libraries). Another option would be Midori, but again, Chromebook.

drinkcat commented 11 years ago

It is indeed a bit odd, although it is nice to shift the implementation elsewhere so that targets are easier to create. Sounds like Arch/Gentoo will simply implement it as an error message. But really, it's no different from having a custom function defined for a distro--this way at least it has a predefined syntax. Maybe change the wording in the comments to say that install_pkg can be implemented as an error message if the distro doesn't support downloadable packages?

Yes, I thought that it would be more appropriate as a custom function for the distro. But, it's up to you ,-)

If you update some of the comments, could you try to update the description of install to describe a bit more in details what the purpose of the 2nd set is? (the one after --) I had to scratch my head quite a bit to understand what it could be about, and even go back to the code in crouton/master. And I'm still not sure I fully got it, probably because I'm not used to how apt-get works. (something to do with preventing apt-get from pulling some recommended dependencies? But then how different is it from --minimal?)

Unless the user does something crazy like upgrading the release...

That's what -uu is for.

If we write to $CHROOT/etc/crouton/release during install, and check that file during enter-chroot, and the user upgrades the release from inside the chroot, we could get into trouble, as -u would think it's still the old release (/etc/lsb-release would be the new one, and $CHROOT/etc/crouton/release would still be the old one).

/etc/dbus-1/system.conf is not too hard to parse (it would get messy if the user decides to extend it with files in /etc/dbus-1/system.d/, though). I'll give it a go later.
is that...XML? Yikes. Hopefully you can come up with some elegant way of parsing it.

I was thinking about something quick and dirty. Maybe I need to rethink now ,-)

Can we drop the dependency on chromium? I never had it installed on Arch. Most WM/DE install chromium. But chromium takes quite a bit of space, and is more of a personal preference. We could still provide it as a target, and update the "easy" instructions to -t xfce,chromium so users don't get lost.
I was indeed really hesitant to add that when I did. I'd be okay removing the dependency once we have a method for opening new tabs on the host side, in which case that can be the default URL handler. Until then, I'd rather not leave the people who blindly install DEs without a built-in web browser (how else would they search for "how to switch back to Chromium OS"?), and Chromium makes the most sense for default browser in the repo since it's available on all architectures and, well, you ARE on a Chromebook (or box). If I recall correctly, with dependencies it's smaller than Firefox (and Chrome, since unlike Chrome, Chromium builds against shared libraries). Another option would be Midori, but again, Chromebook.

I see your point. I guess I need to figure out the URL handler thing then ,-) It can easily be integrated in the clipboard app, but that's not the solution we want, if we can avoid it... I tried searching for a DBUS interface, found other interesting stuff (volume, screen locking: I'll push those to you soon), but nothing about URL handling or clipboard...

dnschneid commented 11 years ago

Yes, I thought that it would be more appropriate as a custom function for the distro. But, it's up to you ,-)

I think I'll leave it as-is for now. Once a few more distros are added we can see how useful it is to define it consistently and relegate it to a distro extension if it doesn't find any other usage.

If you update some of the comments, could you try to update the description of install to describe a bit more in details what the purpose of the 2nd set is? (the one after --) I had to scratch my head quite a bit to understand what it could be about, and even go back to the code in crouton/master. And I'm still not sure I fully got it, probably because I'm not used to how apt-get works. (something to do with preventing apt-get from pulling some recommended dependencies? But then how different is it from --minimal?)

Yeah, it's like --minimal, except it only prevents the installation of specific packages, allowing the other recommended packages to be auto-installed. For example, Firefox is recommended for Unity or something. We don't want to not include the other recommended packages, but we don't want Firefox installed by default either. So the second set of packages is a list of things to avoid installing automatically. In the case of apt-get, this is implemented by checking if the package is installed, and if it is not, adds a - to the end of it in the install command to mark that it should be uninstalled (which effectively means not automatically-installed). This...needs a better explanation in the comments. Anyway, I imagine Arch has a similar way of selectively disallowing suggested dependencies?

If we write to $CHROOT/etc/crouton/release during install, and check that file during enter-chroot, and the user upgrades the release from inside the chroot, we could get into trouble, as -u would think it's still the old release (/etc/lsb-release would be the new one, and $CHROOT/etc/crouton/release would still be the old one).

Good point. How about this: each distro provide a release script that just prints out the installed release when run, and that gets installed as the chroot's /usr/local/bin/croutonrelease. That way the upgrader can run it to sanity-check, and croutonversion can run it to print out more information about the chroot.

I was thinking about something quick and dirty. Maybe I need to rethink now ,-)

There's nothing quick-and-dirty about XML, unfortunately. My vote is to brute-force try both :)

drinkcat commented 11 years ago

Yeah, it's like --minimal, except it only prevents the installation of specific packages, allowing the other recommended packages to be auto-installed. For example, Firefox is recommended for Unity or something. We don't want to not include the other recommended packages, but we don't want Firefox installed by default either. So the second set of packages is a list of things to avoid installing automatically. In the case of apt-get, this is implemented by checking if the package is installed, and if it is not, adds a - to the end of it in the install command to mark that it should be uninstalled (which effectively means not automatically-installed). This...needs a better explanation in the comments.

Ok, thanks for the clarification.

Anyway, I imagine Arch has a similar way of selectively disallowing suggested dependencies?

Yes. It simply doesn't install anything that is "suggested" (it does suggest other packages from time to time, but only prints them out). Only strict dependencies are pulled in. (same with Gentoo BTW)

So basically I will ignore the second set in Arch.

Good point. How about this: each distro provide a release script that just prints out the installed release when run, and that gets installed as the chroot's /usr/local/bin/croutonrelease. That way the upgrader can run it to sanity-check, and croutonversion can run it to print out more information about the chroot.

I guess the script will need to run in the chroot. Does it mean that the test is moved in prepare.sh? Or would we enter-chroot twice? Also, we would be installing something in the chroot that is only useful for the installer, and some future modification of that script may force users to use -uu (say if we want to check that the architecture is correct, once people start playing with backups...). Not 100% convinced this is better, simpler or more flexible than defining a function in distro/defaults, or having that script in the installer directory (e.g. distro/getrelease.sh)

dnschneid commented 11 years ago

I guess the script will need to run in the chroot. Does it mean that the test is moved in prepare.sh? Or would we enter-chroot twice?

It can be run with an explicit call to chroot, similar to how the dbus daemon is launched.

Also, we would be installing something in the chroot that is only useful for the installer, and some future modification of that script may force users to use -uu (say if we want to check that the architecture is correct, once people start playing with backups...).

It's also useful for croutonversion (which is run inside the chroot), which makes bug reporting easier.

Not 100% convinced this is better, simpler or more flexible than defining a function in distro/defaults, or having that script in the installer directory (e.g. distro/getrelease.sh)

I'd like to get to the point where you don't need to specify -r for upgrades. If we install croutonrelease into the chroot, we just need to run that to grab the release. If we add a script in the distro directory, we can iterate through each one until we hit one that successfully returns a release.

I think you're right, though, chrooting to check the release seems kinda strange. The "iterate through" technique seems simple enough for upgrades/sanity checking, and we can start writing the release to /etc/crouton/release (and not reading it back) for the sake of croutonversion (understanding that it may be out-of-date if the user manually upgraded).

So then the solution is:

Add installer/*/getrelease.sh, which prints out the release name for a given $CHROOT ($1), or exits with a non-zero return code if the chroot doesn't belong to that distro.
Run it with sh -e when checking the release if release is specified when upgrading.
Iterate and run all of them when upgrading if release is not specified.

drinkcat commented 11 years ago

So then the solution is:

Add installer/*/getrelease.sh, which prints out the release name for a given $CHROOT ($1), or exits with a non-zero return code if the chroot doesn't belong to that distro.
Run it with sh -e when checking the release if release is specified when upgrading.
Iterate and run all of them when upgrading if release is not specified.

Sounds perfect to me.

mmirg commented 11 years ago

I tried an encrypted install with xfce and it seemed to work well. It seems like drinkcat's xkb modifications haven't been merged into croagh yet? (As an aside, I added a comment to the pull request for the xkb overlay modifications but failed to note that it was closed, should I open a new bug?)

I also took a quick look over at drinkcat's fork with arch support. I didn't see any specific work done yet for gentoo, but I'd be happy to brainstorm/help/test with that since I would probably consider that. I have run funtoo as an ordinary install on my Pixel and have got a make.conf and some config files that may come in handy.

dnschneid commented 11 years ago

xkb is in there; install the keyboard target. Go ahead and open a new bug for brainstorming Gentoo support if you'd like; or better yet, fork the croagh branch and start scripting :)

dnschneid commented 11 years ago

Okay, release detection implemented, bunch of drinkcat's awesome fixes merged, and some functions have been factored out into a library. More testing please!

tedm commented 11 years ago

updated precise xfce with croagh ( sudh sh -e ~/Downloads/croagh -t xfce -u ) and seems to be OK. The text at the end of script is inconsistent, it says 'sudo startxfce4') and just 'enter-chroot', the latter will need a sudo if run as crosh / shell user $ and would be good if it said ...enter-chroot -n precise or ...startxfce4 -n precise as if I actually typed what it said, and had a chroot with name < 'p' that would start or enter instead.

tedm commented 11 years ago

since update with croagh, have been getting this message upon exit of chroot:

Unmounting /usr/local/chroots/precise... Sending SIGTERM to processes under /usr/local/chroots/precise... grep: /proc/11431/environ: No such file or directory <---------- new chronos@localhost / $

not sure if it is from croagh, or just from the update?

dnschneid commented 11 years ago

Actually, that's an issue in crouton as well. I fixed one of the two sources of that but missed the other. Thanks for pointing that out.

dnschneid commented 11 years ago

Please stress the croagh installer as much as you can this week (play with all of the installer parameters). Baring any explosions, I hope to merge it into master at the end of the week.

tedm commented 11 years ago

xfce / quantal appears to install, run, and exit properly with croagh.

mmirg commented 11 years ago

I've got my encrypted xfce on raring updated with the latest chroagh. I'm guessing it's probably something very trivial that I'm overlooking, but something strange seems to be going on with xkb. I had previously made changes to /usr/share/X11/xkb/symbols/chromebook to edit the keyboard shortcuts (notably the latch key, so I can have super back again). This no longer seems to be loading with the most recent chroagh. I tried deleting the chromebook symbols files and xorg continued to load the default shortcuts that drinkcat specified in the chromebook symbols file. Are the overlay symbols being sourced from somewhere else? I glanced at the code and don't see anything that has been changed recently and is noteworthy save for the addtrap/settrap stuff in the keyboard target, so it's highly likely that this is some kind of stupidity on my part.

Re bug #199 (why I'm messing with xkb and not using janky tools that edit keyboard layouts on the fly) I was previously adjusting my keyboard settings with every possible permutation of xmodmap/xdotool/xbindkeys/keylaunch but there are several drawbacks and inconsistencies when using those tools, particularly when switching between multiple input methods and keyboard layouts. To the extent that there is a "right way" to do this, I think it's by adding an overlay like what drinkcat has done and leaving as little as possible to other tools.

dnschneid commented 11 years ago

You probably have to delete the xkb cache to apply changes: rm /var/lib/xkb/* I agree that the overlay is a very clean approach. Kudos to @drinkcat :D

dnschneid / crouton

RFT: croagh framework #176