Build

[john-peterson]

Travis CI

Patch

https://github.com/john-peterson/dolphin-emu/compare/build4

Problem

A commit should be build tested because

• the program build on more than one systems
• the code is different for different systems
• a build fix commit use space in the commit log without meaning because every commit should build on all systems

There's no easily accessible build test because Travis CI compared to

Tools/buildbot-try.sh

• test git diff rather than git log master..HEAD
• require an additional command after git push
• using Tools/buildbot-try.sh require registration
• git commit --amend -C HEAD -a for this build bot is opposed
• it doesn't send a notification when the build result change between fixed and broken
• has less users and it's a higher probability that there's bugs in software with less (compared to more) users

pushing to Google Code origin

• test building a commit with http://buildbot.dolphin-emu.org require write access to origin
• git commit --amend -C HEAD -a; git push -f to this repo is opposed
• it's more efficient to amend (compared to create another commit) a build fix because the build fix should be included in the commit because every commit should build on all systems

.travis.yml should be in origin rather than being pushed by the GitHub user together with the test commit because

• if .travis.yml is missing from a Travis CI enabled repo the repo owner receive a build fail notification when pushing, f.e.

[Failed] john-peterson/dolphin-emu#15 (gui - c6bda9a) [Still Failing] john-peterson/dolphin-emu#16 (gui - 898f977)

No Rakefile found (looking for: rakefile, Rakefile, rakefile.rb, Rakefile.rb)

• it's additional work to push a Travis CI script commit together with the test commit

Solution

Adding a Travis CI script .travis.yml because

• that allow anyone to push to a GitHub branch to build a commit
• without the need to include a Travis CI commit in the test branch
• there are GitHub users that write this program and therefore benefit from this
• it doesn't have a disadvantage because .travis.yml is ignored by code.google.com

Platform

Information

The output from

uname
nproc
lscpu
system_profiler

No multi-platform build

The build is on either Linux or OS X rather than both

The reason to build in Linux is

• the build time is significantly lower

The reason to build in OS X is

• it runs less efficiently (compared to Linux) in a virtual machine (described in the topic "OS X")

OS X

OS X runs less efficiently (compared to Linux) in a virtual machine

regarding

file access because

• vmhgfs doesn't close a file handle to CMakeFiles/CMakeTmp/CMakeCache.txt during cmake so that read and write access is denied (until vmware-vmx is terminated) and the build can't continue
• mount volume smb:// is unreliable because around 1/1000 file access during build result in "file not found"
• mount -t smbfs mount with permission that prevent read and write access for all users and prevent stat resulting in ls printing ? instead of stat information
• mount -t nfs is unreliable because even the best Windows NFS server [Allegro NFS]((https://github.com/franzinc/nfs) is unreliable because its free build has a memory limit

debugging because

• gdb lack -ex

Linux

Simultaneous make jobs

The make command is make -j $((nproc/ 2)) rather than make -jnproc`` because

• make -jnproc`` return "Killed (program cc1plus)" because memory is inadequate
• nproc return "32" which use much memory

Build time

• Linux: 2:05
• OS X: 36:48

IRC

The setting

notifications:
    irc: "irc.freenode.net#dolphin-dev"

isn't used because

• it's primarily meaningful for the author to know when a build failed
• it's not always meaningful for everyone that read the communication in #dolphin-dev to know that a build failed

Discussion

Billiard

[02:05] @Billiard JPeterson: I don't care if you add w/e this is, another build system file, w/e

[03:36] @Billiard lol, so what is travis CI?

Sonicadvance1

21:47] @Sonicadvance1 JPeterson, wtf is that?

The travis ci script is described in https://github.com/mirror/dolphin-emu/issues/39#issuecomment-19506030

Why is .travis.yml removed in https://github.com/mirror/dolphin-emu/commit/448e0900d11fa1fece832e2f6034710528637d9a ?

[03:44] @Sonicadvance1 JPeterson, My response is no. I'm not going to fight you on these things

Why don't you want to describe the reason that .travis.yml shouldn't be in the repo?

You should describe the reason because

• reasoned arguments should in general receive reasoned arguments in response (if there's a disagreement with the arguments) because of the benefit of reasoned communication for society

[03:55] @Sonicadvance1 I'm an unreasonable person

I request that you describe the reason for removing .travis.yml

delroth

@delroth

So... what's the point compared to the job that the buildbot currently does?

The reply is in the topic "Problem" and "Solution"

Does this test the build on as many platforms as we currently support (I guess it does not, I don't see a Windows build recipe).

The reply is in the topic "Platform"

Has anyone reviewed this change prior to you pushing it to the repository …

Approval was

• requested in #dolphin-dev around the date of this post
• granted by a reply from Billiard quoted in this topic

… and expressed his agreement with the idea of duplicating some of our systems?

It's not clear if the reply agree with the arguments for the patch in the topics "Problem" and "Solution" because it doesn't reply directly to them

Cool, why is that on Github?

The reply is in the topic GitHub benefit

I didn't know we were using Github. I should probably update the buildbot to go fetch stuff from there, you always leave me out of these discussions!

There's no benefit in fetching from GitHub compared to Google code because

• the mirror is up to one day behind
• Google Code has comparable download speed

Otherwise, this is just so stupid. Does Travis CI test Windows builds? Does Travis CI test Android builds?

No it only builds in Linux

… Does Travis CI test several configurations like Debug/Release on several GCC versions?

No, however it can be changed to run any bash command

Have you ever read our documentation? Pages like https://code.google.com/p/dolphin-emu/wiki/BuildbotTry for example, which give the correct solution to your problem?

Even when considering buildbot-try.sh a Travis CI script has value because it doesn't require

• an additional command than git push to run
• additional registration than on GitHub

[23:36] if you want to replace the dolphin infrastructure, fine, go ahead

Complement is a better description than replacement because

• Travis CI can't adequately replace http://buildbot.dolphin-emu.org because it only support build on one platform which is an important limitation

[23:37] but if you do anything about it, I don't want to have anything to do with it anymore [23:37] (wouldn't want people to confuse my work for yours - for reasons that should seem obvious to some people)

If the reason for a change to buildbot is logical it should be implemented

[23:37] so basically, if you want to have fun duplicating stuff for no valid reason, and saying that "our tools are broken, I should be able to push -f" [23:38] then tell me as soon as possible about it so I can shut down overlord.dolphin-emu.org (which I basically host for free) and transfer dolphin-emu.org to someone

I don't agree that http://buildbot.dolphin-emu.org should be disabled because

• it can test a build on multiple platforms which Travis CI doesn't support

Matt_P

[03:12] @Matt_P @Billiard JPeterson: I don't care if you add w/e this is, another build system file, w/e [03:12] @Matt_P that is not a yes [03:12] @Matt_P if someone says they dont care, its NOT a yes

[03:14] @Matt_P (and you have access to try build now)

There exist arguments n the topic "Problem" and "Solution" that a Travis CI script has value despite the presence of Tools/buildbot-try.sh¨

What's your argument that .travis.yml shouldn't be in the repo?

[03:15] @Matt_P That is an invalid argument, first provide a reason it should exist to begin with.

The reason is in the topic "Problem" and "Solution" in this post

[03:36] @Billiard lol, so what is travis CI? [03:37] @Matt_P I honestly have no fucking clue.

[03:40] @Matt_P 13:35 @Matt_P But afaik JPeterson, you are the only real user. [03:40] @Matt_P 13:35 @Matt_P So keep it in your own branch and dont add it to gcode repo.

There's a reply to this in the topic "Problem" after the text

.travis.yml should be in origin rather than being pushed by the GitHub user together with the test commit because …

[04:03] @Matt_P include it int he test branch

[04:04] @Matt_P We currnetly have a build platform.

[04:04] @Matt_P If you don't want to use it, then don't. But don't add your random files to the repo.

[03:40] @Matt_P weird, looks like I have'

no you haven't described the cost of having it in the repo that makes the net utility negative

describe the cost of having .travis.yml in the repo that's larger than its utility so that the net utility is negative

Jasper

The arguments from Jasper are paraphrased because the author

• doesn't approve copying messages from a chat (rather than post) interface f.e. a IRC channel to GitHub because of the arguments described in the topic "Saving communication" in IRC

[03:14] @Jasper [paraphrased] The advantage with having a Travis CI build scrupt in the repo is

• GitHub has Travis CI integration which has meaning because that makes it efficient to receive information about the build test
• it doesn't have a significant disadvantage because the Travis CI script doesn't have a negative effect

The disadvantages with the Travis CI service are

the setting to enable it could be easier it doesn't run the build on more than one platform it's not possible to send a message from Ttravis CI to an IRC channel Travis CI doean't have a use because the buildbot service with the page http://buildbot.dolphin-emu.org build in the same environment as Travis CI

RachelB

[03:57] @RachelB JPeterson: i think in the time we've spent discussing this, you could have just added it to your clone on github and left it out of the main repo

It's described in the topic "Problem" and the "Solution" that it's inefficient

[03:58] @RachelB it is unlikely anyone else will benefit from it being in the main repo, so there's little benefit to any communication, reasoned or not

[03:59] @RachelB now, i don't see the harm in including it, but the benefit does not seem to be worth the effort required to convince those who's permission is required

[04:07] @RachelB JPeterson: there are 6 people with a fork of dolphin on github. 3 of them have never committed anything. 1 of them is me, and i've made 1 commit, which i know builds fine, and i probably will never use it again. 1 just argued against adding it to the repo, and the last one is you.

[04:07] @RachelB so i'm not really seeing any demand for it

[john-peterson]

OS X

Library

The pre-build .travis.yml command is

brew install cmake

because

cmake isn't in the environment

These packages are excluded because

git lzo

is already in the environment

pkg-config gettext libao ffmpeg

aren't found by cmake

portaudio sound-touch wxmac sdl glew

the repo versions are used instead

Reference

• How to build Dolphin in Mac OS X doesn't suggest a library installation
• About OSX Travis CI Environment
• homebrew/tree/master/Library/Formula
  • PulseAudio 3.0

Package search

-- Could NOT find ALSA (missing:  ALSA_LIBRARY ALSA_INCLUDE_DIR) 
ALSA NOT found, disabling ALSA sound backend

ao NOT found, disabling ao sound backend

libav not found, disabling AVI frame dumps

bluez NOT found, disabling bluetooth support

Using static lzo from Externals

-- Found OpenAL: /System/Library/Frameworks/OpenAL.framework  
OpenAL found, enabling OpenAL sound backend

OpenMP parallelization disabled

PulseAudio NOT found, disabling PulseAudio sound backend

pkg-config is required to check for libav

-- Performing Test PORTAUDIO
-- Performing Test PORTAUDIO - Failed
PortAudio not found, disabling mic support

Using static SDL from Externals

Using static SFML 1.5 from Externals

Using static SOIL from Externals

Using static soundtouch from Externals

Using static GLEW from Externals

-- Could NOT find Gettext (missing:  GETTEXT_MSGMERGE_EXECUTABLE GETTEXT_MSGFMT_EXECUTABLE)

-- Could NOT find wxWidgets (missing:  wxWidgets_FOUND) 
-- Could NOT find wxWidgets (missing:  wxWidgets_FOUND) 
Using static wxWidgets from Externals

Xrandr not found

-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.5") 
Using shared zlib

[john-peterson]

Linux

Reference

• How to build dolphin on Linux

Library

The install command is

sudo add-apt-repository -y ppa:fransschreuder1/usbpicprog-stable
sudo apt-get -qq update
sudo apt-get -qq install git cmake gettext libwayland-dev wx2.9-headers libwxbase2.9-dev libwxgtk2.9-dev libgtk-3-dev libsdl1.2-dev nvidia-cg-toolkit libxrandr-dev libxext-dev libglew-dev libavcodec-dev libavformat-dev libsoil-dev libsfml-dev libopenal-dev libao-dev libasound2-dev libpulse-dev libportaudio-dev libsoundtouch-dev libbluetooth-dev libreadline-dev libswscale-dev liblzo2-dev

Package search

OpenMP parallelization enabled

-- Found ALSA: /usr/lib/x86_64-linux-gnu/libasound.so 
ALSA found, enabling ALSA sound backend

ao found, enabling ao sound backend

bluez found, enabling bluetooth support

PulseAudio found, enabling PulseAudio sound backend

-- Found OpenAL: /usr/lib/x86_64-linux-gnu/libopenal.so 
OpenAL found, enabling OpenAL sound backend

-- checking for modules 'wayland-egl;wayland-client;wayland-cursor'
--   package 'wayland-egl' not found
--   package 'wayland-cursor' not found
Wayland support disabled

X11 support enabled

Xrandr found

-- checking for modules 'libavcodec>=53.35.0;libavformat>=53.21.0;libswscale>=2.1.0;libavutil>=51.22.1'
--   found libavcodec, version 53.35.0
--   found libavformat, version 53.21.1
--   found libswscale, version 2.1.0
--   found libavutil, version 51.22.1
libav found, enabling AVI frame dumps

-- Performing Test PORTAUDIO
-- Performing Test PORTAUDIO - Failed
PortAudio not found, disabling mic support

Using shared lzo
Using shared soundtouch

-- Looking for include files CMAKE_HAVE_PTHREAD_H
-- Looking for include files CMAKE_HAVE_PTHREAD_H - found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE 

-- Could NOT find SDL2 (missing:  SDL2_LIBRARY SDL2_INCLUDE_DIR) 
Using shared SDL

Found SFML: /usr/include

Using static SOIL from Externals

-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.3.4")
Using shared zlib

GLEW found

-- Found wxWidgets: TRUE 
Found wxWidgets version 2.9.3
Found wxWidgets version 2.9.3

gthread-2.0 found

pangocairo found

-- Found Gettext: /usr/bin/msgmerge 

Discussion

[21:47] @Sonicadvance1 and why does it have nvidia-cg-toolkit in it? :/

It runs apt-get -qq install nvidia-cg-toolkit because it's in the build instruction in

• How to build dolphin on Linux

[john-peterson]

Cygwin buildbot

Problem

Sometimes this message is displayed for 20 min (perhaps indefinitely)

uname -a
CYGWIN_NT-6.1-WOW64 pc 1.7.18(0.263/5/3) 2013-04-19 10:39 i686 Cygwin

Tools/buildbot-try.sh
Branch name: input
Change author: john_peterson
Short rev: 3.5-1430
using 'pb' connect method

Changing buildbot to buildbot --verbose print

Tools/buildbot-try.sh
Branch name: input
Change author: john_peterson
Short rev: 3.5-1430+
2013-06-17 01:06:24+0200 [-] Log opened.
2013-06-17 01:06:24+0200 [-] using 'pb' connect method

On another system the command appear to complete

cat /etc/issue
Ubuntu 13.04 \n \l

Tools/buildbot-try.sh
Branch name: input
Change author: john_peterson
Short rev: 3.5-1430+
2013-06-17 01:08:29+0200 [-] Log opened.
2013-06-17 01:08:29+0200 [-] using 'pb' connect method
2013-06-17 01:08:35+0200 [-] job created
2013-06-17 01:08:35+0200 [-] Starting factory <twisted.spread.pb.PBClientFactory instance at 0x29073b0>
2013-06-17 01:08:35+0200 [Broker,client] Delivering job; comment= None
2013-06-17 01:08:35+0200 [Broker,client] job has been delivered
2013-06-17 01:08:35+0200 [Broker,client] not waiting for builds to finish
2013-06-17 01:08:35+0200 [Broker,client] Stopping factory <twisted.spread.pb.PBClientFactory instance at 0x29073b0>
2013-06-17 01:08:35+0200 [-] Main loop terminated.

Solution

What's the solution?

[john-peterson]

buildbot twisted.cred.error.UnauthorizedLogin

Problem

Sometimes this message is displayed

Tools/buildbot-try.sh
Branch name: input
Change author: john_peterson
Short rev: 3.5-1430
using 'pb' connect method
job created
Unhandled Error
Traceback from remote host -- Traceback (most recent call last):
Failure: twisted.cred.error.UnauthorizedLogin:
twisted.cred.error.UnauthorizedLogin:

Solution

What's the solution?