CI and GitHub

[173210]

I'm not good at testing and sometimes my broken pull requests are merged, even though @urherenow and others are reviewing my pull request. Also I'm sorry for bothering with those pull requests.

So, I suggest to integrate CI with GitHub to solve the issue. The build check will be automated and in the future we can introduce other tests.

We have several options for introducing CI for pull requests. Please let me know which you prefer.

@lavanoid's Buildbot

It is working to provide nightly builds, but nothing to do with testing and pull requests. By introducing GitHub integration with a tool like github_buildbot.py, it can be an automated testing bot.

Jenkins

Jenkins is known better and it has plugins for better GitHub integration.

GitHub and Jenkins pull request checking

Travis CI

Travis CI is also known well because it's free and integrated with GitHub. Therefore, it has addons useful for projects on GitHub, and especially I'm interested in Coverity Scan.

Coverity Scan

Coverity Scan is a free static code analysis tool for Java, C, C++, C# and JavaScript. It analyzes every line of code and potential execution path and produces a list of potential code defects.

In short, the bot blame you for your bad code before others (including me) do.

[ghost]

I like the last one xD

Jenkins, sounds really good and I like the fact that it has plugins, making it more modular. It's 4AM ATM to I'll look deeper into it, later.

I'd also consider Travis CI, but again I'll look into it.

Like I said on GBATemp, I gotta put my VPS back online xP Been down for a week.

[Syphurith]

@lavanoid To use the Coverity it must have some tests to ensure most of the code is covered. Appveyor for windows, Travis for Linux? Found some links.. https://github.com/Lectem/TravisCI-for-devkitPro/blob/master/REAMDE.MD https://github.com/smealum/ctrulib/blob/master/.travis.yml https://github.com/cpp3ds/cpp3ds/blob/master/.build-deps.sh

[173210]

@Syphurith I don't know Appveyor, but I have used Travis CI before. This uses devkitARM. https://github.com/psp2sdk/libs/blob/master/.travis.yml

[ghost]

Uuh. Welp, if anyone wants Coverity working, they can work on it xP College is enough work.

Will look into those links later this evening. Mobile web sucks.

[Syphurith]

@173210 well.. All those except the last use devKitARM for travis. I just find some as examples. @lavanoid Oh? Thanks for every effort on this. Note To whoever wanna create a valid Travis-CI file, that you might need to test it many times. For this, i suggest you to use your own repo as the build repo first, so your test commits would spoil the main repo. Once the script is done, you can easily port it (API key might differ.).

[ghost]

I have a question. Can I build a web page that you guys can sign in, where you can test your code and stuff? Just an idea. Looking into the suggested tools.

[ghost]

Got Travis CI sorta working but it downloads devkitPRO dependencies every time :/ Very slow compared to my VPS.

[173210]

@lavanoid See my .travis.yml This should be better. https://github.com/psp2sdk/libs/blob/master/.travis.yml

sudo: false

cache:
 directories:
   - $HOME/devkitPro

before_install:
 - set -e
 - curl -Ls https://download.sourceforge.net/devkitpro/devkitARMupdate.pl | perl

Note: the before_install part is different from the one in psp2sdk because it has a dirty hack to ignore an issue which devkitPro hosting service had.

[ghost]

Shouldn't sudo be true, to install imagemagick via apt-get (travis supports apt-get install)

[173210]

@lavanoid Travis CI has an apt-get addon. Use it. Don't call apt-get install directly.

[173210]

addons:
 apt:
  packages:
   - doxygen
   - graphviz

[ghost]

should imagemagick be added to those packages or?

[173210]

@lavanoid You need neither doxygen nor graphviz, right? :D

[ghost]

I wouldn't think so xP

[ghost]

https://api.travis-ci.org/jobs/99668221/log.txt?deansi=true

https://gist.github.com/lavanoid/5763967f9985d127aa6d

[173210]

@lavanoid You have already ran devkitARMupdate.pl. Why do you run again? remove perl devkitARMupdate.pl.

[ghost]

I must've got distracted Dx Serious brain farting xD

Will fix ;)

[173210]

I guess you don't know what this does:

curl -Ls https://download.sourceforge.net/devkitpro/devkitARMupdate.pl | perl

curl will download the file and output to the standard output (i.e. on RAM). perl will execute it. That's the line does. So you don't need perl devkitARMupdate.pl.

[173210]

Also, note that all logs is public. So I can see your log here. https://travis-ci.org/lavanoid/rxTools Of course, your repository is open, so I can also see your .travis.yml https://github.com/lavanoid/rxTools/blob/master/.travis.yml

Everything is public. Please note that.

[ghost]

I know what it does dude -_- I got habits that's all. Easily distracted. Curl will pipe the output of the file into perl, where it then gets executed instead of writing to a file.

Like I said, easily distracted.

Note taken.

You should see one of my Linux scripts, if you wanna see how good I am with pipes ;)

https://gist.github.com/lavanoid/2c1bdd47c0825eba07fd

[ghost]

I downloaded this version https://github.com/roxas75/rxTools/archive/a08cfdd014cbd4740db2918028444e5b8ddd2af4.zip of rxTools, the last known version to be "compilable" and it doesn't build.

linking rxtools.elf built ... rxtools.bin make[1]: Leaving directory /root/debug/rxTools-a08cfdd014cbd4740db2918028444e5b 8ddd2af4/rxtools' 0+1 records in 1+0 records out 917504 bytes (918 kB) copied, 0.00145973 s, 629 MB/s make[1]: Entering directory/root/debug/rxTools-a08cfdd014cbd4740db2918028444e5 b8ddd2af4/CakeHax' make[1]: * No rule to make target bigpayload'. Stop. make[1]: Leaving directory/root/debug/rxTools-a08cfdd014cbd4740db2918028444e5b 8ddd2af4/CakeHax' make: * [code.bin] Error 2

[ghost]

So I presume I have configured Travis correctly, seeing as the build fails even on my own systems.

[173210]

@lavanoid I guess the zip doesn't include submodules.

[ghost]

Will try a direct git clone. One sec.

[173210]

I had look at your latest log. Submodules of submodules don't seem cloned. You may add git submodule --init --recursive at the end of before_install. Why don't you remove these?:

before_install:
  # Travis has an OLD doxygen build, so we fetch a recent one
  # Prepare devkitArm update
  - sudo apt-get install -y imagemagick cmake
  - export DEVKITPRO=$HOME/devkitPro
  - export DEVKITARM=${DEVKITPRO}/devkitARM
  - mkdir -p $DEVKITPRO
  - cd $DEVKITPRO

[ghost]

Will do. Multiasking.

[ghost]

git submodule --init --recursive Doesn't work

[173210]

I forgot update.

git submodule update --init --recursive

[ghost]

I have to have the devkitpro variables specifed in the .yml, so I'll add them back into the file (Mr, you told me to remove them xD)

[173210]

@lavanoid You have not read my .travis.yml yet, have you?

env:
 global:
  - DEVKITPRO="$HOME/devkitPro"
  - DEVKITARM="$DEVKITPRO/devkitARM"

[ghost]

Nope.

Anyways, I managed to get my .yml corrupted -_- Probably my connection messed up.

Will add your stuff in a sec (P.S. I'm lazy)

[173210]

@lavanoid If you are lazy, do everything in a commit. That's my lazy coding. :)

[ghost]

Eh. BTW, since I'm compiling my fork of rxTools, I also have to modify my path variable as Cakes didn't use the devkitARM variable when compiling some of its crap.

[ghost]

Failed to compile again -_-

[173210]

@lavanoid I had failed so many times when I made .travis.yml for PSP2SDK, so don't worry. ;)

[173210]

Oh, yes. The problem is the crappy CakeHax! Update the submodule.

[ghost]

That's a relief. Have you managed to get Travis working with rxTools? Call me ignorant but every time I had a compile issue with rxTools, it stopped at building Cakes -_- Really frustrating. Had to use a PATH variable "hack", even in my own scripts just so Cakes found the armeabi (whatever it's called) files, which were in /opt/devkitPro/devkitarm/bin

Thank God! Someone else called CakeHax crap besides me xD

[ghost]

May I ask how you update the submodules? Never done it before.

[173210]

@lavanoid This is in my .profile.

export DEVKITPRO="$HOME/devkitPro"
export DEVKITARM="$DEVKITPRO/devkitARM"
PATH="$DEVKITARM/bin:$PATH"

So this should work.

env:
 global:
  - DEVKITPRO="$HOME/devkitPro"
  - DEVKITARM="$DEVKITPRO/devkitARM"
  - PATH="$PATH:$DEVKITARM/bin"

And that's which my .travis.yml does.

git -C CakeHax checkout master

That's it. I guess Google or something will answer faster than me. ;)

[ghost]

Okey doke.

As for the PATH variables that you shown, that's exactly what I did :)

[ghost]

I think the problem is the newer devkitPro. My fork always built on my systems which had older versions :/

Still failed to build.

[ghost]

I'll just rip my old devkitpro folder from my other computer, later and attempt to compile with that. I don't see why even older forks of rxTools won't compile even with the old submodules when previously they did build.

:/

[173210]

@lavanoid CakeHax was not written in correct C (or correct GCC option), so it failed building. devkitPro is completely OK. libkhax may have different problems with the newer libctru. libctru has changes breaking compatibility, so libkhax also need to be up-to-date.

[ghost]

I see. So for the most part, I did make my .travis.yml correct, right?

I'm gonna go play TankiOnline (in Japan, it's called 3dtank.com xD. I know, I gossip too much, Sorry) so I'll tinker with Travis some more, later :)

[173210]

@lavanoid You do not need to clone the repository by yourself! Remove all lines in script except make release It will solve the problem.

Also make sudo: false.

[urherenow]

I forgot why rxtools started sticking with an older branch of Cakehax. Didn't a change break something in rxtools? Either way, the current Cakehax and Cakebrah work fine with r45. The specific issue is ctrulib. If you want to continue using the same Cakehax as before, simply install ctrulib 0.6.0 and it will build fine (but I can't say if it will actually run on an N3DS because I figured all of this out after the Makefile changes that broke stuff on Windows...)

[173210]

@urherenow I don't think so. I think that's just because nobody doesn't update. Why don't you do that? ;)

[urherenow]

I did. AFTER you broke the makefiles, and the result wouldn't run on the 3DS. I haven't bothered trying again with the tree from the Dec 5 release. I guess I'll start that now.

Whether I have time to build and test depends on my wife. When it's time to go out, I don't argue :P

[173210]

@urherenow Sorry for bothering you. So #330 should be merged at first, @AlbertoSONIC.

[urherenow]

Just reinstalled devkitpro r45, cloned rxtools and did a git checkout a08cfdd (from Dec5). Updated Cakehax and Cakebrah to the latest, and it builds and runs perfectly! I thought I was going to have a black screen, but my SD card is just a bit slow. The .3dsx file also works and it's a lot smaller than the previous one (launched from menuhax->ctrbootmanager->latest HBL).