Nightly builds

[GoogleCodeExporter]

This is not a issue is more of a suggestion 
A system of Nightly builds that way more people code help testing.

Original issue reported on code.google.com by ivolu...@gmail.com on 25 Oct 2010 at 1:50

[GoogleCodeExporter]

Our release cycle is quite short, so nightly builds probably will not be a big 
improvement.
But if one has a server to share, creating nightly builds is just running make 
and then uploading the system.img and boot.img, so that should be manageable.

Original comment by polyth...@gmail.com on 26 Oct 2010 at 10:48

[GoogleCodeExporter]

The Dolphin project have a system like this, and it's awesome :

http://code.google.com/p/dolphin-emu/source/list

http://www.dolphin-emulator.com/download.html

Maybe you could ask them how to do the same thing here.

Original comment by ounga...@gmail.com on 27 Oct 2010 at 8:51

[GoogleCodeExporter]

I think Polytheus has a good idea of how to do it.

It's more an hosting problem apparently.

I think we could ask cwhchew for that maybe ?

Original comment by Tetsuo6...@gmail.com on 27 Oct 2010 at 9:22

[GoogleCodeExporter]

Original comment by polyth...@gmail.com on 28 Oct 2010 at 9:35

[GoogleCodeExporter]

Original comment by polyth...@gmail.com on 29 Oct 2010 at 7:45

• Changed state: NeedVolunteer

[GoogleCodeExporter]

Hi all, 

I'm a sysAdmin of a HPC cluster, and it has  available a node with 16 
processors, 16G ram and 3TG of hard disk (1T at my disposal).

I can reproduce the process of "repo sync & make" and then send the build to 
some external server. but I know that exisit automation for such tasks, if it 
is very urgent I can do a research  this weekend and propose some.

buildbot is a system to perform these tasks, I've never implemented this but I 
think I can do so within 3 to 5 days.

Original comment by cmcg...@gmail.com on 29 Oct 2010 at 1:43

[GoogleCodeExporter]

Hi cmcglnx,

I can provide the hosting.
You may add me on google chat - cwhchew@gmail.com

Lets work together on this, i'm keen on the idea of nightly builds too.

Original comment by cwhc...@gmail.com on 29 Oct 2010 at 3:44

[GoogleCodeExporter]

Great Idea!

but i've to warning you that my english is very poor  :p

this weekend start to work on the server...

Original comment by cmcg...@gmail.com on 29 Oct 2010 at 7:46

[GoogleCodeExporter]

its alright !

do let me know when you are done with your side. can you somehow get the builds 
to upload via ftp ?

Original comment by cwhc...@gmail.com on 30 Oct 2010 at 12:10

[GoogleCodeExporter]

Original comment by cwhc...@gmail.com on 30 Oct 2010 at 12:12

• Changed state: Started
• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

cwhchew, 

in the first instance I will make the process manually using a script and cron 
scheduled job.

here the script:

#!/bin/sh

cd ~/android-2.2/device/lg/eve/

~/repo sync || $(echo "Repo Sync Error!"  && exit )

SDATE=$(date +%Y%m%d_%H%M)

[ -x  ./extract-files.sh  ]  && ./extract-files.sh || $(echo "Extract Files 
Error!"  && exit)

cd ~/android-2.2

make -j16 || $(echo "Make  Error!"  && exit )

SIMG=out/target/product/eve/system.img
SIMGOUT=system-$SDATE.img
BIMG=out/target/product/eve/boot.img
BIMGOUT=boot-$DATE.img

lftp -u login,password some.host.com -e \
     "cd NightlyBuilds/ ;  put ${SIMG} -o${SIMGOUT} ${BIMG} -o${BIMGOUT} ; exit "  \
|| $(echo "Upload Files Error!"  && exit)

that would be the manual and rudimentary way to make the nightly build.

in the coming days I hope to implement the buildbot system for a more complete 
system: error handling, logging, etc...

now i'm doing local tests.

Original comment by cmcg...@gmail.com on 30 Oct 2010 at 4:42

[GoogleCodeExporter]

sure. how much space do you think the nightly builds will take up ? will it 
replace the previous build everytime it runs ?

Original comment by cwhc...@gmail.com on 30 Oct 2010 at 5:21

[GoogleCodeExporter]

i can provide a stable mirror for downloading or compiling with 100Mbit 
bandwidth on my linux box (debian/ubuntu/gentoo/any other, gentoo is preferred, 
but i couldn't setup building environment on it)
contact me if interested

Original comment by emiron...@gmail.com on 30 Oct 2010 at 9:56

[GoogleCodeExporter]

hi all,

I just finished my local tests (compile and upload the files to a server).

took me about 15 minutes to do the first compilation (not counting the time it 
takes to do repo sync and extract-files.sh)

BY NOW the builds can be downloaded from:

http://146.83.123.120/~openetna/ (http://portada.disc.ucn.cl/~openetna/)

but I must warn that this server is used in other services at my university. 
also is only temporary as I documented about buildbot system (I need a few more 
days)

The build takes place at 1:00 a.m. (-4 GMT) regardless of whether there are 
changes or not (which is why I hope to implement the system buildbot)

greetings!

Original comment by cmcg...@gmail.com on 31 Oct 2010 at 5:05

[GoogleCodeExporter]

cannot download system.img. Error message "You don't have permission to access 
/~openetna/NightlyBuilds/system-20101031_0232.img on this server". Boot.img 
starts downloading

Original comment by iareh...@gmail.com on 31 Oct 2010 at 5:38

[GoogleCodeExporter]

works fine for me. thanks cmcglnx !

I have created an address on openetna, http://nightly.openetna.com . This will 
direct users to the available mirrors. OpenEtna users, if you have hosting, 
drop me an email at cwhchew@gmail.com. thanks !

@emironoff : Hi, do you think you can provide an ftp account for us to transfer 
the builds ? I think it would be better if all the mirrors have the same build.

Original comment by cwhc...@gmail.com on 31 Oct 2010 at 12:02

[GoogleCodeExporter]

works for me too now.

Original comment by iareh...@gmail.com on 31 Oct 2010 at 12:12

[GoogleCodeExporter]

I would suggest removing the "out" directory and 
"device/lg/eve/proprietaries"-directory before each new compilation to make 
sure its clean.

Original comment by polyth...@gmail.com on 1 Nov 2010 at 8:28

[GoogleCodeExporter]

@polytheus, i'll change the script to make a clean build. 

Original comment by cmcg...@gmail.com on 1 Nov 2010 at 9:49

[GoogleCodeExporter]

is this of any use to flash a nightly build version? how to report problems if 
flashed nightly build version?? How could we know which problems are fixed in a 
nightly build?

Original comment by iareh...@gmail.com on 2 Nov 2010 at 3:52

[GoogleCodeExporter]

as a general rule, you should't report problems in the nightly build.

Original comment by cwhc...@gmail.com on 2 Nov 2010 at 4:14

[GoogleCodeExporter]

you answered only one part. how do we know if any bug is fixed in a build? is 
this of any use if no new bugs fixed? :)

Original comment by iareh...@gmail.com on 2 Nov 2010 at 4:21

[GoogleCodeExporter]

It is possible to extract the comments in commit's from the git during the 
source update and create a file whit that information?

The comments can help to now whats new in the night build.

It's only a idea, i don't now if it is doable :)

Original comment by ivolu...@gmail.com on 5 Nov 2010 at 1:44

[GoogleCodeExporter]

A little sugestion, can also the boot images be zipped, to prevent a posible 
crc errors in the download.

Original comment by cmgonza...@gmail.com on 5 Nov 2010 at 2:03

[GoogleCodeExporter]

I found the MD5 sums to be good to check if the file is corrupted...

Original comment by Tetsuo6...@gmail.com on 5 Nov 2010 at 2:31

[GoogleCodeExporter]

whit no doubt, but if the zip is corrupted it's brings also a good measure.

Original comment by cmgonza...@gmail.com on 5 Nov 2010 at 2:57

[GoogleCodeExporter]

hi all,

@cmgonzalez :
I see no reason to compress the file boot.img. The download error can happen 
with the zip or img  file. Also there is not much difference in size of the 
files.

@iarehman, @ivolucas :
I'm not familiar with the  repo/git system. I have not very clear how to get 
the record or summary of the changes from the repository.

if anyone can help with that, I can add the necessary changes to the script and 
upload the log file with the img and md5sum

i've been trying with: repo statu, repo diff, .... but i get no result :-(

I'm open to ideas and suggestions.

greetings

Original comment by cmcg...@gmail.com on 6 Nov 2010 at 4:09

[GoogleCodeExporter]

changelog for nightly builds?

Original comment by croxz3...@gmail.com on 6 Nov 2010 at 12:47

[GoogleCodeExporter]

no builds for nov 7

Original comment by iareh...@gmail.com on 7 Nov 2010 at 7:17

[GoogleCodeExporter]

today's build failed at MAKE

I modified the script to upload the build log to the mirrors. so you can see 
why it has failed if it failed

also changed the script to run every 12hrs (1:00 and 13:00 -4GMT) and to remove 
files older than 7 days. the script takes care of sync mirrors 

I estimate the size of the mirrors will be approximately 700MB in total (7 
days, two builds per day)

Original comment by cmcg...@gmail.com on 7 Nov 2010 at 4:21

[GoogleCodeExporter]

hi, great work in nith builds. i a using one right now, to avoid the gps on off 
bug. 
but in the last 2 days, the builds have fail. :(

Original comment by ivolu...@gmail.com on 14 Nov 2010 at 12:42

[GoogleCodeExporter]

The nith build failed, this time :
<built-in>:0: fatal error: when writing output to /tmp/ccm8CVST.s: No space 
left on device

Original comment by ivolu...@gmail.com on 15 Nov 2010 at 1:50

[GoogleCodeExporter]

Doesn't that just mean the mirror has no space left ?

To be honest two builds everyday is kind of too much :/
There is not that many commits everyday to justify so many updates ?

Needed space would be divided by two but we may get more mirros that way ;)

Original comment by Tetsuo6...@gmail.com on 15 Nov 2010 at 1:59

[GoogleCodeExporter]

Is at the start of the build. So it's the machine use to make the build that 
don't have more space (our the user that is use don't have more cota).
In the last 3 days the build have failed, maybe the garbage left from the 
failed builds is occupying to much space.

Original comment by ivolu...@gmail.com on 15 Nov 2010 at 2:13

[GoogleCodeExporter]

Hi  @cmcglnx, I'm not familiar with the  repo/git system to, but take a luck on 
the command. 

git log

It may be what we need to make a summary of the changes from the repository.

Original comment by ivolu...@gmail.com on 15 Nov 2010 at 2:56

[GoogleCodeExporter]

hi all,

sorry about the error,

the mirror space is OK,  
the worker node that build the sources has some users that filled out the tmp 
directory, but now it's ok.

Original comment by cmcg...@gmail.com on 15 Nov 2010 at 3:33

[GoogleCodeExporter]

Hi, @polytheus can you confirm that the code in the repository compiles, 
because the nith builds are falling.

thanks for the grate work you are doing.

Original comment by ivolu...@gmail.com on 15 Nov 2010 at 5:08

[GoogleCodeExporter]

We use some repositories directly from cyanogenMod or aosp upstream, so changes 
there broke our build. I have merged the needed changes to our forked 
repositories.

Original comment by polyth...@gmail.com on 16 Nov 2010 at 9:00

[GoogleCodeExporter]

I have attached my script which I used to generate the ChangeLog.
Call it in the root of the source with
./changes .repo/manifests/release-system6.1-rc4-boot2.4.2.xml HEAD
to show the changes since that specific release.
Maybe you can use that to show the changes of each nightly against the last 
release.

Original comment by polyth...@gmail.com on 16 Nov 2010 at 9:04

Attachments:

• changes

[GoogleCodeExporter]

There seems to be one breakage remaining.
You will have to do
rm -R packages/apps/AndroidTerm
before make. I don't know why this package won't build as this is thirdparty.

Original comment by polyth...@gmail.com on 16 Nov 2010 at 9:52

[GoogleCodeExporter]

you can just remove packages/apps/AndroidTerm/res/values_zh_rCN actually.

Original comment by GiorgioM...@gmail.com on 16 Nov 2010 at 11:24

[GoogleCodeExporter]

@polytheus,
thanks for the changes script, i'll use this to get the change log and then 
upload it to the mirror.

Original comment by cmcg...@gmail.com on 16 Nov 2010 at 6:23

[GoogleCodeExporter]

Looks like the 17th november build made an error :/

Original comment by Tetsuo6...@gmail.com on 17 Nov 2010 at 11:21

[GoogleCodeExporter]

Error occurring on build:

Checking API: checkapi-last
Checking API: checkapi-current
Compiling SDK Stubs: 
out/target/common/obj/JAVA_LIBRARIES/android_stubs_current_intermediates/classes
.jar
(unknown): error 4: Added public method 
android.widget.AbsListView.getOverscrollWeight

******************************
You have tried to change the API from what has been previously approved.

To make these errors go away, you have two choices:
   1) You can add "@hide" javadoc comments to the methods, etc. listed in the
      errors above.

   2) You can update current.xml by executing the following command:
         make update-api

      To submit the revised current.xml to the main Android repository,
      you will need approval.
******************************

Original comment by ivolu...@gmail.com on 22 Nov 2010 at 5:30

[GoogleCodeExporter]

execute:
make update-api

Original comment by skun...@gmail.com on 22 Nov 2010 at 6:20

[GoogleCodeExporter]

I do not know why cyanogenMod has decided to break the build. I hope that this 
does not happen too often.

Original comment by polyth...@gmail.com on 22 Nov 2010 at 10:03

[GoogleCodeExporter]

@skunk73, @ivolucas: done :-), now make goes fine

sorry for the delay... I've been busy at work and in studies

Original comment by cmcg...@gmail.com on 23 Nov 2010 at 4:31

[GoogleCodeExporter]

@cmcglnx 
 please add the changelog to the build :):)

Thanks

Original comment by ivolu...@gmail.com on 23 Nov 2010 at 10:23

[GoogleCodeExporter]

Nighlty with date 20101127_0100 won't boot. Stuck at lg screen. Same with build 
from source..

just for info.

have tryed fastboot -w no change

//Figgelus

Original comment by figgelu...@gmail.com on 27 Nov 2010 at 7:05

[GoogleCodeExporter]

Nightly build 20101203_0100 stuck at lg screen. Is it due to new kernel?

Original comment by iareh...@gmail.com on 3 Dec 2010 at 4:48