search

JayNakrani / saltfs

Salt Stack; A tool to automate servo's infra
http://build.servo.org/
0 stars 0 forks source link

Add benchmark runners in buildbot config. #1

Open JayNakrani opened 8 years ago

JayNakrani commented 8 years ago

This issue is intended for discussion and keeping track of the progress on this task

From @larsbergstrom's comment on IRC, I tried to understand buildbot config and I am now trying to add a benchmark runner there. From what I understood so far, I am trying to do it this way. I am planning to add a builder—dromaeo-benchmark that instead of building servo, just runs some benchmark. For running the benchmark, we can specify those steps in build factory—dromaeo_benchmark_factory

A few things are still unclear to me.

  1. How does buildbot work? I mean do all the builds happen on the same machine or different ones? (jdm answered this already)
  2. How do I know what's current directory? Just do os.getcwd()?
  3. We need to get machine information (OS, Hardware etc) while submitting the results to AWFY. How can I get them? Would standard python code (something like platform) for doing that work here too?!

@jdm, @larsbergstrom: Please give your feedback on this. Links to any documentation that answers my questions would be helpful too. Also let me know if I misunderstood anything here. :smiley:

PS: These questions may sound stupid. Bear with me on this, as I am not familiar with how this all works!

jdm commented 8 years ago

Builds happen on separate machines (see http://build.servo.org/buildslaves). I'll let Lars chime in on the other questions.

jdm commented 8 years ago

cc @edunham for this stuff, too!

JayNakrani commented 8 years ago

Great. Thanks for the input!

JayNakrani commented 8 years ago

Just came across this. One way to solve platform-os issue is to fix OS & hardware for each build-slave. And add their entries into AWFY's awfy_machine table. Pass their ids to each build-slave as properties argument. Now inside each buildslave, we can access properties dictionary and get the machine id.

Any thoughts on this?

larsbergstrom commented 8 years ago

@dhananjay92

For #2, you should be able to ask for the cwd from python. But, in general buildbot will create a directory along the lines of /Users/servo/buildbot/slave/mac-rel-css/build, where the name mac-rel-css is whatever you call your "build" configuration.

For #3, all of the mac minis have the same configuration: i7 2.3GHz quad-core, 8GB RAM, 256GB SSD.

Let me know if there's anything I can do to help! The only other step that will be required is if there are any libraries that need to be provisioned to the machines (e.g., stuff you would use brew install), which I can also help with.

Thanks for looking into adding this support for us!

JayNakrani commented 8 years ago

Thanks @larsbergstrom. :smiley:

Ah, so all the machines have same hardware configs!

@jdm: Do we want to run benchmarks on machine with same config? IMO it would be more helpful to benchmark Servo on machines with diff hardware-OS config. If so, would it be possible to have some buildbot-slaves with specific config?

For now, I can just add benchmark runner on servo-head.

jdm commented 8 years ago

I am inclined to start small with the machines we have.

JayNakrani commented 8 years ago

Okay, sounds good.

JayNakrani commented 8 years ago

I have done this much so far. Still I have a couple of questions. I have put comments in that commit. @larsbergstrom Can you please take a look at it and tell me how can I do this and this?

JayNakrani commented 8 years ago

@larsbergstrom Just a friendly reminder. Can you give some hints on this? :smiley:

larsbergstrom commented 8 years ago

@dhananjay92 Super sorry for the delay - had some pretty urgent medical issues with the family and only had time to handle urgent crises / things on fire at work. Should be better this coming week!

Thanks for your patience with me :-)

JayNakrani commented 8 years ago

@larsbergstrom Oh! It's alright. I understand. :pensive: