search

ucrcsedept / galah

An automated grading system geared towards processing computer programming assignments.
Apache License 2.0
42 stars 8 forks source link

Rework galah-sheep-producer into vmfactory. #398

Closed itsjohncs closed 10 years ago

itsjohncs commented 10 years ago

The sheep's producer thread is being split off into its own component, vmfactory. This will involve creating a proper entry point for the component along with changing the communication method from ZeroMQ to Redis along with moving much of the logic into the new core module.

The choice of which component to migrate to the new architecture is an important one. I chose the vmfactory because it is not as trivial as the taskrunner component which may trick me into making too many simplifying assumption, while not being as difficult as the other components which might make me get lost in this task for too long.

Data models will have to be created to interface with Redis, the exact layout of the Redis database will have to be decided on, and the base module will have to be partially implemented. Various forms of testing should also be explored for this component. Each of these large subtasks should receive their own issues as I imagine each will end up being fairly complex and I'd like threads of thought for each.

itsjohncs commented 10 years ago

I noticed that my telnet connection to Redis hasn't expired over the past few days. Since I'm not sending any kind of heartbeat signal and TCP isn't doing it for me I think Redis just leaves a connection open until it receives an explicit close. I'm not sure how desirable this behavior is because I can imagine connections can start stacking up if nodes start crashing a lot.

I'll want to keep an eye out for this.

itsjohncs commented 10 years ago

With the close of #402 this issue is close to being completed. All that remains is to get automated testing up and convince myself that the vmfactory is stable and robust.

itsjohncs commented 10 years ago

I think the vmfactory is pretty solid. I'm excited to see it perform with the whole system though, so I'll leave the tests we have for it alone and not add any more just yet. The method of testing we're using (starting it up, waiting to see if it does things to the DB) is so hacky anyways I question its usefulness. Supposedly the fact that the API is "stable" should make it so the tests last through any big changes to the vmfactory, but I have my doubts.

Anyways, I'm happy. Closing this issue and moving on to the next component.