search

hongzimao / decima-sim

Learning Scheduling Algorithms for Data Processing Clusters
https://web.mit.edu/decima/
286 stars 90 forks source link

Updating Tensorflow 1.14 to 2 #32

Open larissayukimiz opened 3 years ago

larissayukimiz commented 3 years ago

Hi, Mao.

There's a specific reason why you didn't update Decima to work with Tensorflow 2? Despite using the tf.contrib...

hongzimao commented 3 years ago

No specific reason. The project was initially developed before Tensorflow 2 was available.

A side story: you might have noticed we had to deal with the static graph in tensorflow even though our problem inherently has dynamic input graph size (since the number of jobs is not pre-determined). Indeed, our best choice was pytorch, but it did not support certain sparse matrix operations that we needed. It was an unfortunate decision that we had to live with Tensorflow 1.x at that time. Later on, we actually implemented a pytorch version of the core graph neural network: https://github.com/hongzimao/gcn_pytorch

Sijing0608 commented 3 years ago

Hi, It is a very interesing project to me and I'd like to know a runable enviroment setup. Can anyone share it? Thanks!

hongzimao commented 3 years ago

Hi, It is a very interesing project to me and I'd like to know a runable enviroment setup. Can anyone share it? Thanks!

This might be helpful: https://github.com/hongzimao/decima-sim/issues/31#issuecomment-805412033