A new sequence to sequace example for better underastanding the platfarm better

[HariKrishna-Vydana]

minpy is interesting and informative platform...of neural nets.....A sequence to sequence example would give a complete understanding of the platform...code in the format of linear regression example would be very useful. 1.without using solver.py script @ZihengJiang @zihaolucky @piiswrong @samuela @jermainewang @lryta

[samuela]

@harikrishnavydana did you mean to tag me?

[HariKrishna-Vydana]

@samuela i thought you were also a member minpy development team. I have tagged some of the members hoping a positive reply for my request

[lryta]

@harikrishnavydana This is in plan. Sorry for keeping you waiting. Currently we are working on merging as a subsystem of MXNet, so our schedule is a little off-track. I will do this example ASAP.

[HariKrishna-Vydana]

can u spot the error in the code...... i could not compute gradient. @lryta @jermainewang

seq_RNN.txt

[lryta]

@harikrishnavydana Can you paste your error message?

[HariKrishna-Vydana]

@lryta i dont have an error, the grad function does not get computed..it hills the whole ram of 8gb

[jermainewang]

Does it hang or something?

[lryta]

@jermainewang OOM?

[HariKrishna-Vydana]

@jermainewang it dosent hang i have pasted the code can u check it

[jermainewang]

Traceback (most recent call last):
  File "seq_RNN.py", line 133, in <module>
    a=npp.loadtxt('mddc0_si789.new')
  File "/usr/lib/python3.5/site-packages/numpy/lib/npyio.py", line 805, in loadtxt
    fh = iter(open(fname))
FileNotFoundError: [Errno 2] No such file or directory: 'mddc0_si789.new'

@harikrishnavydana could you paste the code that could run without data?

[jermainewang]

Hmm, I found the bug. It is the same as #146 . Working on it right now.

[jermainewang]

The PR #149 should fix this. Sorry for the late fix. We are busy with other MXNet team on the merge process right now. We want to fix most of the minpy overhead and have more stable support for operators in the future. I hope this issue won't block your progress too much. Please tell us if there are any other issues. Thanks!

[HariKrishna-Vydana]

@jermainewang i could not understand this

[jermainewang]

The problem is in autograd. When you do forward computation, minpy will record every operators you've called and their data dependencies (a computation graph), so we could do BP for you automatically. The problem is not all the operators you called need corresponding backward computation. For example:

def foo(x):
   y = x + 1
   print(np.argmax(y, axis=0))
   return y

The add operator here needs backward computation, while the argmax is not since it does not contribute to the target. Therefore, we need to prune them out in the backward graph, which is what the function _prune_gradient_path does. It does a DFS from the target array and mark all the operators that contribute to the target array as required, so they will be executed. Other operators will simply be removed.

The bug is actually quite stupid. The DFS is not correctly implemented so it will be deadloop sometimes. Does this make sense to you?

[jermainewang]

@harikrishnavydana any further question?