[API][v0.3] Connect HeteroCL with Keras via Relay

[PBC48]

I have implemented a frontend for HeteroCL in the hlib directory. In the hlib directory there are two directories: frontend and op. Frontend contains the parser/transpiler and also a test directory that contains the tests that verify both the parser/transpiler and the backend functions in the op directory. Inside the op directory, there are three files: math.py, nn.py, and op.py. All of these files were formatted to meet the PEP8 standards.

I have not extended CircleCI to the tests, but I can do so in another pull

[seanlatias]

Please merge with the upstream to fix both the merging issue and the compilation error.

[PBC48]

Fixed the issue. Also added a README to the hlib directory

[PBC48]

Btw, where is heterocl.local? My code is throwing an error saying heterocl has no attribute local

[seanlatias]

We replaced hcl.local with hcl.scalar now. It's just a name change.

[PBC48]

In addition, test_vgg.py and test_xception.py have been moved to model_vgg.py and model_xception.py respectively since these require the use of a local dataset and shouldn't be included in the testing framework

[PBC48]

Should I remove model_*.py files from the tests folder and put them into another folder for demos?

[PBC48]

Also, is hcl.log and hcl.sqrt working properly? They're failing tests

[seanlatias]

We should create a "samples" directory and put all the models there.

[seanlatias]

I'll double-check hcl.log and hcl.sqrt. Where are they used?

[PBC48]

They are used in test_elemwise.py and test_math.py. hcl.power also seems to be acting up.

[seanlatias]

I will be traveling soon. So please allow me to fix some issues after two days. Basically, I'll remove the support for Python 2 and I'll also take a look at the hcl.log, hcl.sqrt, and hcl.pow issue.

[PBC48]

Added documentation for ImageNet stuff. Also made gen_imagenet.py more understandable. I'll update my report for internal usage by tomorrow.

[seanlatias]

Please merge with the latest upstream/master branch.

[PBC48]

The pythonInterp from the test says it is version 1.4 but require a python of version 2.7 or newer

[seanlatias]

Can you try updating the v1.03 to v.104 in .circleci/config.yml?

[PBC48]

I don't know how to include the relay stuff in setup.py. It's failing saying that tvm.relay can't be found

[seanlatias]

I think you'll also need to add TVM installation into the config.

[PBC48]

how do I do that?

[seanlatias]

Let me try.

[seanlatias]

Please try #145.

[seanlatias]

I thought you don't need topi. In this case, just add this line to "install_tvm.sh" under ".circleci".

cd ../topi/python; python setup.py install --user; cd ../..

[PBC48]

I also added nnvm too

[seanlatias]

Sorry, the path I gave you was wrong. It should be

cd topi/python; python setup.py install --user; cd ../..

[seanlatias]

Seems like you just need a very simple fix. I'll start reviewing the rest of the code.

[PBC48]

It's just an incompatibility between old hlib and new hlib with the lenet and sobel samples

[PBC48]

Yay!

[seanlatias]

Could you modify the file according to the review? I'll modify the rest. Thanks.

[PBC48]

Sorry for the delay. I'll get these done by today

[PBC48]

How do I know if my frontend tests were added to the testing framework?

[seanlatias]

We use pytest for testing. You can check the file .circleci/config.yml. You'll see that I use python -m pytest dir to test the test cases under certain directory.

[seanlatias]

@PBC48 Please fix the tests if possible. Thanks!

[PBC48]

It's strange. I think the issue is coming from heterocl.log and heterocl.pow. Were there any changes in these functions?

[PBC48]

Also sqrt is weird.

[PBC48]

hcl.power*

[seanlatias]

Can you show me how can I reproduce the bug with a simpler code?

[PBC48]

Try this: """ import heterocl as hcl import hlib import numpy as np import numpy.testing as tst

hcl.init(hcl.Int())

def elemwise_pow(input1, input2, name='elemwise_pow'): return hcl.compute( input1.shape, lambda *x: hcl.power(input1[x], input2[x]), name=name)

A = hcl.placeholder((1,3)) B = hcl.placeholder((1,3)) O = hcl.placeholder((1,3)) s = hcl.create_schedule([A,B],elemwise_pow) f = hcl.build(s) A = hcl.asarray(np.random.randint(5, size=(1,3))) B = hcl.asarray(np.random.randint(5, size=(1,3))) O = hcl.asarray(np.zeros((1,3))) f(A,B,O) print(A,B,O) """ result: [[3 1 4]] [[3 4 3]] [[1104674816 1065353216 1115684864]]

[PBC48]

I should expect: """[27,1,64]"""

[seanlatias]

Please check #147.

[PBC48]

If that's the case, then it should work, but I can't seem to fetch and merge that into my repo

[seanlatias]

Try git fetch --all then git pull upstream master if you set the upstream remote correctly.

[PBC48]

It works now

[PBC48]

Just kidding. the merge didn't work for some reason

[seanlatias]

Then I guess you need to fix some conflicts. Shouldn't be too hard. Let me know otherwise.

[PBC48]

I says everything is up to date

[seanlatias]

After git pull, you don't need to commit. You can directly push.

[PBC48]

From the git log, it looks like the fix was included, but nothing has changed

[seanlatias]

@PBC48, can you check if you can fix test_keras.py? I'm not sure how to fix that. Please pull before you test. I have fixed some bugs.

[seanlatias]

Nevermind, I figured out the bug.

[PBC48]

Yay

[seanlatias]

@PBC48 Just one question, why do we need to return moving_mean and moving_var for the batch_norm function?