Dataset and Transform abstraction + Pytorch Geometric Compatibility

typedb / typedb-ml

TypeDB-ML is the Machine Learning integrations library for TypeDB

https://vaticle.com

Apache License 2.0

550 stars 93 forks source link

Dataset and Transform abstraction + Pytorch Geometric Compatibility #154

Closed hkuich closed 2 years ago

hkuich commented 3 years ago

What is the goal of this PR?

see title

What are the changes implemented in this PR?

split all tensorflow-related code into separate folder
moved all generally important code for construction of grakn-networkx-dataset into kgnc_data_loader
added standard_kgcn_transform.py to transform networkX graph by example, rather than iterating through entire dataset
adjust all to latest client-python (alpha-10)
tensorflow example still works...

BEWARE

I will update to new client as soon as you release it. Then we will test tensorflow diagnosis example, & do the same in pytorch --> then the PR is really "finished"

hkuich commented 2 years ago

@jmsfltchr : are you guys still working ont his? Can I help in some way?

jmsfltchr commented 2 years ago

@jmsfltchr : are you guys still working ont his? Can I help in some way?

We got blocked by an issue where the performance dropped after the upgrades we (@hkuich, @jorenretel and I) had added. The convergence was way slower for just the TensorFlow model. I think the biggest help would be to debug that!

jmsfltchr commented 2 years ago

Ok I've revised what progress we made: In order to see the tests running in our CI, and to use TypeDB, I built on top of this PR on this branch. I was never able to merge it because of the regression I just mentioned. At the moment I see that test-end-to-end was failing, but I think because of a reason other than the performance regression. Any attempts to build of work on that branch welcome.

jmsfltchr commented 2 years ago

The changes in this PR were merged in https://github.com/vaticle/kglib/pull/158 such that the latest updates to the build system and CI could be used on the changes