feat: ResultSet& ValueWrapper sugars for Graph Visualisation & Data Sci

wey-gu commented 8 months ago

implement: https://github.com/vesoft-inc/nebula-python/issues/318, https://github.com/vesoft-inc/nebula-python/issues/324

What type of PR is this?

[ ] bug
[x] feature
[ ] enhancement

What problem(s) does this PR solve?

Issue(s) number: #318 , https://github.com/vesoft-inc/nebula-python/issues/324

Description:

Docs polished, straightforwardly getting started
sugars to better handle ResultSet for Graph Vis/Rendering && as Pandas Dataframe
Optimized Config, now user could omit them to enable ease of first-time try, fixed SessionPool Config Check bug

`resultSet.dict_for_vis()`

easily get node list/ edge list and more information in ANY query as a result.

result = session.execute('GET SUBGRAPH WITH PROP 2 STEPS FROM "player101" YIELD VERTICES AS nodes, EDGES AS relationships;')
v = result.dict_for_vis()

# v is like:
{
    'nodes': [
        {
            'id': 'player100',
            'labels': ['player'],
            'props': {
                'name': 'Tim Duncan', 
                'age': '42', 
                'id': 'player100'
            }
        },
        {
            'id': 'player101',
            'labels': ['player'],
            'props': {
                'age': '36', 
                'name': 'Tony Parker', 
                'id': 'player101'
            }
        }
    ],
    'edges': [
        {
            'src': 'player100',
            'dst': 'player101',
            'name': 'follow',
            'props': {
                'degree': '95'
            }
        }
    ],
    'nodes_dict': {
        'player100': {
            'id': 'player100',
            'labels': ['player'],
            'props': {
                'name': 'Tim Duncan', 
                'age': '42', 
                'id': 'player100'
            }
        },
        'player101': {
            'id': 'player101',
            'labels': ['player'],
            'props': {
                'age': '36', 
                'name': 'Tony Parker', 
                'id': 'player101'
            }
        }
    },
    'edges_dict': {
        ('player100', 'player101', 0, 'follow'): {
            'src': 'player100',
            'dst': 'player101',
            'name': 'follow',
            'props': {
                'degree': '95'
            }
        }
    },
    'nodes_count': 2,
    'edges_count': 1
}

`resultSet.as_pandas()`

Get pandas dataframe with all data primitive typed

In [3]: result = session.execute('GET SUBGRAPH WITH PROP 2 STEPS FROM "player101" YIELD VERTICES AS nodes, EDGES AS relationships
   ...: ;')

In [4]: df = result.as_pandas()

In [5]: df
Out[5]: 
                                               nodes                                      relationships
0  [{'vid': 'player101', 'tags': ['player'], 'pro...  [{'src': 'player101', 'dst': 'team204', 'type'...
1  [{'vid': 'team215', 'tags': ['team'], 'props':...  [{'src': 'player111', 'dst': 'team215', 'type'...
2  [{'vid': 'player146', 'tags': ['player'], 'pro...  [{'src': 'player146', 'dst': 'team222', 'type'...

In [7]: x = df.iloc[0, 0]

In [8]: type(x)
Out[8]: list

In [9]: x
Out[9]: 
[{'vid': 'player101',
  'tags': ['player'],
  'props': {'age': 36, 'name': "Tony Parker"}}]

In [10]: type(x[0])
Out[10]: dict

wey-gu commented 8 months ago

TODOS:

[x] docs/ example, via c303cad
[x] test, via c303cad
[x] as pandas dataframe, where all cells are primarily typed(no nebula bytes) , via 74469cd
[x] Example of Frontend, via 8ce2af023597e30c3e38595b002b83178c76d721

codecov-commenter commented 8 months ago

Codecov Report

Attention: Patch coverage is 44.34783% with 64 lines in your changes are missing coverage. Please review.

Project coverage is 75.61%. Comparing base (effa675) to head (fc65c90).

Files	Patch %	Lines
nebula3/data/ResultSet.py	4.83%	59 Missing :warning:
nebula3/data/DataObject.py	90.90%	4 Missing :warning:
nebula3/gclient/net/ConnectionPool.py	80.00%	1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #323 +/- ## ========================================== - Coverage 76.90% 75.61% -1.29% ========================================== Files 18 18 Lines 2516 2625 +109 ========================================== + Hits 1935 1985 +50 - Misses 581 640 +59 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

wey-gu commented 8 months ago

@Nicole00 @haoxins what do you think, please?

haoxins commented 8 months ago

maybe we can move example/apache_echarts.html into a separate repo? After all, this is the client for Python. And we can make it as a SPA by React/Vue/Svelte/Next.js and even more.

wey-gu commented 8 months ago

maybe we can move example/apache_echarts.html into a separate repo? After all, this is the client for Python. And we can make it as a SPA by React/Vue/Svelte/Next.js and even more.

I think we could for now put it here as the self-contained hint, or in a gist to refer it to, but YES, we could further create FE libs to work with it! What do you think?

haoxins commented 8 months ago

maybe we can move example/apache_echarts.html into a separate repo? After all, this is the client for Python. And we can make it as a SPA by React/Vue/Svelte/Next.js and even more.

I think we could for now put it here as the self-contained hint, or in a gist to refer it to, but YES, we could further create FE libs to work with it! What do you think?

👍

HarrisChu commented 8 months ago

how about we do this in a new repo, e.g. graph-sci ? and it could generate dafaframe from:

nebula-graph
neo4j
ISO-GQL in future?

#example 
sci = load_wrapper("nebula")
df = sci.to_df(result)
xxx

wey-gu commented 8 months ago

how about we do this in a new repo, e.g. graph-sci ? and it could generate dafaframe from:

nebula-graph

neo4j

ISO-GQL in future?
#example 

sci = load_wrapper("nebula")

df = sci.to_df(result)

xxx

Yes we can!

And that one would be super function complete and with proper dependencies with those sci toolchain.

In this repo I kept the minimal feature workable with opinionated defaults(not tweak-able), yet with all dependencies import in function, thus our client won't depend on pandas etc...

haoxins commented 8 months ago

And that one would be super function complete and with proper dependencies with those sci toolchain.

Maybe it's not suitable to post in this thread, but I think it's worth to share the emerging project.

https://github.com/Pometry/Raphtory

Which has some great ideas!

wey-gu commented 8 months ago

And that one would be super function complete and with proper dependencies with those sci toolchain.

Maybe it's not suitable to post in this thread, but I think it's worth to share the emerging project.

https://github.com/Pometry/Raphtory

Which has some great ideas!

Lovely embedded one!quite inspiring for the API!

vesoft-inc / nebula-python