search

usc-cloud / goffish

USC GoFFish Graph Analytics Framework
32 stars 11 forks source link

Support enum types in GoFS #75

Closed simmhan closed 11 years ago

simmhan commented 11 years ago

Trivially support enum types by mapping them to byte type. E.g. for twitter data, there are several different edge types. Usng strings is costly and using ints/bytes is opaque. Can we add some semantic sugar?

sooniln commented 11 years ago

I'm not entirely sure what we need here, do you have an example?

simmhan commented 11 years ago

Yes. E.g. we’re looking at a twitter graph where the edges can have the values “mentions, retweets, wasMentioned, wasRetweeted”. While the GML/Slices could store these four values as strings, it may be costly in terms of I/O storage/access/parsing. Instead, if we could have the template GML define their type as:

type: “enum: mentions(0), retweets(1), wasMentioned(2), wasRetweeted(3)”

Then the slices could store the int values rather than the string values, with the template holding the mapping. In the graph API, we could provide a resolver that gets the property value either as int or string.

--Yogesh

Yogesh Simmhan | mailto:simmhan@usc.edu simmhan@usc.edu | http://ceng.usc.edu/~simmhan ceng.usc.edu/~simmhan | skype skype:simmhan simmhan | cel tel:+15404494770 +1 (540) 449 4770

From: Soonil Nagarkar [mailto:notifications@github.com] Sent: Sunday, July 14, 2013 1:14 PM To: usc-cloud/goffish Cc: Yogesh Simmhan Subject: Re: [goffish] Support enum types in GoFS (#75)

I'm not entirely sure what we need here, do you have an example?

— Reply to this email directly or view it on GitHub https://github.com/usc-cloud/goffish/issues/75#issuecomment-20942891 .

sooniln commented 11 years ago

Why don't we leave this up to the client to implement? This is 5-10 lines of code to add on the client side, but a decent amount of work to implement properly in GoFS, as well as being a hacky extension to GML.

simmhan commented 11 years ago

That’s fine. Low priority if it is non-trivial. Support for Booleans fall in this space too. Retain this issue but push it for 2.x milestone when we can revisit and close/implement it.

Yogesh Simmhan | mailto:simmhan@usc.edu simmhan@usc.edu | http://ceng.usc.edu/~simmhan ceng.usc.edu/~simmhan | skype skype:simmhan simmhan | cel tel:+15404494770 +1 (540) 449 4770

From: Soonil Nagarkar [mailto:notifications@github.com] Sent: Sunday, July 14, 2013 1:26 PM To: usc-cloud/goffish Cc: Yogesh Simmhan Subject: Re: [goffish] Support enum types in GoFS (#75)

Why don't we leave this up to the client to implement? This is 5-10 lines of code to add on the client side, but a decent amount of work to implement properly in GoFS, as well as being a hacky extension to GML.

— Reply to this email directly or view it on GitHub https://github.com/usc-cloud/goffish/issues/75#issuecomment-20943113 .

sooniln commented 11 years ago

we support booleans, ints, longs, floats, doubles and strings at the moment.

simmhan commented 11 years ago

Oh, it thought we only supported int, double and string.

I’ll update the documentation I just checked in.

So the vertex ID properties are int and edge ID properties are long?

Yogesh Simmhan | mailto:simmhan@usc.edu simmhan@usc.edu | http://ceng.usc.edu/~simmhan ceng.usc.edu/~simmhan | skype skype:simmhan simmhan | cel tel:+15404494770 +1 (540) 449 4770

From: Soonil Nagarkar [mailto:notifications@github.com] Sent: Sunday, July 14, 2013 1:31 PM To: usc-cloud/goffish Cc: Yogesh Simmhan Subject: Re: [goffish] Support enum types in GoFS (#75)

we support booleans, ints, longs, floats, doubles and strings at the moment.

— Reply to this email directly or view it on GitHub https://github.com/usc-cloud/goffish/issues/75#issuecomment-20943190 .

sooniln commented 11 years ago

GML itself only supports long, double and string. however GoFS properties are allowed to be int, long, float, double, and string, so we map the GML types to GoFS types as specified in the wiki page on GML. vertex id and edge id are not 'properties' per se, so they don't undergo the same conversion, but they are both longs.

simmhan commented 11 years ago

Ok. I’m documenting the JSON format (which Santosh is translating to GML). I’ll retain the full set of supported types since this is meant for external use. We can continue to do our internal mapping within GML.

Yogesh Simmhan | mailto:simmhan@usc.edu simmhan@usc.edu | http://ceng.usc.edu/~simmhan ceng.usc.edu/~simmhan | skype skype:simmhan simmhan | cel tel:+15404494770 +1 (540) 449 4770

From: Soonil Nagarkar [mailto:notifications@github.com] Sent: Sunday, July 14, 2013 1:36 PM To: usc-cloud/goffish Cc: Yogesh Simmhan Subject: Re: [goffish] Support enum types in GoFS (#75)

GML itself only supports long, double and string. however GoFS properties are allowed to be int, long, float, double, and string, so we map the GML types to GoFS types as specified in the wiki page on GML. vertex id and edge id are not 'properties' per se, so they don't undergo the same conversion, but they are both longs.

— Reply to this email directly or view it on GitHub https://github.com/usc-cloud/goffish/issues/75#issuecomment-20943291 .