search

valueflows / agent

agent has moved to https://lab.allmende.io/valueflows/agent
10 stars 4 forks source link

experiments to produce JSON-LD agent data #20

Closed fosterlynn closed 4 years ago

fosterlynn commented 8 years ago

Starting a new issue for this to track it separately, I will continue to provide updates.

As a first experiment, I have been working on getting json-ld out of NRP for agents and their relationships, partly for my own education, but also so we can eventually actually do that as part of this collaboration. I'm posting my current results, which are NOT DONE.

You can see them here: http://dhen.webfactional.com/accounting/agent-jsonld/. This is our local herbal network, which has a much simpler org structure than Sensorica. We'll get Sensorica too, but need to get the latest code out there.

The code is here: https://github.com/valnet/valuenetwork/blob/master/valuenetwork/valueaccounting/views.py, the last method in the file, apprx line 10624, or do a find on "agent_jsonld". (This is if you are a glutton for punishment and have some time on your hands!)

Status:

  1. A few known bugs, which involve mostly me figuring out how to get what we want out of the python library we picked. Like a triple within a triple; and ordering for human readability.
  2. I think I need to separate it into 2 things: a context local to the network for agent types and relationship types, and a pull of agent and relationship data that will reference that context.
  3. I am just using whatever existing vocab is convenient for the sake of experiment, not making any statements or recommendations. When we get all of that decided, I'll change it.
  4. LOTS and LOTS of questions I have on how to do this best. At some point if I can get some help from @elf-pavlik about those, that would be great. But I need to do some more work to define and compile them, and also see if I can figure some more out myself. Will post when that is done.
ahdinosaur commented 8 years ago

wow this is really great. :tada:

@simontegg now to feed this data (and any that complies with our common vocab) into our Holodex ui. :smile:

fosterlynn commented 8 years ago

@ahdinosaur don't get too excited, see notes and caveats above. Not ready to feed in yet!! (Although I am excited, really happy to be following this path at the moment, and I'm glad you are excited too.)

And, when you get the Holodex data structure pretty finalized, I'll create whatever you like - csv is fine. This may involve needing to add some fields to NRP, so I'd like to wait until Holodex settles down.

OR - If you decide to take in json-ld, cool! But not required, I'll make what you like. No problems with interim experiments, happy if trying other datasets helps you guys, and fun to see what our networks' data looks like with a nice UI. :smile:

cc: @simontegg

bhaugen commented 8 years ago

fun to see what our networks' data looks like with a nice UI

Yes yes yes! Our current radial graph is out of control with the quantity of data in Sensorica's network (for example), and I don't want to take the time to rework it if Holodex might become available.

fosterlynn commented 8 years ago

I have a better version of the NRP json-ld feed going. Not perfect yet but definitely better. I need to switch off of this for a few days and do some NRP coding, will have more time after that.

Link to try: http://dhen.webfactional.com/accounting/agent-jsonld/

Known bug: I haven't figured out how to make the "nested rdf" thing work, to get the following result:

    "owl:inverseOf": {
        "rdfs:label": {
            "en": "is mentored by",
    },

I have a question into the rdflib support people, even though I suspect it is my lack of knowledge of how to contruct the rdf triples to make this. @elf-pavlik if you have a minute, and can easily write up triples that would make the above, that would be awesome.

Question: My Relationship instances only have id's in them. The id's of course relate to other things in the feed. Is this OK?

And it is making its own context until ours has a place to be available.

fosterlynn commented 8 years ago

P.S. I'm sure there are other bugs there too that I just don't recognize. And I haven't even put it in the jsonld playground. If anyone spots one, let me know. But where I want to go with it in terms of other bugs is to complete the cycle, i.e. read it in to something. If holodex or someone else gets there, excellent! If not, I'll work on getting a read into NRP (another instance) working.

elf-pavlik commented 8 years ago

@fosterlynn I get 404 for all those:

I have a question into the rdflib support people, even though I suspect it is my lack of knowledge of how to contruct the rdf triples to make this. @elf-pavlik if you have a minute, and can easily write up triples that would make the above, that would be awesome.

do you mean to write it in Turtle instead of JSON-LD ? you can always use JSON-LD Playground to get N-Quads (on of lower tabs) which people who read Turtle also should undertand...

http://www.w3.org/TR/rdf11-new/#section-serializations rdf serializations

fosterlynn commented 8 years ago

@elf-pavlik

I got a response from rdflib support, so please ignore my triple request! Sorry if you spent some time on it! :frowning: I haven't had a chance to follow through and fix my code, am working on something for one of our networks, done soon I think.

fosterlynn commented 8 years ago

An update: The NRP jsonld output is now working technically, at last! It is not correct per our more recent discussions, like using only vf, questions of user defined relationship types, etc. I'll probably wait to fix it until we have some more decisions or we need to use it. In the meantime, if you are bored :) , you can see results here:

http://dhen.webfactional.com/accounting/agent-jsonld/ http://nrp.webfactional.com/accounting/agent-jsonld/

The last link is a site we re-purposed for the valueflows group. Right now it has only the data for the bob and lynn example. We were thinking this might be a fun place to store all of us and our relationships, and a new valueflows group agent. If anyone wants to enter themselves and their relationships into the database, we'll get you a login. Here is a page that shows the agent structure: http://nrp.webfactional.com/accounting/projects/. Or here: http://nrp.webfactional.com/accounting/agents/. If you're logged in, you can create agents on these.

fosterlynn commented 8 years ago

@all, I am thinking I will go ahead and write a query method in NRP to take in agent jsonld and make an ugly display of it. I feel a personal need to complete the round trip to better understand the implications we need to take into account for querying, and just how this stuff works technically. And I would like a working example. (Understanding it will need continual adjusting as we morph the vocab.)

However, this might take me a bit, big learning curve, so if anyone thinks I should be spending this time on something that seems to you more on the critical path, please feel free to let me know!

ahdinosaur commented 8 years ago

hungry for more agent jsonld? here's holodex/enspiral-data processed into the current format: github, raw, playground. :smiley:

fosterlynn commented 8 years ago

@ahdinosaur hungry, stomach growling.... :smile: .... However, my code doesn't work yet, so all in good time I guess.....

elf-pavlik commented 8 years ago

hungry for more agent jsonld? here's holodex/enspiral-data processed into the current format: github, raw, playground.

very cool @ahdinosaur !

few observations:

1) one can NOT traverse it (discover relationships) https://github.com/valueflows/valueflows/issues/59

eg. starting from HTTP GET to https://holodex.enspiral.com/api/agents/derek

response

{
  "@context": [
    "https://rawgit.com/valueflows/agent/master/context.jsonld",
    {
      "dex": "https://holodex.enspiral.com/api/"
    }
  ],
  "@id": "dex:agents/derek",
  "@type": "Person",
  "displayName": "Derek Razo",
  "image": "https://lh4.googleusercontent.com/-eyaiiPvDQD4/AAAAAAAAAAI/AAAAAAAAAAA/6G9Kw5SoZ8A/photo.jpg"
}

2) use of relation property only in one way (which I like) https://github.com/valueflows/valueflows/issues/52 eg. "dex:relationshipTypes/has-member" & "dex:relationshipTypes/has-subgroup" not used :smile:

fosterlynn commented 8 years ago

Updated my data creation method. Can be tried here: http://dhen.webfactional.com/accounting/agent-jsonld/. (Note: order is confusing to a human.)

Also used it to update the bob and lynn example, which I re-ordered some.

fosterlynn commented 8 years ago

P.S. The @id links are not done yet.

P.P.S. If anyone has feedback on incorrect syntax, incorrect understanding of our vocab, etc., feel free! :smile:

elf-pavlik commented 8 years ago

Thank you @fosterlynn ! few comments below each snippet

{
            "@id": "http://dhen.webfactional.com/accounting/agent-relationship/125/",
            "@type": "Relationship",
            "object": "http://dhen.webfactional.com/accounting/agent/139/",
            "relationship": "http://dhen.webfactional.com/accounting/agent-relationship-type/isOwnerOf",
            "subject": "http://dhen.webfactional.com/accounting/agent/63/"
        }

isOwnerOf sounds evil between agents, if you consider Organization a sub type of a Group, than you might want to have distinction between the Group and the Brand (a resource) as @ahdinosaur has in https://github.com/valueflows/agent/blob/master/examples/enspiral.jsonld#L113-L117

Same goes to hasOwner

{
            "@id": "http://dhen.webfactional.com/accounting/agent-relationship-inv/83/",
            "@type": "Relationship",
            "object": "http://dhen.webfactional.com/accounting/agent/38/",
            "relationship": "http://dhen.webfactional.com/accounting/agent-relationship-type/hasHarvestingSite",
            "subject": "http://dhen.webfactional.com/accounting/agent/3/"
        }

I have impression of conflating an Agent (Group) with a Resource(Site), which schema.org does for example in schema:LocalBusiness and which personally I don't really like. It also looks similar to org:hasSite

Same goes to isHarvestingSiteFor, isStorageSiteFor, isDryingSiteFor - does not sound like Agent - Agent relationships!

One more suggestion, except my recommendation to try avoiding to conflate Agents with Resources:

{
  "@context": {
    "dhen": ""http://dhen.webfactional.com/accounting/"
  }
}

added to the context will reduce noise in flattened form of JSON-LD, in similar way as Mikey uses "dex": "https://holodex.enspiral.com/api/"

once again, glad to see all those concrete, real world examples in JSON-LD coming in :smiley:

fosterlynn commented 8 years ago

@elf-pavlik :+1: on adding dhen (or whatever nrp instance) to the context, I will do that. And thanks for looking at this in detail!

isOwnerOf sounds evil between agents, if you consider Organization a sub type of a Group, than you might want to have distinction between the Group and the Brand (a resource)

This actually clarifies why we need Organization, as subclass of Agent, not subclass of Group. (Or in the US we could subclass Person..... :tongue: referring to our horrible supreme court decision that corporations are people. What a country!)

So organizations have agency, and organizations can be owned by people - especially if they are legally incorporated for example. When I did IT contract work, I owned (100% of stock) my little consulting company, which was an S-corp. It was not me and was not a group. I was owner of it, and also was only employee of it. So until we are rid of such artifacts in the real world, I think we need them explicitly in the model. And now I am much more clear on why they can't be Groups.

Side note: I don't think we need the type of agent constraints on type of relationship in the vocab, but the way @ahdinosaur originally proposed would be able to constrain such things, so for example people could not own people.

I have impression of conflating an Agent (Group) with a Resource(Site)

Agree this is a danger, and we need to be careful of our modeling in this area. There are different aspects of a farm, but what we need for the farm to engage in economic activity is the farm as an Organization, or could be subclass of Organization. They often are formal organizations, but in any case, they have agency different from the people's individual agency. They also have relationships with lots of resources, including the land. They are also located at one or more locations. But all of that is part of different use cases.

For example: If someone has a farm, and they buy the neighbor's property to make their farm larger, it is still the same exact agent. If they also move into their neighbor's house because it is nicer, the farm is still the same exact agent. And the farm will most likely file different income taxes than the owner.

The use of the term "site" is confusing; and it is possible the admin people in DHEN are confused, but I don't think so. They are just not ontologists, they are herbalists. It is also possible that the harvesting sites should not always be agents in the DHEN situation (they are not always farms for example, sometimes they are publicly owned parks), which is a much more complex discussion, and would just remove them from this example, not change the essence of the example, where the flow would then just start with the harvester, not the harvesting site. The drying sites are definitely agents, and they become owners of the herbs while they are drying.

Note the harvesters are people: Megan isHarvesterFor DHEN. Just means they are trained, and can be called on to harvest when the need arises.

elf-pavlik commented 8 years ago

So organizations have agency, and organizations can be owned by people - especially if they are legally incorporated for example.

I know that we commonly may tend to explain things this way, I still think that only a Person (Individual) or Group (Collective) have agency, leaving Bot/BotNet out of conversation for now. When we discuss Organization, we may see it as Group of people, who own a brand and various other resources (inventory?). But those people as individuals or as a group have agency, not some umbrella construct - Organization.

So on a vocabulary level, we most likely shouldn't restrict constructs like Person owns a Person, even that we would never use it ourselves (I don't even say that someone owns a pet but 'takes care' of a pet!). At the same time we could do our best to present what we can agree as 'best practices' and explain possible consequences of using other patterns.

How does it sound Organization - role:president -> Person ?

If you say that Person 'owns' organization, and organization 'has' members it gets supper confusing as far as I can tell. By saying Person 'owns' organization, what exactly do you mean that this Person owns? Members, inventory, brand, data, ... ?

fosterlynn commented 8 years ago

Other examples:

Sensorica created an organization (a legal non-profit) called CAKE-ACES that they needed because they need a bank account, liability coverage for their products, etc. CAKE needs and has agency.

Bob and I live in a small co-housing group that we decided to make a legal cooperative because we needed to have protection for our personal assets if for example someone has a lot of medical expenses and the state tries to go after the property. Canyon Homestead Cooperative needs and has agency.

Now, both of these are only needed because of the system we live in. But they are also not part of the evil empire. I think we need to support the reality of our transitional times, and that means whatever organizations beyond group-ness that people need. And definitely what the people we collaborate with need.

By saying Person 'owns' organization, what exactly do you mean that this Person owns? Members, inventory, brand, data, ... ?

I think this type of organization owns the inventory, brand, etc. (not the people obviously). People might own the organization as a legal entity.

ahdinosaur commented 8 years ago

I think we need to support the reality of our transitional times, and that means whatever organizations beyond group-ness that people need. And definitely what the people we collaborate with need.

this. we need to support whatever economic organizing system someone wants to model, whether it be traditional or alternative or anywhere in between.

as @elf-pavlik mentioned, @joshuavial at Enspiral is championing the idea of brands, legal entities, bank accounts, etc as resources rather than organizations, so i modelled my example as such, but that doesn't mean we shouldn't support more traditional organizational models (e.g. how @fosterlynn modelled the DHEN, how most people would model Enspiral, etc).

elf-pavlik commented 8 years ago

I think discussion in #54 can possibly help us with modeling various complexities captured here.

elf-pavlik commented 8 years ago

@fosterlynn what do you think, that for now we would conflate aspects of Agent and Resource on the level of instance. So instead defining class like http://schema.org/LocalBusiness - defined as sub class of both, Organization and Place

Each instance can do something similar by explicit assertion "@type": [ "vf:Agent", "vf:Resource"] which means that one can consider it both - an Agent and a Resource. Later we should think more about possibility of introducing classes which do that, but maybe this way we could postpone that work at least for little longer, while whoever needs to do it can already use this approach.

fosterlynn commented 8 years ago

@elf-pavlik @ahdinosaur

what do you think, that for now we would conflate aspects of Agent and Resource on the level of instance.

Is there something we are actually confronting now that we need to do that for? If not, it can be a very good discussion to continue to have in the background. If so, sure, let's give it a try.

championing the idea of brands, legal entities, bank accounts, etc as resources rather than organizations

I think we should be careful and try to work out our definitions. Initial thoughts: In this list, I would think that brand and bank account are only resources, neither one can have agency, true? Legal entity needs to be an agent because in our world it has agency, and both in the traditional and the alternative worlds, of necessity. A legal entity can own or manage or whatever a brand or a bank account. I can see that one could also consider a legal entity as a resource, as in some person can own or manage or sell it, etc. It is an ambiguous and complex construct embedded in private property systems.

bhaugen commented 8 years ago

@ahdinosaur

@joshuavial at Enspiral is championing the idea of brands, legal entities, bank accounts, etc as resources rather than organizations

I agree with Joshua on most of those concepts, except for legal entities. But even that is an interesting idea. You can buy and sell legal entities.

bhaugen commented 8 years ago

@fosterlynn - can we differentiate legal entity as the legal charter from organization as something with agency?

We know of organizations with agency that do not have legal charters.

Examples:

fosterlynn commented 8 years ago

can we differentiate legal entity as the legal charter from organization as something with agency?

@bhaugen possibly something like that is valid. But when you sell a legal entity, you are selling all of the assets (resources) of that organization in one swoop. Can one sell an organization without a legal charter? Don't know. Anyhow, food for future thought.

bhaugen commented 8 years ago

Can one sell an organization without a legal charter?

No.

fosterlynn commented 8 years ago

Another update on my jsonld feed from NRP: I have updated it for all the latest decisions and suggestions. Also made it so all the @id links return valid jsonld. Of course I don't know if all of this is correct, it is just visually checked, not having anything yet to read it in.

So, the following should work: http://dhen.webfactional.com/accounting/agent-jsonld/ (herbal network) http://nrp.webfactional.com/accounting/agent-jsonld/ (Bob/Lynn sample data)

Also I deployed for the first time to Sensorica, so this should also work, if you are a glutton for a lot of confusing data: http://nrp.sensorica.co/accounting/agent-jsonld/

To see the Sensorica data in an easier way, you can look at http://nrp.sensorica.co/accounting/projects/. Not beautiful like Holodex, but it's what we've got..... :smile: .

Note this also works for the other sites above, just substitute "projects" for "agent-jsonld".

One thing I don't have: Bringing back individual agent data, was thinking to include all the agent relationships. I'll do that next. This will give data to follow the nose....

Another note on the Bob/Lynn sample site: If anyone wants to play around with entering their own data into the system, so that it makes some jsonld, let me know and I'll set you up with a login and a little training, or can enter it myself for you.

ahdinosaur commented 8 years ago

Is there something we are actually confronting now that we need to do that for?

what you have now is fine by me, this is better as a background discussion.

elf-pavlik commented 8 years ago

@fosterlynn awesome !!!

fosterlynn commented 8 years ago

I pulled out some examples from this, added to the repo, and added to the md file.

fosterlynn commented 8 years ago

Oh forgot, I added relationships to the agent specific links. Hoping this facilitates following the nose. Next, moving back to the get/parse, to see if I can read any of this stuff.....

fosterlynn commented 8 years ago

Another update which I am very happy with. :smile: Now able to parse the data coming out of one nrp instance to another, at least into rdf triples. See http://nrp.webfactional.com/accounting/agent-jsonld-query/, which reads data from the dhen instance and displays the triples.

Next: Get that into django objects and display an ugly list from that. I'll consider this a meaningful round trip, but still pretty specific to NRP data. (Like, we don't have images, other things probably missing and not tested by this.) And I haven't tried it with the specific @id jsonld feeds.

Looking forward to what you guys come up with!

fosterlynn commented 8 years ago

One more update: @bhaugen took the data out of the graph into python and is displaying the minimal ugly page. :smile: So I'm happy that there is a round trip and that there is some level of verification of the feed. I don't think the context actually got verified, because there is no sparql or anything that needed to use the context definition. Anyway, here it is, reads from nrp instance into dhen instance:

http://dhen.webfactional.com/accounting/agent-jsonld-query/

I'm going to move on from this experiment for now.

bhaugen commented 8 years ago

I'm going to move on from this experiment for now.

Not yet, for me. I got some other things to do first, but then I want to return to this and make objects out of it.

almereyda commented 4 years ago

We have moved the ValueFlows organization from GitHub to https://lab.allmende.io/valueflows.

This issue has been closed here, and all further discussion on this issue can be done at

https://lab.allmende.io/valueflows/agent/-/issues/20.

If you have not done so, you are very welcome to register at https://lab.allmende.io and join the ValueFlows organization there.