Areas of contribution

[Logic-gate]

What are some of the areas where users can contribute to keypeer at this early stage in the project?

[poetaster]

• Ideas for apis that apps use or might use.
• Abstract api design idea
• A logo
• Contacts to people in the SFOS community and outside who might have an interest.

[poetaster]

If you are a German speaker, feedback on the prototype.md application are appreciated.

[Logic-gate]

Unfortunately I am not a German speaker. But I do have some experience constructing API's, both professionally in my former employment where we worked on exposing Odoo's API's to our own and privately through personal projects like (https://ki.tc). I am preparing a response to share here.

[poetaster]

I am preparing a response to share here.

Great! It's been a couple of years since I did API design, so input is very welcome!

[Logic-gate]

This is a basic API implementation scheme based on the notes in the project's page. It would be useful to push it as a file to allow for further amendments and contributions.

The idea behind {version} is allow for updates to the system without affecting the current implementation.

There are a few obfuscation techniques to circumvent interception of key data. These are not necessary and will only add computational overhead.

I had the idea of exposing {provider_id}'s API through keypeer for certain services. Something to the effect of /api/{version}/{provider_id}/{exposed_api} which would allow for app requests through keypeer to the {provider_id}. If this is to be implemented, I would argue that it should remain within generic API services the likes of News. Although this would add complexity and therefore unnecessarily bloat the project given the required bandwidth and backend capabilities to achieve it.

Node	Description	Method	Response
/api/{version}/generate	It should allow for multiple {provider_id} in a singular request.	GET	{TOKEN}
/api/{version}/submit	App submission	POST	{random_id_of_length_256}
/api/{version}/request/{provider_id}	API key request given status response from submit node	GET	{TOKEN ⊕ instance_id}
/api/{version}/status}	Generic Status Node, KeyPeer should be included in {provider_id}	GET	BOOLEAN

---

/api/{version}/generate GET

{
  "provider_id": ["news_service_id", "map_service_id"],
  "datetime": "DATETIME",
  "app": "app_name"
  "...": "...",
}

/api/{version}/generate Response

{
  "_id": "SHA256(SHA256(app_name)+salt)",
  "uid" : "random_bits_of_length_XYZ",
  "datetime": "DATETIME",
  "...": "...",
}

---

/api/{version}/submit POST

{
  "instance_id": "SHA256((_id+random_bits_of_length_XYZ))",
  "...": "...",
}

/api/{version}/submit Response

{
  "token": "TOKEN ⊕ instance_id",
  "datetime": "DATETIME",
  "...": "..."
}

---

/api/{version}/request/{provider_id} GET

{
  "token": "TOKEN ⊕ instance_id",
  "...": "...",
}

/api/{version}/request/{provider_id} Response

  "api_key": "api_key ⊕ TOKEN",
  "...": "...",

---

/api/{version}/status GET

  "provider_id": ["news_service_id", "map_service_id"],
  "...": "...",

/api/{version}/status Response

  "status": "BOOLEAN",
  "...": "...",

[Logic-gate]

I would also suggest starting work on the "Keypeer paper". Since Keypeer is a novel idea, I would assume that some IEEE publication would accept it even if it was not academic in nature.

[poetaster]

Thanks! One question, why not rest conventions and HTTP methods like PUT/DELETE (REST style)?

[poetaster]

I would also suggest starting work on the "Keypeer paper". Since Keypeer is a novel idea, I would assume that some IEEE publication would accept it even if it was not academic in nature.

I've had a conversation with rubdos that might take us in that direction but for one or more of his postdoc students. I'll try to follow up...

[Logic-gate]

We can use PUT to update existing records, but it is frowned upon to use it instead of POST for the creation of ones. Take the following example from my scheme:

PUT /api/{version}/submit; "instance_id": "SHA256((_id+random_bits_of_length_XYZ))", --> updates a token for example. POST /api/{version}/submit; "instance_id": "SHA256((_id+random_bits_of_length_XYZ))", --> creates a token for example.

DELETE could also be used, however I didnt want to overstep the scheme in a direction not stated in the workflow.

We should eventually use all methods: GET, POST, PUT, PATCH, DELETE. It just depends on what the direction is and what kind of capabilities are to be given to developers.

Also, the API scheme is by no means final nor should it be. It would be useful to have it either in a gist or as part of the project to allow for feedback and updates.

As for the paper; yes, I think it would be extremely beneficial to the project since it is a truly a novel idea. If no one steps up, we can start work on it maybe on overleaf to allow for collaborations. I am not an academic, but I have worked on various academic papers in various capacities(mostly editing); here's my latest project(darft) https://github.com/Logic-gate/Tidy/blob/main/Tidy-Draft.pdf I am just waiting for HPC lab time approval. I could probably assist in writing some sections of the paper. We can then pivot on rubdos as an advisor on the paper given his credentials and expertise. After asking him of course.

[Logic-gate]

It really depends on what the backend looks like; if you want you can have a PUT request delete a record and vice-versa. In flask for example:

@app.route('/submit', methods = ['PUT', 'DELETE'])
def submit():
       if request.method == 'PUT':
              DETELE_SOMETHING()
       if request.method == 'DELETE':
              CREATE_SOMETHING()

[poetaster]

Ok, so what would make sense is to create a document root for api design and, perhaps, an initial flask/werkzeug src tree?

Monorepo? Discrete?

I haven used flask for ages, but always like using it (built some blueprint apps).

I'd suggest we split out the the paper question into a new issue?

[Logic-gate]

an initial flask/werkzeug src tree?

Yeah. It would make sense since the project is still in it's infancy code-wise. It would allow us to toy with different concepts pragmatically. Not just ideas being conveyed but also a prototype-ish code base to base the actual project upon.

Monorepo? Discrete?

I would opt for Monorepo since it's better suited for scalability.

I'd suggest we split out the the paper question into a new issue?

Yes. Makes sense.

There is also the matter of DB, the age old SQL vs no-SQL.

[poetaster]

I've invited you to be a contributor for the repo and created /doc/api.md from your sketch above.

My older flask projects where just running using :

#!/usr/bin/python
from flup.server.fcgi import WSGIServer

I was using flask-marshmallow and marshmallow-sqlalchemy , DB marshalled to JSON, json directly rendered to forms dynamically via JS. Just looking at some code now, I'd need to look a more modern (heh!) approach.

[Logic-gate]

Cheers, will add/remove to the api doc.

Flask has come a long way. I've been told that Cherrypy too has come a long way, however I have never used it.

I think it best to go for a framework/DB that the core dev team is comfortable with regardless of the apparent benefits of other frameworks.

[Logic-gate]

In regards to #2 and rinigus's last two posts. It seems that my API proposal is somewhat wrong. If the intent is to have a singular API key for all available services, then I will need to change that. Also, after thinking about it, there will be no need for data obfuscation. I would appreciate input from @rinigus in regards to the scheme.

[poetaster]

If the intent is to have a singular API key for all available services

That is the intention, but may not be the reality. There may be a base case but I'm guessing that modalities like time (different billing schemes) may make it necessary to have more than one. I'll ping rinigus ! Thanks

[rinigus]

Sorry for taking time to reply. Was dealing with other projects and will have to catch up with this thread. Will try to read it tonight and reply tonight or tomorrow.

[rinigus]

Looked into the proposed API and discussion around it. Thank you very much for working on it!

Re version: yes, that's a great idea to tag it with the version.

Re use of keypeer as a proxy: maybe, in far future. It would require making sure that it is in agreement with the provider privacy policy and could mean that we need to ensure that nothing will leak.

Re /submit endpoint: I would suggest to drop it and use TOKEN for user identification. We don't really need to register apps as such

Re /generate: I presume that is for "Subscription" and generation in https://keypeer.org/application-workflow.html . Makes sense to call it "generate". According to that workflow, 2 keys have to be returned: API key and TOKEN for payments . Proposed /generate response seems like an overkill to me.

Re /request/{provider_id}: just API key from app-workflow would be sufficient. It should also return datetime of service key expiry

Re /status: not sure what is that for. Is it for checking if its all OK?

Would be good to provide users a way to check their balance. First as an overall sum and later maybe with the details on how it was used.

[Logic-gate]

@rinigus Thank you for the feedback;

I have amended the proposal to reflect your notes. I will push it once approved:

I couldn't find any threads or mention of the token type. I am assuming JWT will be used.

Term	Definition
provider_id	Internal ID for available services
token	Also refereed to as payment_token, used to label payment. Token is of JWT type?
keypeer_api_key	Keypeer API key; returned from /api/{version}/generate
service_api_key	provider_id API key.

---

Node	Description	Method	Response
/api/{version}/generate	It should allow for multiple {provider_id} in a singular request.	GET	{TOKEN}, {keypeer_api_key}, datetime
/api/{version}/request/{provider_id}	Request for service API key	GET	{service_api_key}, expiry
/api/{version}/status	Checks balance	GET	balance

---

/api/{version}/generate GET

{
  "provider_id": ["news_service_id", "map_service_id"]
}

/api/{version}/generate Response

{
  "token": "{payment_token}",
  "keypeer_api_key" : "{keypeer_api_key}",
  "datetime": "DATETIME",
  "...": "...",
}

---

/api/{version}/request/{provider_id} GET

{
  "keypeer_api_key": "{keypeer_api_key}"
}

/api/{version}/request/{provider_id} Response

  "service_api_key": "{service_api_key}",
  "expiry": "DATETIME"

---

/api/{version}/status GET

  "keypeer_api_key": "{keypeer_api_key}"
  "...": "...",

/api/{version}/status Response

  "balance": "{float}",
  "...": "...",

---

Annotated flowchart based on https://keypeer.org/introduction.html

[poetaster]

Sorry to be a bit late in replying.

I couldn't find any threads or mention of the token type. I am assuming JWT will be used.

I'm not a fan of JWT. But that's a point for further discussion.

If you go REST style, then we should avoid VERBs in the api. The HTTP methods suffice, and it is the bits of the model or models plural that should appear in the api.

/api/{version}/generate 

becomes

/api/{version}/provider

Where generate becomes something like:

/api/{version}/admin/provider

And the public facing (other authentication path) provides keys for api service end points.

I'm not certain provider generation is even required in the api context, although even administrative functions should have an api. I'd decouple admin from consumer. The endpoints that come to mind:

• /api/{version}/provider (keys)
• /api/{version}/application (app <-> provider mapping)
• /api/{version}/consumer (user ID/token, payment status, etc).

[poetaster]

Would be good to provide users a way to check their balance. First as an overall sum and later maybe with the details on how it was used.

Perhaps another argument for splitting the administrative api apart from the key specific?

[Logic-gate]

Mark, thank you for the feedback.

You are correct about the naming convention. I however thought it best to keep it close to the action at this early draft stage for clarity. But to avoid confusion later on, I will amend it.

Never really thought about it in terms of admin and consumer. Don't know how I missed it.

Should we go with specific paths/endpoints to differentiate between admin and consumer or handle it in the backend based on db user type lookup?

Meaning that if you are an admin, the api call remains the same, however the return differs.

[poetaster]

Mark, thank you for the feedback.

You are correct about the naming convention. I however thought it best to keep it close to the action at this early draft stage for clarity. But to avoid confusion later on, I will amend it.

I've started on a document in the 'doc' directory. api-rest.md. I find it a bit more useful to work on an actual document with revisions. Could we move to that?

Never really thought about it in terms of admin and consumer. Don't know how I missed it.

It's a very common pattern. If one looks at elaborate apis, for instance for matrix server, one inevitably has separate routing: https://spec.matrix.org/v1.6/client-server-api/ for example compared with https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/

Should we go with specific paths/endpoints to differentiate between admin and consumer or handle it in the backend based on db user type lookup?

The matrix folk distinguish between matrix protocol consumers (chat clients / servers) and server implementations: /_synapse/ is the endpoint on a synapse server for admin while /_matrix/ the endpoint for client and server to server message communication. we could have something similar, perhaps /keys/ and /admin/ or the like.

Meaning that if you are an admin, the api call remains the same, however the return differs.

Both are top level namespaces. I'm sketching a bit in the document I mentioned above.

[poetaster]

Things to keep in mind that I've started to flesh out in the api-rest.md ...

• many to one consumer -> application
• many to one application -> provider

For accounting we are going to need a relation of consumer -> application -> provider to get enough granularity for usage info.

My initial thought was that we eyeball a payment per provider and time period and just check consumer/application access time compared with time of payment. But that's pretty vague.

[poetaster]

Ok, sorry for all the info, but.... I'm inclined to do this, api specification first like with https://connexion.readthedocs.io/en/latest/

In a flask context: connexion[swagger-ui] is used to automatically getting routing all HTTP methods and field level validation. And swagger ui for examing.

Since you can use swagger.yaml or openAPI you can move to specifying the apis directly and turn that into functional code in very little time. The point being, would spare us writing documents in markup. Generating docs is, for free.

[Logic-gate]

Agreed. It will make things quite simpler. I started work on a mock api on my fork(https://github.com/Logic-gate/keypeer.org/commit/f762bce96e9b5df160f428a85784d60219dfbb90) I will push it once I am happy with it. I have a few RL things that I need to attend to for the next two days or so. Will re-visit it and continue work after that.

[poetaster]

Agreed. It will make things quite simpler. I started work on a mock api on my fork(Logic-gate@f762bce) I will push it once I am happy with it. I have a few RL things that I need to attend to for the next two days or so. Will re-visit it and continue work after that.

Ah, cool! I wanted to start on this, but am still working on the grant application. I've mentioned going with an OpenAPI approach and started playing with the combination of flash and connexion. It's nifty.

[Logic-gate]

Created a pull request(https://github.com/poetaster/keypeer.org/pull/17). I should preface it by stating that at times I had to scratch my head, maybe I am slow, but it seems to me that's it's a tad convoluted. Maybe I am wrong.

[Logic-gate]

Cheers for push.

Just a heads up, I used https://editor-next.swagger.io to render it. Might be useful.

[rinigus]

Sorry for being late to reply to many aspects. Was/still am swamped with SFOS projects.

Where could I comment on API? In the docs we still have application_id running through logic. I would suggest to remove it as it is not really needed.

Which of the APIs should be considered - api-rest or just api.md?

From the discussion above, looks like https://github.com/poetaster/keypeer.org/issues/15#issuecomment-1467826101 was closest to earlier discussion of API. I don't follow admin/consumer split as it is a consumer that would request the payment and auth token that (s)he will use for service access. Again, I don't think we need any application-specific tokens, just for services.

Hopefully this above makes sense :) .

[poetaster]

Which of the APIs should be considered - api-rest or just api.md?

Right now stuff is happening in the api.yaml file, but I'd wait a day or two and I'll set up an action to make that readable.

The consumer consumes keys, producers (api enpoints) produce em. The admin idea is for purely administrative stuff like adding providers.

The application specific thing is for keeping track, for instance, which app for provider X has more requests. We can abstract that from the 'consumer' (person) and still know which apps consume for the same api. That kind of thought.

[rinigus]

Will wait :)

[poetaster]

Quick note: I'm reworking what we have so far and have set up a 'mock server'. When it's running (next day or two) it'll be on api.keypeer.org ... just running swagger auto generated flask app. Please give me some time (I'm on the poetaster-api branch) before going back to the api.yml.

[poetaster]

I've invited all here to the api.keypeer.org repo. It's the current state of what's live at api.keypeer.org/v1.0/ui

[poetaster]

@Logic-gate in the root of the new repo is a file keypeer.yaml I'm editing that with the swagger editor and dumping the generated code directly into the root of the repo. That's 'fast and dirty' but makes it less complicated to deploy. The main.py in the repo will not work since it sets a host ip. But, that get's re-written when you generate new code and will then work.