Port to Atomese API

[linas]

Change request, per discussions with @edyirdaw in pull req #6

Get rid of the RESTful API and port to Atomese. If you don't understand why, let me know, I can explain. If you don't understand how, I can explain that too.

There are three ways to port to Atomese.

• One is to open a connection to the cogserver, and just send whatever scheme you want, and parse the replies. This allows you to do pretty much anything at all. If you open the connection in "quiet" mode, then parsing the results will be easier, as you won't get the extra markup to make it human-readable. This requires you to write a brand-new parser, in js, for Atomese. This would be all-new code, no one has done this before.

• A second option is to do the above, but limit yourself to the StorageNode API. This is almost the same as above, except that it's faster, tailored for good network performance and ... prints no error messages, making it harder to debug. See https://wiki.opencog.org/w/StorageNode half-way down the page.

• Either of the above would be written in js, and, because its generic, should probably be a brand-new js project, so that others can use it (and not just the explorer) I can create a git repo for you, to hold this new code.

• Write a brand-new cogserver plugin to return json. This would replace the RESTful thing. The returned json would need to be designed to fix all the mistakes in the RESTful API. This might be easier to do (??) than the first two options above. The brand-new code would be in C++. I don't really recommend this option, but ... it wouldn't be that bad. The API would be the same StorageNode API, except sending and receiving json instead of s-expressions.

So basically, if you want to write js, pick the first few options; if you want to write C++, pick the last option. I can help with any of these.

[chris-altamimi]

Hello Linas!

We're very interested in helping this with this. Our extensive experience in Javascript / Typescript with both Restful APIs as well as React based UIs should allow us to knock this out easily using option 1 or 2.

@mforrest20 and I will likely get started on this later this week or early next week.

Edit:

What do you suppose is the better of the two options? I can recognize some of the benefits that one brings over the other such as option 2 having better maintainability and ease-of-development.

However, I would love to know what you would choose if you had the resources to accomplish any of these options?

[mjsduncan]

i'm not linus but i'm eager to use a good graphical interface to the atomspace! one possibility to consider is porting the metagraph rendering code here to this visualizer, which has a good interface (using option 1 i think) but only visualizes the scheme s-expressions.

there's also a guile module to parse json for options 1 & 2

[linas]

Hi Chris!

That would be great! I'm not sure what I need to say to get things moving. Do you want to have a long conversation here in github comments? Or maybe chat ... there's an opencog server in discord, I sort-of-ish glance at it every now and then. The task is "easy" in that I know exactly what to do; it's hard only in that I would have to go over the details, so that you don't end up wasting time & breaking your head on dead-ends.

[linas]

Oh, to answer your question:

I would love to know what you would choose if you had the resources to accomplish any of these options?

I'm neutral. Here are some pros-n-cons:

• Option 3 is to write a cogserver plugin (in C++) to send-receive json. This allows "anyone" intersted in json to use it. I could even volunteer to write the code myself: a day or two to get the basics working, a day or two to polish it up, make it pretty, a day or two to write the docs. My only complaint is that this is about 5 days longer than I can easily spare...
• Option 1 & 2 are "almost identical", require no new C++ code, no effort from me to make anything work. Mostly, I'd have to make sure that you clearly understand what the AtomSpace actually is, so that you know what to do with the returned data. Well, option 3 requires this as well: if you don't understand what the AtomSpace is, then it's ... tough going.

The key concepts are "what is an Atom" "what is a Value" "what is the incoming set" and "what is the outgoing set". Wrap your mind around those, I think you'll be OK ...

[Ontopic]

Would love to have a go at exposing interacting with Atomspace through different web-based interfaces. A basic C++ implementation (Docker would be extra nice) for a JSON send-receive would I feel open up the possibilities of exploring the OpenCog universe to quite a few interested developers to build cool stuff on, not just me.

[chris-altamimi]

I am a fan of the C++ implementation only because we can rely natively on the C++ api rather than having to develop a completely separate parser for the atomspace. I would expect this to be relatively more cumbersome and require extra work to maintain if the atomspace is extended or edited.

However, at the same time, I feel that the JS approach would open things up to UI focused developers since they tend to favor JS/TS ( React, Angular, Vue etc etc ), which I feel could very likely make a much more elegant and user friendly visualizer if they have a javascript API to work with.

On top of this, I think that there would be great benefit to having an Atomese Parser outside of the visualizer such as analytics outside visualization.

@Ontopic are you saying you could likely pull something together in C++ fairly quickly? If this is the case then I don't want to step on any toes, please feel free to take over. As we've just now started to piece together a js approach.

Ultimately, I'd rather approach with the best solution for the community.

[chris-altamimi]

Also,

@linas We will join the discord server, however I think a conversation at this point might be premature and result in wasting your time, which I am keen to avoid, given your noted lack of time to spare.

I truly would like to hear everyone's opinions regarding the best approach.

While I feel I have the expertise to tackle this issue ( heavy experience in both C++ and JS ), I think the experience using OpenCog is invaluable, and thus I am all ears to those with more of it than I.

As soon as we land on a choice, then I feel we would be ready to have a discussion on your requirements, constraints and anything else you wish to share about the matter at hand.

[linas]

I could knock out a basic cogserver plugin (written in c++) in about a day or two, if you're serious about using it and ask me nicely. You would use it by opening a socket to the cogserver, sending any one of about a dozen rather simple JSON snippets, and then getting back a (usually large) answer, in JSON.

I'm offering to do this mostly because its a cut-n-paste of an existing plugin, which changeups for handling json instead of s-expressions. If course, you could do this cut-n-paste job too. It's not hard.

I've failed to look at discord for a few weeks. Ooops! Looking now.

[Ontopic]

My C++ skills are sadly limited, especially in an unfamiliar codebase. I whole heartedly love all OpenCog is doing and would love to use it in an interactive way. I'm sure there's a lot more people that would be able to grasp the concepts of Atomspace when given a proper in browser visualizer / editor (repl) in a modern UI.

If this JSON interface would be here, I'd be building an editor and Atomspace playgrounds right now

[Ontopic]

So, kind sir linas, if you might find yourself in a position with some time to throw at this, and in the mood to do so, I hope you do! Thanks for all your efforts anyhow. Was following the issue in the REST repo, I feel it was a little unfair how you were treated, still, with this out of the way, next time you can say "you can implement the REST yourself, based on the JSON interface" ;)

[linas]

Huh. Well, looking around, it seems that a bare-bones json interface already exists. See the README here: https://github.com/opencog/atomspace/tree/master/opencog/persist/json It's maybe enough to get you started. I will poke around a bit today, and try to add some of the missing features.

More generally, this is all kind of useless, unless you have some specific dataset you want to work with. I can quickly and easily offer two: one is a gene+protein dataset, another is a language dataset. You can run them locally, or I can set up a remote server running these.

[linas]

Added more stuff w/ pull req https://github.com/opencog/atomspace/pull/2952

Notes:

• The API is pseudo-JSON -- if it looks weird or isn't pure-json-enough or is otherwise broken, open a bug report. (or send a pull req)
• There might be bugs. I hand-tested case by case but was too lazy to write unit tests. Yes, skipping unit tests is bad coding practice. Today, I made an exception.

[linas]

I'll give write permission to https://github.com/opencog/atomspace-js to anyone who wants to create javascript wrappers for this thing. ... or anything else.

[Ontopic]

Thank you! Will see what I can do about those wrappers.

Apart from exposing the JSON to some JS apps (node deno, vanilla, React Typescript, Vue et cetera), I've been meaning to do a WASM (AssemblyScript) / Neon (Rust for Node, https://neon-bindings.com/) implementation, that might (although for now inefficient with the JSON interface) perhaps be a good starting point later on to create "real bindings" to JS, that can truly integrate with the backend (at which point I am gonna need help again), but at least the "exposed" functionality will be clear. So hopefully will be easier for someone with knowledge of the OpenCog codebase to integrate smoothly. Perhaps looking into running the Python bindings in PyScript (yes, Python in your browser is also very real) might also be interesting. Switching the storage to sqlite probably one of the first things I'll look into once I get closer to the "core" ;)

[Ontopic]

I might have overlooked it during my quick scan, but did you get a chance to put any of the datasets up somewhere? To me the language dataset would be the most interesting.

[Ontopic]

While I'm at it anyway; what is currently the recommended method of hosting the server? I saw some instructions on https://github.com/opencog/atomspace/blob/master/opencog/persist/json/README.md#network-api But the amount of different (outdated) Docker repos and install instructions with a certain order is starting to get a little confusing. What would be the quickest way to get a backend with JSON api rolling?

I feel like such a spoiled webdeveloper right now 😅 Once the setup is clear to me I'll contribute a docker-compose to start things together with the JSON API

[Ontopic]

And about those unit tests; the "frontend" will test against the JSON API, so that should also be a good point where we could connect the "backed" and "frontend" in the future, making not writing tests yourself in this case an efficient exception. What working functionality I can extract from your backend, I'll put in tests, functionalities I wish I'd had, would be broken tests that hopefully someone could assist me with at some point ;)

I consider this a good starting point for bringing Atomspace closer to the browser and an interactive experience, of course far from perfect for now. Really hope I can assist with my type of knowledge to get things there, just need some help getting things clear on the required backend for now. Thanks, truly, again for the effort that was put in already.

[Ontopic]

TLDR;

1. Was there a dataset made available for testing? Language one would have my preference
2. Is there a good "starting" setup for an OpenCog server right now, that would work correctly with the JSON-api and expose all of its available integrations? Preferably Docker-based, but I will bring things together in a docker-compose setup myself anyway, including storage [Postress vs RocksDB?] and other services needed to quickly get a JSON-api exposed, so I might as well write clean Dockerfiles myself with the instructions from the (right) set of repos that are needed
3. Thank you and the OpenCog contributors

[linas]

But the amount of different (outdated) Docker repos and install instructions with a certain order is starting to get a little confusing

I only know of just one docker repo, and yes, it is outdated and unmaintained.

Where are you finding install instructions (that seem confusing or outdated)?

The process shouldn't be that hard: install the atomspace, and then install the cogserver, and I think that's it. Then run the json instructions.

I'm trying to figure out where the datasets are now.

[linas]

There's a language dataset at r13-all-in-one.tar.bz2

[Ontopic]

Alright, the docker repo seems so outdated and geared toward specific (abandoned) projects that I'm not sure anymore what is really required, and what was once useful and what is now worth to perhaps add. Then the "direct install" assumes knowledge on things as e.g. guile setup. F.e. a question for me, would it be worthwhile to include Moses into a docker-compose when it comes to the JSON-api? I'm sure I'll figure these things out along the way, and your confirmation on "atomspace, then cogserver" is already helping me a lot on feeling I'm on the right track, yet, as said, it's far from a quick-start. So any other "pro-tips" as a rough guidance on where to look and what to keep in mind are very much appreciated. All I wanted to convey ;)

Hoping to contribute my efforts into making that quick-start for other newbies though, so why I'm somewhat allowing myself to ask stupid questions to hopefully get things right. I'll post the docker-compose with JSON-interface integration with basic frontend integrations here for early review once I have something. Is there a preferred platform / channel to discuss any issues / demos along the way?

[Ontopic]

Perhaps just archiving repos that are no longer in use and mentioning DEPRECATED on their wiki pages could really help others getting started quicker a lot aready? Currently it is unclear to me which pages / repos are essential to the foundation of OpenCog and which are forgotten experiments. Perhaps a docker-core repo? To get a quick repl and some sample commands to load in the all-in-one dataset quickly. The try-at-home and move-to-production-later starter

[linas]

There is a learning curve. This isn't a chatbot, where you open a window and chat to it.

Docker is a pain in the neck, LXC is a zillion times better.

MOSES is NOT needed.

For general use, you'll want to have the RocksDB backend atomspace-rocks

For language, you'll also need lg-atomese

For bio data, agi-bio

There's several steps to load the datasets (I have some automation scripts for this) and then there's even more to explain what the heck is in them. I can provide private access to a running server, with everything preloaded, if you email me.

[Ontopic]

I know it isn't a chatbot 😅 I know it's a superior system for modelling knowledge. Why I mentioned "I feel like such a webdeveloper right now". Implying; I expect an npm install and npm run all-the-things with 100's of examples of different implementations.

In the consumer market "knowledge-bases" are a hot topic though. I feel OpenCog is in the right position, with some extra tooling, to simplify entry into being able to contribute "knowledge" into a solid structure [with the contribution of other "AI's"], like OpenCog offers, while working on their personal knowledge base and a shared one.

[Ontopic]

Oh, and thanks again! I'll e-mail you for now, to see where you'd like me to post in the future.

[linas]

Perhaps just archiving repos that are no longer in use and mentioning DEPRECATED on their wiki pages could really help others getting started quicker a lot aready?

This is currently the case. There are several dozen repos in "active" use, depending on the faction who is interested in and using them. Despite this, not all of these are maintained.

Currently it is unclear to me which pages / repos are essential to the foundation of OpenCog and which are forgotten experiments.

Only one is central: the atomsspace. However, you need another half-dozen before you get sometihing useful. Which half-dozen depends on the specific project. There are/have been projects in biology robotics, perception, game-playing, 3D virtual worlds, language learning, chatbots, visualizers and more. I can't maintain all of them. I only maintain the stuff that I currently need and use.

Perhaps a docker-core repo?

Someone would need to volunteer to do that.

To get a quick repl and some sample commands to load in the all-in-one dataset quickly.

No. Go through the tutorials first. After that, we can have meaningful discussions about what's in those datasets. The data is complex and structured. For the language data, you have to be interested in linguistics, and grasp some of the basis of grammar, before any of it will make sense. That's a much bigger task, than trying to create a "visualizer"

The try-at-home and move-to-production-later starter.

These are science data dumps. There's no particular "try-before-you-buy" in large dataset dumps. I can explain what's in there, but ... (a) its complex and (b) it makes most people yawn. Like all large datasets, its all just ones and zeros and is unrewarding without putting in elbow grease.

[Ontopic]

I'm playing the devil's advocate here, I believe I understand the complexity that brings OpenCog is part of it intrinsic value. Yet, there's a lot to be exposed to onboard a larger crowd, in which regard I feel OpenCog may be failing compared to what it has to offer compared to inferior system, that did focus on presentation more than on content. Feel it's worth a shot trying to blance things out!

[linas]

I expect an npm install

Well, there was a debian maintainer who had built all the packages, and so you could say apt get and everything would install. But these haven't maintained...

npm run all-the-things with 100's of examples

Have you ever played with a database? You download it, and install it, and ... then what? Where's the data? Well, databases don't come with any, and that makes them rather wonky: you have to know why you need one and what you plan to do with it. The AtomSpace is a kind-of database.

In the consumer market "knowledge-bases" are a hot topic though.

They're "hot" mostly because there are dozens of small and medium corporations with marketing departments trying to sell you services that can only be solved with whatever knowledge-base they're peddling. Cough up enough $$$ and it will solve your problems.

The core issue here is the AtomSpace doesn't have a startup, a corporation with a marketing department, doing outreach to cloud developers. So there's no "buzz" -- no one is talking about it over lunch, and saying things like "I use it at work too, my boss said that ..." There are demos, but there's no "visual bling" -- without a visualizer, without some snazzy app that runs on your cellphone, its all very ... underwhelming.

And the stuff that is well-maintained and polished, it's got a steep learning curve that scares everyone off after a week or two.

[Ontopic]

I disagree with the knowledge-base building being a corporate inspired effort, it's mainly (semi) open-source based, heavily taking inspiration from the unix-philosophies, just including a broader public. Again, as said, I feel a lot of opportunity to include Atomspace as a graph on top, through the assistance of tooling and other "AI's".

That there is no marketing department and buzz is what I'm trying to solve, or at least somewhat bridge. No corporate plans or dreams, just trying to get a bootstrap going for the "web generation". With some integration into the open source tools, I'm sure some new souls will get inspired as well and contribute to further exposing things (which will still of course be as deeply complex as it always was, just more "guidance" along the way in the form of interactive tooling to assist understanding and lowering the barrier to entry).

OpenCog's honesty and fairness in science is part of the reason why I think it's so suitable, so I appreciate your defense of its values! Despite your hesitant reactions (although very informative and helpful), I'll continue to foolishly see if I can get something going to convince you and the rest of the OpenCog contributors to embrace a more "public-facing" effort, no startup marketing-nonsense necessary, just a little bit of how the young kids do open-source these days ;)

[linas]

Thank you! My apologies if I sometimes sound cranky; it's not you, I wasn't born naturally charming, I have to work at it.

The broader task you outline requires hard thinking and discussion. The biggest stumbling block might be how the atomspace stores data. Due to historical reasons, it has a specific format, and that format must surely feel quiet alien to most users who are simply looking for a way to store and access their data.

Now, other styles can be supported. I created a proof-of-concept for storing generic s-expressions, but that elicited no interest. I created stubs for doing the same with arbitrary javascript, and arbitrary python. So, like a JSON database, or a PySON database. Interested users would need to show up. Chicken and egg.

So that's one issue. I can broaden it slightly -- what data do they want to store? What do they want to do with it? Why? If that became clear to me, I could propose solutions, but otherwise, its a vacuum.

Visualization poses similar challenges. Simply drawing lines between nearest neighbors doesn't accomplish much, if the actual data model has some complex structure. But that structure depends on the data model. For example: to store python, it would be converted to an "abstract syntax tree" (AST, look it up) and that AST would be stored in the atomspace. A proper visualizer would display either the AST, or the reconstructed python snippet. Simply drawing nearest neighbors would just ... look messy.

And if you want to interface the atomspace to other systems, e.g. deep-learning neural nets, or whatever .. that leads to more technical discussions. It would have to start with "what the hek are you trying to do?" and "what is the best way of doing that?", and whether or not that involves the atomspace is ... well it would be a highly techical discusin.

[mjsduncan]

there is a huge amount of biology data available, i can help you set up an example "bioatomspace" next week if you're interested. in the mean time you can checkout: https://github.com/opencog/agi-bio https://github.com/MOZI-AI/knowledge-import

this has an example of opencog with a bioatomspace running in docker but i suspect it has been broken by linus' (much needed and appreciated) recent refactorization of the opencog repos https://github.com/MOZI-AI/annotation-service

mike d

On Sun, Jun 12, 2022 at 10:14 PM Linas Vepštas @.***> wrote:

Thank you! My apologies if I sometimes sound cranky; it's not you, I wasn't born naturally charming, I have to work at it.

The broader task you outline requires hard thinking and discussion. The biggest stumbling block might be how the atomspace stores data. Due to historical reasons, it has a specific format, and that format must surely feel quiet alien to most users who are simply looking for a way to store and access their data.

Now, other styles can be supported. I created a proof-of-concept for storing generic s-expressions, but that elicited no interest. I created stubs for doing the same with arbitrary javascript, and arbitrary python. So, like a JSON database, or a PySON database. Interested users would need to show up. Chicken and egg.

So that's one issue. I can broaden it slightly -- what data do they want to store? What do they want to do with it? Why? If that became clear to me, I could propose solutions, but otherwise, its a vacuum.

Visualization poses similar challenges. Simply drawing lines between nearest neighbors doesn't accomplish much, if the actual data model has some complex structure. But that structure depends on the data model. For example: to store python, it would be converted to an "abstract syntax tree" (AST, look it up) and that AST would be stored in the atomspace. A proper visualizer would display either the AST, or the reconstructed python snippet. Simply drawing nearest neighbors would just ... look messy.

And if you want to interface the atomspace to other systems, e.g. deep-learning neural nets, or whatever .. that leads to more technical discussions. It would have to start with "what the hek are you trying to do?" and "what is the best way of doing that?", and whether or not that involves the atomspace is ... well it would be a highly techical discusin.

— Reply to this email directly, view it on GitHub https://github.com/opencog/atomspace-explorer/issues/8#issuecomment-1153386756, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGJSQQKIWIW5WHW6GLEC6TVO2KQFANCNFSM5WF3FZBQ . You are receiving this because you commented.Message ID: @.***>

[linas]

suspect it has been broken by linus' (much needed and appreciated) recent refactorization

None of the AGI-bio code should have broken. It should continue to work fine. None of the main opencog repos get broken by these changes; the changes are backwards-compatible. How do I know this? Unit tests. The unit tests pass. (Note the corollary: if there's something vital, but its not in a unit test, then ... sure, there's some random stochastic chance it might break. Unit tests are important.)

[linas]

Hey @mjsduncan

i can help you set up an example "bioatomspace" next week if you're interested.

I keep thinking that it could be useful to set up a publicly-accessible server, serving data. I've set one up for a subset of the language data, anyone on the net can reach it; but I don't announce how to access it because I don't want to deal with the hackers and crackers and ddos attackers. Sysadmin is a pain in the neck. But if we want to grow usage, then setting up public data servers seems like a needed step. If we could set up a data server, and then also have a matching web-browser viewer for it, say, like the other major data sites, that would be even better.

[Ontopic]

Thank you! My apologies if I sometimes sound cranky; it's not you, I wasn't born naturally charming, I have to work at it.

No work needed at all. As said, I appreciate your down-to-earth attitude and realistic (insightful) replies. I didn't come here to get love ;)

I do love the concepts and structures behind Atomspace though. With modern dev-ops and browser features a lot can be achieved nowadays. I'll be busy processing all of this and hopefully turning it into a basic Docker setup, that could be easily used to start a (public or private) self-hosted Atomspace. After that I would be interested in seeing what an OpenCog-veteran and script-kiddie can whip up together. If you'd like to discuss it I'll drop you an e-mail.

[Ontopic]

The nice thing about all these writers and students moving their efforts to knowledge bases, is that they will actually start off with some actual personal edges that could be transpiled into Atomspace quite easily, and then work on them incrementally. The added value to people that tend to their "knowledge gardens" will be clear, even if the flashy visualisation is just there for show, these people are actually motivated to get the most value out of their efforts to structure their knowledge/studies. Atomspace feels like a perfect fit to me.

Secretly these writers are being tricked into coding already ;) S-expressions are one of the easier concepts to actually grasp as a non-developer I feel. Integrating them into a intuitive frontend editor with some helpers would be a feasible option to ease people into it.

[Ontopic]

I think the whole OpenCog team is so much ahead of the curve, that you developed tools way before there were any users (apart from lack of "promotional activities"). You all basically figured things out a decade ago. Right now though might be a good time to dust things off a little, those early efforts — despite lack of usage so far — are certainly not wasted.

[Ontopic]

I could, without any guarantees for uptime or continuation, supply a server with 300gb ram for a trial

[linas]

Docker

If you feel you have to have docker, then this dockerfile should work. I think. It worked half-a-year ago:

https://github.com/opencog/docker/blob/master/opencog/core/Dockerfile

[linas]

I could, without any guarantees for uptime or continuation, supply a server

well, as mentioned, I have a server running with language data on it, and if you send me ssh public keys, I can tell you how to connect to it. That way, you can start using it instantly, no muss, no fuss, no install .. it "just works".

I'd like to set up infrastructure for something like that, more broadly. So, sure, maybe with docker images. it would make it easier for newcomers to just .. try it out. ready-to-go sandboxes. Some empty ones with just hello-world in them. Others with agi-bio data in them. I think for @mjsduncan this would be ideal: the server would just always be there, would just stay up and running, and new devs could just use it.

[edyirdaw]

@Ontopic, really 300GB RAM? How many cpu cores does it have? It would be good to do something cool with it 😁

[mjsduncan]

@linus i'm refering specifically to the docker in abdu et al's snet agent that uses opencog on the back end to do queries of a bioatomspace. he made a custom rpc server which is where i'm guessing the breakages might occur: https://github.com/Habush/atomspace-rpc

the scheme bioscience module in the bio-ai repo that adds the custom GeneNode and MoleculeNode types worked the last time i tried it a couple weeks ago.

On Mon, Jun 13, 2022 at 2:00 PM Linas Vepštas @.***> wrote:

suspect it has been broken by linus' (much needed and appreciated) recent refactorization

None of the AGI-bio code should have broken. It should continue to work fine. None of the main opencog repos get broken by these changes; the changes are backwards-compatible. How do I know this? Unit tests. The unit tests pass. (Note the corollary: if there's something vital, but its not in a unit test, then ... sure, there's some random stochastic chance it might break. Unit tests are important.)

— Reply to this email directly, view it on GitHub https://github.com/opencog/atomspace-explorer/issues/8#issuecomment-1154217836, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGJSQSVX45IL2AGM6POUZDVO5ZL3ANCNFSM5WF3FZBQ . You are receiving this because you commented.Message ID: @.***>

[mjsduncan]

you have to register, but you can try out a free demo of the gene annotation service (refered to above) on singularitynet to see what a domain specific atomspace based knowledge service can look like: https://beta.singularitynet.io/servicedetails/org/mozi/service/gene-annotation-service

On Tue, Jun 14, 2022 at 12:56 PM Michael Duncan @.***> wrote:

@linus i'm refering specifically to the docker in abdu et al's snet agent that uses opencog on the back end to do queries of a bioatomspace. he made a custom rpc server which is where i'm guessing the breakages might occur: https://github.com/Habush/atomspace-rpc

the scheme bioscience module in the bio-ai repo that adds the custom GeneNode and MoleculeNode types worked the last time i tried it a couple weeks ago.

On Mon, Jun 13, 2022 at 2:00 PM Linas Vepštas @.***> wrote:

suspect it has been broken by linus' (much needed and appreciated) recent refactorization

None of the AGI-bio code should have broken. It should continue to work fine. None of the main opencog repos get broken by these changes; the changes are backwards-compatible. How do I know this? Unit tests. The unit tests pass. (Note the corollary: if there's something vital, but its not in a unit test, then ... sure, there's some random stochastic chance it might break. Unit tests are important.)

— Reply to this email directly, view it on GitHub https://github.com/opencog/atomspace-explorer/issues/8#issuecomment-1154217836, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGJSQSVX45IL2AGM6POUZDVO5ZL3ANCNFSM5WF3FZBQ . You are receiving this because you commented.Message ID: @.***>

[Ontopic]

92 cores. Let's just say I have access to research equipment.

I'll e-mail you, linas, to see if we can do a trial run for a (semi-)open server to allow for some experiments in exploring all of this.

[Ontopic]

I would make sure that everything that does happen on the server, will be properly stored and made available (within reason) somewhere for the respective data-owners. I can, however, not guarantee it will always be available. I hope to supply a long-term (free) service though, but for now let's just assume it could be nice setup that later others could also quickly deploy as well, allowing them to host their own personal or public server quickly. Perhaps a central place somewhere in the OpenCog documentation to list the currently available open-servers would then be a nice place for somewhat tech-savvy users to get started a little quicker in the future (perhaps later on moving to self-hosting, but at least not requiring first diving into the whole backend setup beforehand)

[chris-altamimi]

If we're containerizing things ( lxc, docker, or other ) why wouldn't we use something more container-orchestration friendly like kubernetes? We can still install it on @Ontopic's machine if desirable, but for scalability and openness it would be better option imo for an "open server". If we did so on a cloud provider, of which im happy to cover costs, it could always be available and not require someone to manually maintain it. Provided that the cloud provider isnt experiencing downtime.

This would allow us to open up on-demand containers so people who want to just "get started" can do so at a click of a button in a clean environment. We could even spin up some of the atomspace examples on-demand, or better yet, some of the more complex datasets as mentioned.

I have no problem hosting this kubernetes cluster and giving anyone / everyone access to it.

In fact we can do this using IaC ( terraform ) in a repository so anyone who wants to make changes to the infrastructure / servers can do so with a simple git commit. It's how the kids are doing infrastructure these days as @Ontopic would say 😄

I'll give write permission to https://github.com/opencog/atomspace-js to anyone who wants to create javascript wrappers for this thing. ... or anything else.

I've started to work on a small/local visualizer diagram based editor proof of concept. I have a decent amount of code atm So @linas could I possibly get access to that repo so I can put up a PR for the atomspace-specific js code soon?

[Ontopic]

I would be open to explore different ways to setup containerisation. Assume a clean Ubuntu 20 setup, with a Wireguard, Ansible and Traefik

I'm not sure what possibilities there are for Atomspace to utilize lightweight solutions like; https://firecracker-microvm.github.io/ https://github.com/nanovms/ops

Kubernetes always feels like a heavy solution for serious business infrastructures, less for allowing userland facing applications to quickly start a container to play with, but I may be wrong. More than happy to discuss the options we have though. For now I'll focus on just getting linas' setup to a working point and take it from there.

[Ontopic]

I've started to work on a small/local visualizer diagram based editor proof of concept. I have a decent amount of code atm So @linas could I possibly get access to that repo so I can put up a PR for the atomspace-specific js code soon?

I would love to see your efforts! If you get a chance to put it online as a fork, in preparation for the PR, I'd love to have a link.

[Ontopic]

The true benefactor behind this (not sure if they'd wanna be mentioned at this point, so refraining to) would I feel be open to guaranteeing more long-term solutions, certainly when we present them an initial setup that shows the possibilities. That is why I say; hope to assist in supplying this long-term, but that's not up to me. Work on open-sourcing the setup needed for anyone to fire up an open-server, will no-matter what, be a worthwhile step to take for Atomspace's adoptation.

[linas]

currently available open-servers

I have now been convinced that publishing Docker containers is the right way to do things. That way, you can just start a container, and "everything works". Unfortunately, the containers are bit-rotted, I'm trying to update them now. See https://github.com/opencog/docker/tree/master/opencog

I know nothing about kuberbnetes or terraform ... or about Wireguard, Ansible, Traefik ... or firecracker or nanovms. So let me describe my standard workflow, and see how it could be adapted to above technologies.

• I have a "processing pipeline" which consists of:

  • large directories of assorted data
  • a collection of bash shell scripts to start various servers and scripts and feed data around.
  • a tmux terminal of like 6 or 8 or 12 windows, to keep an eye on things.
  • At the center of it all, a running instance of the opencog atomspace, in which all the heavy data-mangling is done.

• That running instance of the atomspace includes the cogserver which exposes a tcpip network telnet shell, which provides telnet interfaces for guile, python and now, a custom psuedo-json subset.

  • I open 2-4 different (network telnet) connections into the atomspace, from the cogserver, so I can watch, poke, run-by-hand, general mainenance, etc.
  • The python and scheme command-lines (network-telnet) exposed by the cogsever are 99.9% compatiable with the ordinary python, guile shells, except, for example, blank lines are handled differently in python, minor-but-annoying things like that.
  • There is ZERO security: if you have access to the cogserver tcp-ip port, you can do anything to any data inside the atomspace. No password, no login, no authorization checks.
  • This last remark is the most troublesome for exposing an open cogserver port on the wide-open internet: anyone can telnet in, and erase all the loaded data. Anyone can come in, and store terabytes of data, exhausting all RAM and disk space. And since the user is working inside the server, maybe there's a way of crashing the server, buffer overflow, standard-hacks-to-get-root type exploits.

How would wiregaurd, etc. help with that?

[linas]

Hi @chris-altamimi

Could I possibly get access to that repo so I can put up a PR soon?

It's public. Anyone can create a pull req: https://github.com/opencog/atomspace-js

For just right now, either create a pull req, or otherwise show me what you're working on, cause its not clear to me if what you're doing is what's described in the README there. If it's that, I'll merge it ... If it's something completely-different, I can create an https://github.com/opencog/foobar repo and give you full-read-write authority, the hardest part is to pick a good name foobar.